[Mpi3-hybridpm] [EXTERNAL] Re: Threading homeworking / next telecon

[Jeff Hammond]

Why can't a user do MPI_COMM_SET_INFO explicitly every time they want
per-communicator semantics?

Jeff

On Mon, Mar 25, 2013 at 7:30 PM, Pavan Balaji <balaji at mcs.anl.gov> wrote:
>
> FWIW, we discussed a similar in the hybrid WG a few meetings ago.  The
> main reason why we didn't go down that path was because per-communicator
> semantics are not fully inherited for child communicators.  For example,
> split does not inherit info arguments or communicator attributes, while
> dup does.
>
>  -- Pavan
>
> On 03/25/2013 05:31 PM US Central Time, Sur, Sayantan wrote:
>> This is interesting. It might be useful for implementers if the app
>> could inform the MPI library that in its usage model, per-communicator
>> queues might lead to a performance benefit. Such as in the case of many
>> threads (among others).
>>
>>
>>
>> Info key? Assert?
>>
>>
>>
>> Sayantan
>>
>>
>>
>> *From:*mpi3-hybridpm-bounces at lists.mpi-forum.org
>> [mailto:mpi3-hybridpm-bounces at lists.mpi-forum.org] *On Behalf Of
>> *William Gropp
>> *Sent:* Monday, March 25, 2013 2:24 PM
>> *To:* mpi3-hybridpm at lists.mpi-forum.org
>> *Subject:* Re: [Mpi3-hybridpm] [EXTERNAL] Re: Threading homeworking /
>> next telecon
>>
>>
>>
>> An implementation is free to use separate queues for each communicator;
>> some of us have discussed this in the past, in part to permit use of
>> lock-free structures for the queue updates, particularly as this is the
>> only place there are no wild cards, ever.  I believe that this is within
>> the existing semantics.  It even has benefits for single threaded
>> execution, since the communicator matching is done once, rather than in
>> every query on the queue.
>>
>>
>>
>> In terms of progress, the standard is deliberately vague on the details,
>> and thus I don't believe we have the requirement that you quote.  And
>> some of the other interpretations of progress would not be helped by any
>> thread-safety restriction.
>>
>>
>>
>> Bill
>>
>>
>>
>> William Gropp
>>
>> Director, Parallel Computing Institute
>>
>> Deputy Director for Research
>>
>> Institute for Advanced Computing Applications and Technologies
>>
>> Thomas M. Siebel Chair in Computer Science
>>
>> University of Illinois Urbana-Champaign
>>
>>
>>
>>
>>
>>
>>
>> On Mar 25, 2013, at 4:15 PM, Jeff Hammond wrote:
>>
>>
>>
>> On Mon, Mar 25, 2013 at 3:17 PM, William Gropp <wgropp at illinois.edu
>> <mailto:wgropp at illinois.edu>> wrote:
>>
>> I was only addressing the issue of calling the thread level routines before
>>
>>     knowing what thread level you had.
>>
>>
>> Okay, sorry, I cannot tell which tickets people are referring to since
>> I have a bunch of different ones right now.
>>
>>
>> I'm not sure what you are looking for.  In the case of MPI_THREAD_MULTIPLE,
>>
>>     an implementation can provide significant concurrency today without any
>>
>>     change in the MPI standard - that's a major reason for that table
>>     (more to
>>
>>     the point - this table is meant as a guide for not using locks).
>>      Can you
>>
>>     give me an example of something that the current MPI semantics prohibits
>>
>>     that you'd like to achieve with MPI_THREAD_PER_OBJECT?
>>
>>
>> It is my understanding of the progress requirements that any call to
>> MPI must make progress on all MPI operations.  This means that two
>> threads calling e.g. MPI_Recv must walk all of the message queues.  If
>> a thread needs to modify any queue because it matches, then this must
>> be done in a thread-safe way, which presumably requires something
>> resembling mutual exclusion or transactions.  If a call to MPI_Recv
>> only had to make progress on its own communicator, then two threads
>> calling MPI_Recv on two different communicators would (1) only have to
>> walk the message queue associated with that communicator and (2)
>> nothing resembling mutual exclusion is required for the thread to
>> update the message queue in the event that matching occurs.
>>
>> Forgive me if I've got some of the details wrong.  If I've got all of
>> the details and the big picture wrong, then I'll think about it more.
>>
>> Jeff
>>
>>
>> On Mar 25, 2013, at 2:53 PM, Jeff Hammond wrote:
>>
>>
>>
>>     That doesn't do much for me in terms of enabling greater concurrency
>>
>>     in performance-critical operations.
>>
>>
>>
>>     I'd like to propose that we try to make all of "Access Only", "Update
>>
>>     RefCount", "Read of List" and "None" thread safe in all cases.  All of
>>
>>     these are read-only except for "Update RefCount", but this can be done
>>
>>     with atomics.  I am assuming that concurrent reads are only permitted
>>
>>     to happen after the writing calls on the object have completed.  This
>>
>>     is the essence of MPI_THREAD_PER_OBJECT.
>>
>>
>>
>>     Jeff
>>
>>
>>
>>
>>
>>
>>
>>     _______________________________________________
>>
>>     Mpi3-hybridpm mailing list
>>
>>     Mpi3-hybridpm at lists.mpi-forum.org
>>     <mailto:Mpi3-hybridpm at lists.mpi-forum.org>
>>
>>     http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm
>>
>>
>>
>>
>> --
>> Jeff Hammond
>> Argonne Leadership Computing Facility
>> University of Chicago Computation Institute
>> jhammond at alcf.anl.gov <mailto:jhammond at alcf.anl.gov> / (630) 252-5381
>> http://www.linkedin.com/in/jeffhammond
>> https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond
>> _______________________________________________
>> Mpi3-hybridpm mailing list
>> Mpi3-hybridpm at lists.mpi-forum.org <mailto:Mpi3-hybridpm at lists.mpi-forum.org>
>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm
>>
>>
>>
>>
>>
>> _______________________________________________
>> Mpi3-hybridpm mailing list
>> Mpi3-hybridpm at lists.mpi-forum.org
>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm
>>
>
> --
> Pavan Balaji
> http://www.mcs.anl.gov/~balaji
> _______________________________________________
> Mpi3-hybridpm mailing list
> Mpi3-hybridpm at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm



-- 
Jeff Hammond
Argonne Leadership Computing Facility
University of Chicago Computation Institute
jhammond at alcf.anl.gov / (630) 252-5381
http://www.linkedin.com/in/jeffhammond
https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond