[Mpi3-hybridpm] [EXTERNAL] Re: Threading homeworking / next telecon

Sur, Sayantan sayantan.sur at intel.com
Tue Mar 26 11:16:58 CDT 2013 

> The problem was that this wasn't allow us to create a group of
> communicators on which progress is made.  Each communicator was
> independent of everything else.
>

I am not clear why every UPC thread needs to make progress on other communicators belonging to other threads (slide 7 of the endpoint proposal). Each thread just needs to progress its own communications which are associated with its communicator.

Additionally, I have the same questions as Hammond. So, if you guys come to some conclusion in person as to why this doesn't work for the UPC case, please share the reasons with the WG.

Thanks,
Sayantan.
 
> However, our goal was allowing each "UPC thread" to be an MPI rank, where
> all threads share that rank.  Your goal is different, so this might or might not
> be a concern for you.
> 
>  -- Pavan
> 
> On 03/25/2013 07:39 PM US Central Time, Jeff Hammond wrote:
> > Why can't a user do MPI_COMM_SET_INFO explicitly every time they want
> > per-communicator semantics?
> >
> > Jeff
> >
> > On Mon, Mar 25, 2013 at 7:30 PM, Pavan Balaji <balaji at mcs.anl.gov>
> wrote:
> >>
> >> FWIW, we discussed a similar in the hybrid WG a few meetings ago.
> >> The main reason why we didn't go down that path was because
> >> per-communicator semantics are not fully inherited for child
> >> communicators.  For example, split does not inherit info arguments or
> >> communicator attributes, while dup does.
> >>
> >>  -- Pavan
> >>
> >> On 03/25/2013 05:31 PM US Central Time, Sur, Sayantan wrote:
> >>> This is interesting. It might be useful for implementers if the app
> >>> could inform the MPI library that in its usage model,
> >>> per-communicator queues might lead to a performance benefit. Such as
> >>> in the case of many threads (among others).
> >>>
> >>>
> >>>
> >>> Info key? Assert?
> >>>
> >>>
> >>>
> >>> Sayantan
> >>>
> >>>
> >>>
> >>> *From:*mpi3-hybridpm-bounces at lists.mpi-forum.org
> >>> [mailto:mpi3-hybridpm-bounces at lists.mpi-forum.org] *On Behalf Of
> >>> *William Gropp
> >>> *Sent:* Monday, March 25, 2013 2:24 PM
> >>> *To:* mpi3-hybridpm at lists.mpi-forum.org
> >>> *Subject:* Re: [Mpi3-hybridpm] [EXTERNAL] Re: Threading
> homeworking
> >>> / next telecon
> >>>
> >>>
> >>>
> >>> An implementation is free to use separate queues for each
> >>> communicator; some of us have discussed this in the past, in part to
> >>> permit use of lock-free structures for the queue updates,
> >>> particularly as this is the only place there are no wild cards,
> >>> ever.  I believe that this is within the existing semantics.  It
> >>> even has benefits for single threaded execution, since the
> >>> communicator matching is done once, rather than in every query on the
> queue.
> >>>
> >>>
> >>>
> >>> In terms of progress, the standard is deliberately vague on the
> >>> details, and thus I don't believe we have the requirement that you
> >>> quote.  And some of the other interpretations of progress would not
> >>> be helped by any thread-safety restriction.
> >>>
> >>>
> >>>
> >>> Bill
> >>>
> >>>
> >>>
> >>> William Gropp
> >>>
> >>> Director, Parallel Computing Institute
> >>>
> >>> Deputy Director for Research
> >>>
> >>> Institute for Advanced Computing Applications and Technologies
> >>>
> >>> Thomas M. Siebel Chair in Computer Science
> >>>
> >>> University of Illinois Urbana-Champaign
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Mar 25, 2013, at 4:15 PM, Jeff Hammond wrote:
> >>>
> >>>
> >>>
> >>> On Mon, Mar 25, 2013 at 3:17 PM, William Gropp <wgropp at illinois.edu
> >>> <mailto:wgropp at illinois.edu>> wrote:
> >>>
> >>> I was only addressing the issue of calling the thread level routines
> >>> before
> >>>
> >>>     knowing what thread level you had.
> >>>
> >>>
> >>> Okay, sorry, I cannot tell which tickets people are referring to
> >>> since I have a bunch of different ones right now.
> >>>
> >>>
> >>> I'm not sure what you are looking for.  In the case of
> >>> MPI_THREAD_MULTIPLE,
> >>>
> >>>     an implementation can provide significant concurrency today
> >>> without any
> >>>
> >>>     change in the MPI standard - that's a major reason for that table
> >>>     (more to
> >>>
> >>>     the point - this table is meant as a guide for not using locks).
> >>>      Can you
> >>>
> >>>     give me an example of something that the current MPI semantics
> >>> prohibits
> >>>
> >>>     that you'd like to achieve with MPI_THREAD_PER_OBJECT?
> >>>
> >>>
> >>> It is my understanding of the progress requirements that any call to
> >>> MPI must make progress on all MPI operations.  This means that two
> >>> threads calling e.g. MPI_Recv must walk all of the message queues.
> >>> If a thread needs to modify any queue because it matches, then this
> >>> must be done in a thread-safe way, which presumably requires
> >>> something resembling mutual exclusion or transactions.  If a call to
> >>> MPI_Recv only had to make progress on its own communicator, then
> two
> >>> threads calling MPI_Recv on two different communicators would (1)
> >>> only have to walk the message queue associated with that
> >>> communicator and (2) nothing resembling mutual exclusion is required
> >>> for the thread to update the message queue in the event that matching
> occurs.
> >>>
> >>> Forgive me if I've got some of the details wrong.  If I've got all
> >>> of the details and the big picture wrong, then I'll think about it more.
> >>>
> >>> Jeff
> >>>
> >>>
> >>> On Mar 25, 2013, at 2:53 PM, Jeff Hammond wrote:
> >>>
> >>>
> >>>
> >>>     That doesn't do much for me in terms of enabling greater
> >>> concurrency
> >>>
> >>>     in performance-critical operations.
> >>>
> >>>
> >>>
> >>>     I'd like to propose that we try to make all of "Access Only",
> >>> "Update
> >>>
> >>>     RefCount", "Read of List" and "None" thread safe in all cases.
> >>> All of
> >>>
> >>>     these are read-only except for "Update RefCount", but this can
> >>> be done
> >>>
> >>>     with atomics.  I am assuming that concurrent reads are only
> >>> permitted
> >>>
> >>>     to happen after the writing calls on the object have completed.
> >>> This
> >>>
> >>>     is the essence of MPI_THREAD_PER_OBJECT.
> >>>
> >>>
> >>>
> >>>     Jeff
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>     _______________________________________________
> >>>
> >>>     Mpi3-hybridpm mailing list
> >>>
> >>>     Mpi3-hybridpm at lists.mpi-forum.org
> >>>     <mailto:Mpi3-hybridpm at lists.mpi-forum.org>
> >>>
> >>>     http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>> Jeff Hammond
> >>> Argonne Leadership Computing Facility University of Chicago
> >>> Computation Institute jhammond at alcf.anl.gov
> >>> <mailto:jhammond at alcf.anl.gov> / (630) 252-5381
> >>> http://www.linkedin.com/in/jeffhammond
> >>> https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond
> >>> _______________________________________________
> >>> Mpi3-hybridpm mailing list
> >>> Mpi3-hybridpm at lists.mpi-forum.org
> >>> <mailto:Mpi3-hybridpm at lists.mpi-forum.org>
> >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Mpi3-hybridpm mailing list
> >>> Mpi3-hybridpm at lists.mpi-forum.org
> >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm
> >>>
> >>
> >> --
> >> Pavan Balaji
> >> http://www.mcs.anl.gov/~balaji
> >> _______________________________________________
> >> Mpi3-hybridpm mailing list
> >> Mpi3-hybridpm at lists.mpi-forum.org
> >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm
> >
> >
> >
> 
> --
> Pavan Balaji
> http://www.mcs.anl.gov/~balaji
> _______________________________________________
> Mpi3-hybridpm mailing list
> Mpi3-hybridpm at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm

More information about the mpiwg-hybridpm mailing list