[Mpi3-hybridpm] [EXTERNAL] Re: Threading homeworking / next telecon

Jeff Hammond jhammond at alcf.anl.gov
Mon Mar 25 14:47:56 CDT 2013


Bill:

I agree that locking is not an MPI concept, at least in this context.
It might be sufficient to refine the semantics of progress on
communicators and concurrent operations on overlapping windows, as two
examples, to make it possible for implementations to go down the path
of per-object state in such a way as to make MPI_THREAD_MULTIPLE more
efficient.

Brian:

Can you more specific about "Jeff's original proposal"?  Due to the
multiplicity of Jeffs as well as tickets I've created on this topic, I
don't know to what you are referring here.

All:

I propose that people just call me Hammond from now on.  It will make
life much easier for everyone.  If the extra characters are a problem,
I accept Hamm as an acceptable abbreviation that requires nearly
identical keyboard effort as "Jeff".  If this catches on, I may dress
up as a bear and bring canned lager to a future WG meeting :-)

Jeff

On Mon, Mar 25, 2013 at 2:27 PM, Barrett, Brian W <bwbarre at sandia.gov> wrote:
> I generally agree with Bill, although I think that Jeff's original proposal
> did a better job at that than the discussion which followed.  That is, I
> think you could talk about state changing vs. non-state changing calls and
> perhaps localize which states change per-call and end up with a reasonable
> result.  This might be worth doing anyway, in terms of Jeff H's original
> problem, where he needs to query the thread level to determine if he can
> query the thread level.  Since that's clearly not a state changing
> operation, we could perhaps redefine Thread_single in a way that works for
> Jeff H (but probably not Jeff S)?
>
> Brian
>
> On 3/25/13 1:08 PM, "William Gropp" <wgropp at illinois.edu> wrote:
>
> We need to be careful here.  There is nothing about MPI_THREAD_MULTIPLE that
> requires a global lock or even locks at all.  There is great freedom in the
> implementation, and it is also important to note that the MPI implementation
> is not required to enforce any thread-related restrictions; i.e., it is not
> required to detect or correct for erroneous programs.  It can be interesting
> to consider *semantics* that might offer an implementation additional
> opportunities (e.g., single reader/single writer).  Whatever we do, we
> should not be referring to global locks, as that is not an MPI concept.
>
> Bill
>
> William Gropp
> Director, Parallel Computing Institute
> Deputy Director for Research
> Institute for Advanced Computing Applications and Technologies
> Thomas M. Siebel Chair in Computer Science
> University of Illinois Urbana-Champaign
>
>
>
>
> On Mar 25, 2013, at 12:57 PM, Jeff Hammond wrote:
>
> but I realized that it also addresses one of the primary challenges
>
> to upping the requirement of thread-support in the standard, since
>
> MPI_THREAD_PER_OBJECT is one possible way to achieve
>
> MPI_THREAD_MULTIPLE-like behavior without requiring locks in
>
> performance-critical functions.
>
>
> Yes, it could be. I would be happy if this indeed solved two problems. I'd
> be happy to assist you in fleshing out this proposal. Let me know how I can
> help.
>
>
> My primary concerns are:
>
> - To what extent is per-object locking going to lead to better
> performance on the pertinent networks?  I know how MPICH+PAMI works,
> but it is critical to get feedback from e.g. Intel, Cray, etc. about
> the possible problems that may exist.  If you could drive the
> discussion from the Infiniband perspective, that would be great.
>
> - While MPI_Comm, MPI_Win and MPI_File are the most obvious examples,
> at least to me, where per-object locking can be used to achieve
> concurrency, I want to be sure that this makes sense.  Are there
> aspects of the MPI standard that have nonlocal effects and therefore
> corrupt the clean separation of state implied by
> MPI_THREAD_PER_OBJECT?  If so, does this actually matter?  What I mean
> is that, if a function isn't performance critical, it seems
> sufficient, at least to me, to specify global-lock (global within the
> process, of course) semantics.
>
> - When functions take multiple objects as arguments, what are the
> concurrency rules?  When I imagined this idea, it seems
> straightforward to me, but I certainly did not consider all ~500 MPI
> functions, particularly in parts of the standard that I don't
> currently use.
>
> - Is per-object locking actually the right model?  Maybe it is better
> to talk in terms of a single-writer, multiple-reader requirement, i.e.
> that the per-object serialization is only required when the object
> state will be changed.  MPI_Comm is the most complicated situation
> since I am assuming that message-queues will be per-communicator, and
> thus any call that takes an MPI_Comm argument and drives progress will
> potentially modify that communicator.   We need to make the semantics
> of this very clear.  On the other hand, I think that MPI_Info,
> MPI_Datatype, MPI_Group, and possibly other objects are pretty clear
> regarding the need for mutual exclusion only when the object state is
> being changed.
>
> Best,
>
> Jeff
>
>
> -----Original Message-----
>
> From: mpi3-hybridpm-bounces at lists.mpi-forum.org
>
> [mailto:mpi3-hybridpm- bounces at lists.mpi-forum.org] On Behalf Of Jeff
>
> Hammond
>
> Sent: Monday, March 25, 2013 10:08 AM
>
> To: mpi3-hybridpm at lists.mpi-forum.org
>
> Subject: Re: [Mpi3-hybridpm] Threading homeworking / next telecon
>
>
> I finally wrote up another ticket that was my attempt at a more
>
> holistic proposal for thread-safety.  It's a bit raw, so please try
>
> to extract the essence of it before poking at specific flaws.
>
>
> https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/373
>
>
> Jeff
>
>
> --
> Jeff Hammond
> Argonne Leadership Computing Facility
> University of Chicago Computation Institute
> jhammond at alcf.anl.gov / (630) 252-5381
> http://www.linkedin.com/in/jeffhammond
> https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond
> _______________________________________________
> Mpi3-hybridpm mailing list
> Mpi3-hybridpm at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm
>
>
>
>
> --
>   Brian W. Barrett
>   Scalable System Software Group
>   Sandia National Laboratories
>
> _______________________________________________
> Mpi3-hybridpm mailing list
> Mpi3-hybridpm at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm



-- 
Jeff Hammond
Argonne Leadership Computing Facility
University of Chicago Computation Institute
jhammond at alcf.anl.gov / (630) 252-5381
http://www.linkedin.com/in/jeffhammond
https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond



More information about the mpiwg-hybridpm mailing list