[Mpi3-hybridpm] [EXTERNAL] Re: Threading homeworking / next telecon

Mon Mar 25 14:53:50 CDT 2013

That doesn't do much for me in terms of enabling greater concurrency
in performance-critical operations.

I'd like to propose that we try to make all of "Access Only", "Update
RefCount", "Read of List" and "None" thread safe in all cases.  All of
these are read-only except for "Update RefCount", but this can be done
with atomics.  I am assuming that concurrent reads are only permitted
to happen after the writing calls on the object have completed.  This
is the essence of MPI_THREAD_PER_OBJECT.

Jeff

On Mon, Mar 25, 2013 at 2:48 PM, William Gropp <wgropp at illinois.edu> wrote:
> I agree - one way to answer the original question is to require that all of
> the routines with an "X" in the None column for
> http://wiki.mpich.org/mpich/index.php/Thread_Safety (with any corrections,
> and extended to MPI-3) be safely callable in any thread mode.
>
> Bill
>
> William Gropp
> Director, Parallel Computing Institute
> Deputy Director for Research
> Institute for Advanced Computing Applications and Technologies
> Thomas M. Siebel Chair in Computer Science
> University of Illinois Urbana-Champaign
>
>
>
>
> On Mar 25, 2013, at 2:27 PM, Barrett, Brian W wrote:
>
> I generally agree with Bill, although I think that Jeff's original proposal
> did a better job at that than the discussion which followed.  That is, I
> think you could talk about state changing vs. non-state changing calls and
> perhaps localize which states change per-call and end up with a reasonable
> result.  This might be worth doing anyway, in terms of Jeff H's original
> problem, where he needs to query the thread level to determine if he can
> query the thread level.  Since that's clearly not a state changing
> operation, we could perhaps redefine Thread_single in a way that works for
> Jeff H (but probably not Jeff S)?
>
> Brian
>
> On 3/25/13 1:08 PM, "William Gropp" <wgropp at illinois.edu> wrote:
>
> We need to be careful here.  There is nothing about MPI_THREAD_MULTIPLE that
> requires a global lock or even locks at all.  There is great freedom in the
> implementation, and it is also important to note that the MPI implementation
> is not required to enforce any thread-related restrictions; i.e., it is not
> required to detect or correct for erroneous programs.  It can be interesting
> to consider *semantics* that might offer an implementation additional
> opportunities (e.g., single reader/single writer).  Whatever we do, we
> should not be referring to global locks, as that is not an MPI concept.
>
> Bill
>
> William Gropp
> Director, Parallel Computing Institute
> Deputy Director for Research
> Institute for Advanced Computing Applications and Technologies
> Thomas M. Siebel Chair in Computer Science
> University of Illinois Urbana-Champaign
>
>
>
>
> On Mar 25, 2013, at 12:57 PM, Jeff Hammond wrote:
>
> but I realized that it also addresses one of the primary challenges
>
> to upping the requirement of thread-support in the standard, since
>
> MPI_THREAD_PER_OBJECT is one possible way to achieve
>
> MPI_THREAD_MULTIPLE-like behavior without requiring locks in
>
> performance-critical functions.
>
>
> Yes, it could be. I would be happy if this indeed solved two problems. I'd
> be happy to assist you in fleshing out this proposal. Let me know how I can
> help.
>
>
> My primary concerns are:
>
> - To what extent is per-object locking going to lead to better
> performance on the pertinent networks?  I know how MPICH+PAMI works,
> but it is critical to get feedback from e.g. Intel, Cray, etc. about
> the possible problems that may exist.  If you could drive the
> discussion from the Infiniband perspective, that would be great.
>
> - While MPI_Comm, MPI_Win and MPI_File are the most obvious examples,
> at least to me, where per-object locking can be used to achieve
> concurrency, I want to be sure that this makes sense.  Are there
> aspects of the MPI standard that have nonlocal effects and therefore
> corrupt the clean separation of state implied by
> MPI_THREAD_PER_OBJECT?  If so, does this actually matter?  What I mean
> is that, if a function isn't performance critical, it seems
> sufficient, at least to me, to specify global-lock (global within the
> process, of course) semantics.
>
> - When functions take multiple objects as arguments, what are the
> concurrency rules?  When I imagined this idea, it seems
> straightforward to me, but I certainly did not consider all ~500 MPI
> functions, particularly in parts of the standard that I don't
> currently use.
>
> - Is per-object locking actually the right model?  Maybe it is better
> to talk in terms of a single-writer, multiple-reader requirement, i.e.
> that the per-object serialization is only required when the object
> state will be changed.  MPI_Comm is the most complicated situation
> since I am assuming that message-queues will be per-communicator, and
> thus any call that takes an MPI_Comm argument and drives progress will
> potentially modify that communicator.   We need to make the semantics
> of this very clear.  On the other hand, I think that MPI_Info,
> MPI_Datatype, MPI_Group, and possibly other objects are pretty clear
> regarding the need for mutual exclusion only when the object state is
> being changed.
>
> Best,
>
> Jeff
>
>
> -----Original Message-----
>
> From: mpi3-hybridpm-bounces at lists.mpi-forum.org
>
> [mailto:mpi3-hybridpm- bounces at lists.mpi-forum.org] On Behalf Of Jeff
>
> Hammond
>
> Sent: Monday, March 25, 2013 10:08 AM
>
> To: mpi3-hybridpm at lists.mpi-forum.org
>
> Subject: Re: [Mpi3-hybridpm] Threading homeworking / next telecon
>
>
> I finally wrote up another ticket that was my attempt at a more
>
> holistic proposal for thread-safety.  It's a bit raw, so please try
>
> to extract the essence of it before poking at specific flaws.
>
>
> https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/373
>
>
> Jeff
>
>
> --
> Jeff Hammond
> Argonne Leadership Computing Facility
> University of Chicago Computation Institute
> jhammond at alcf.anl.gov / (630) 252-5381
> http://www.linkedin.com/in/jeffhammond
> https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond
> _______________________________________________
> Mpi3-hybridpm mailing list
> Mpi3-hybridpm at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm
>
>
>
>
> --
>   Brian W. Barrett
>   Scalable System Software Group
>   Sandia National Laboratories
> _______________________________________________
> Mpi3-hybridpm mailing list
> Mpi3-hybridpm at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm
>
>
>
> _______________________________________________
> Mpi3-hybridpm mailing list
> Mpi3-hybridpm at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm

-- 
Jeff Hammond
Argonne Leadership Computing Facility
University of Chicago Computation Institute
jhammond at alcf.anl.gov / (630) 252-5381
http://www.linkedin.com/in/jeffhammond
https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond