[Mpi3-hybridpm] [EXTERNAL] Re: Threading homeworking / next telecon

Mon Mar 25 14:48:58 CDT 2013

I agree - one way to answer the original question is to require that all of the routines with an "X" in the None column for http://wiki.mpich.org/mpich/index.php/Thread_Safety (with any corrections, and extended to MPI-3) be safely callable in any thread mode.

Bill

William Gropp
Director, Parallel Computing Institute
Deputy Director for Research
Institute for Advanced Computing Applications and Technologies
Thomas M. Siebel Chair in Computer Science
University of Illinois Urbana-Champaign

On Mar 25, 2013, at 2:27 PM, Barrett, Brian W wrote:

> I generally agree with Bill, although I think that Jeff's original proposal did a better job at that than the discussion which followed.  That is, I think you could talk about state changing vs. non-state changing calls and perhaps localize which states change per-call and end up with a reasonable result.  This might be worth doing anyway, in terms of Jeff H's original problem, where he needs to query the thread level to determine if he can query the thread level.  Since that's clearly not a state changing operation, we could perhaps redefine Thread_single in a way that works for Jeff H (but probably not Jeff S)?
> 
> Brian
> 
> On 3/25/13 1:08 PM, "William Gropp" <wgropp at illinois.edu> wrote:
> 
>> We need to be careful here.  There is nothing about MPI_THREAD_MULTIPLE that requires a global lock or even locks at all.  There is great freedom in the implementation, and it is also important to note that the MPI implementation is not required to enforce any thread-related restrictions; i.e., it is not required to detect or correct for erroneous programs.  It can be interesting to consider *semantics* that might offer an implementation additional opportunities (e.g., single reader/single writer).  Whatever we do, we should not be referring to global locks, as that is not an MPI concept.
>> 
>> Bill
>> 
>> William Gropp
>> Director, Parallel Computing Institute
>> Deputy Director for Research
>> Institute for Advanced Computing Applications and Technologies
>> Thomas M. Siebel Chair in Computer Science
>> University of Illinois Urbana-Champaign
>> 
>> 
>> 
>> 
>> On Mar 25, 2013, at 12:57 PM, Jeff Hammond wrote:
>> 
>>>>> but I realized that it also addresses one of the primary challenges
>>>>> to upping the requirement of thread-support in the standard, since
>>>>> MPI_THREAD_PER_OBJECT is one possible way to achieve
>>>>> MPI_THREAD_MULTIPLE-like behavior without requiring locks in
>>>>> performance-critical functions.
>>>>> 
>>>> Yes, it could be. I would be happy if this indeed solved two problems. I'd be happy to assist you in fleshing out this proposal. Let me know how I can help.
>>> 
>>> My primary concerns are:
>>> 
>>> - To what extent is per-object locking going to lead to better
>>> performance on the pertinent networks?  I know how MPICH+PAMI works,
>>> but it is critical to get feedback from e.g. Intel, Cray, etc. about
>>> the possible problems that may exist.  If you could drive the
>>> discussion from the Infiniband perspective, that would be great.
>>> 
>>> - While MPI_Comm, MPI_Win and MPI_File are the most obvious examples,
>>> at least to me, where per-object locking can be used to achieve
>>> concurrency, I want to be sure that this makes sense.  Are there
>>> aspects of the MPI standard that have nonlocal effects and therefore
>>> corrupt the clean separation of state implied by
>>> MPI_THREAD_PER_OBJECT?  If so, does this actually matter?  What I mean
>>> is that, if a function isn't performance critical, it seems
>>> sufficient, at least to me, to specify global-lock (global within the
>>> process, of course) semantics.
>>> 
>>> - When functions take multiple objects as arguments, what are the
>>> concurrency rules?  When I imagined this idea, it seems
>>> straightforward to me, but I certainly did not consider all ~500 MPI
>>> functions, particularly in parts of the standard that I don't
>>> currently use.
>>> 
>>> - Is per-object locking actually the right model?  Maybe it is better
>>> to talk in terms of a single-writer, multiple-reader requirement, i.e.
>>> that the per-object serialization is only required when the object
>>> state will be changed.  MPI_Comm is the most complicated situation
>>> since I am assuming that message-queues will be per-communicator, and
>>> thus any call that takes an MPI_Comm argument and drives progress will
>>> potentially modify that communicator.   We need to make the semantics
>>> of this very clear.  On the other hand, I think that MPI_Info,
>>> MPI_Datatype, MPI_Group, and possibly other objects are pretty clear
>>> regarding the need for mutual exclusion only when the object state is
>>> being changed.
>>> 
>>> Best,
>>> 
>>> Jeff
>>> 
>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: mpi3-hybridpm-bounces at lists.mpi-forum.org
>>>>>>> [mailto:mpi3-hybridpm- bounces at lists.mpi-forum.org] On Behalf Of Jeff
>>>>>>> Hammond
>>>>>>> Sent: Monday, March 25, 2013 10:08 AM
>>>>>>> To: mpi3-hybridpm at lists.mpi-forum.org
>>>>>>> Subject: Re: [Mpi3-hybridpm] Threading homeworking / next telecon
>>>>>>> 
>>>>>>> I finally wrote up another ticket that was my attempt at a more
>>>>>>> holistic proposal for thread-safety.  It's a bit raw, so please try
>>>>>>> to extract the essence of it before poking at specific flaws.
>>>>>>> 
>>>>>>> https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/373
>>>>>>> 
>>>>>>> Jeff
>>> 
>>> -- 
>>> Jeff Hammond
>>> Argonne Leadership Computing Facility
>>> University of Chicago Computation Institute
>>> jhammond at alcf.anl.gov / (630) 252-5381
>>> http://www.linkedin.com/in/jeffhammond
>>> https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond
>>> _______________________________________________
>>> Mpi3-hybridpm mailing list
>>> Mpi3-hybridpm at lists.mpi-forum.org
>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm
>> 
> 
> 
> 
> --
>   Brian W. Barrett
>   Scalable System Software Group
>   Sandia National Laboratories
> _______________________________________________
> Mpi3-hybridpm mailing list
> Mpi3-hybridpm at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-hybridpm/attachments/20130325/930bbcea/attachment-0001.html>