[Mpi3-hybridpm] [EXTERNAL] Re: Threading homeworking / next telecon

[Barrett, Brian W]

I generally agree with Bill, although I think that Jeff's original proposal
did a better job at that than the discussion which followed.  That is, I
think you could talk about state changing vs. non-state changing calls and
perhaps localize which states change per-call and end up with a reasonable
result.  This might be worth doing anyway, in terms of Jeff H's original
problem, where he needs to query the thread level to determine if he can
query the thread level.  Since that's clearly not a state changing
operation, we could perhaps redefine Thread_single in a way that works for
Jeff H (but probably not Jeff S)?

Brian

On 3/25/13 1:08 PM, "William Gropp" <wgropp at illinois.edu> wrote:

> We need to be careful here.  There is nothing about MPI_THREAD_MULTIPLE that
> requires a global lock or even locks at all.  There is great freedom in the
> implementation, and it is also important to note that the MPI implementation
> is not required to enforce any thread-related restrictions; i.e., it is not
> required to detect or correct for erroneous programs.  It can be interesting
> to consider *semantics* that might offer an implementation additional
> opportunities (e.g., single reader/single writer).  Whatever we do, we should
> not be referring to global locks, as that is not an MPI concept.
> 
> Bill
> 
> William Gropp
> Director, Parallel Computing Institute
> Deputy Director for Research
> Institute for Advanced Computing Applications and Technologies
> Thomas M. Siebel Chair in Computer Science
> University of Illinois Urbana-Champaign
> 
> 
> 
> 
> On Mar 25, 2013, at 12:57 PM, Jeff Hammond wrote:
> 
>>>> but I realized that it also addresses one of the primary challenges
>>>> to upping the requirement of thread-support in the standard, since
>>>> MPI_THREAD_PER_OBJECT is one possible way to achieve
>>>> MPI_THREAD_MULTIPLE-like behavior without requiring locks in
>>>> performance-critical functions.
>>>> 
>>> Yes, it could be. I would be happy if this indeed solved two problems. I'd
>>> be happy to assist you in fleshing out this proposal. Let me know how I can
>>> help.
>> 
>> My primary concerns are:
>> 
>> - To what extent is per-object locking going to lead to better
>> performance on the pertinent networks?  I know how MPICH+PAMI works,
>> but it is critical to get feedback from e.g. Intel, Cray, etc. about
>> the possible problems that may exist.  If you could drive the
>> discussion from the Infiniband perspective, that would be great.
>> 
>> - While MPI_Comm, MPI_Win and MPI_File are the most obvious examples,
>> at least to me, where per-object locking can be used to achieve
>> concurrency, I want to be sure that this makes sense.  Are there
>> aspects of the MPI standard that have nonlocal effects and therefore
>> corrupt the clean separation of state implied by
>> MPI_THREAD_PER_OBJECT?  If so, does this actually matter?  What I mean
>> is that, if a function isn't performance critical, it seems
>> sufficient, at least to me, to specify global-lock (global within the
>> process, of course) semantics.
>> 
>> - When functions take multiple objects as arguments, what are the
>> concurrency rules?  When I imagined this idea, it seems
>> straightforward to me, but I certainly did not consider all ~500 MPI
>> functions, particularly in parts of the standard that I don't
>> currently use.
>> 
>> - Is per-object locking actually the right model?  Maybe it is better
>> to talk in terms of a single-writer, multiple-reader requirement, i.e.
>> that the per-object serialization is only required when the object
>> state will be changed.  MPI_Comm is the most complicated situation
>> since I am assuming that message-queues will be per-communicator, and
>> thus any call that takes an MPI_Comm argument and drives progress will
>> potentially modify that communicator.   We need to make the semantics
>> of this very clear.  On the other hand, I think that MPI_Info,
>> MPI_Datatype, MPI_Group, and possibly other objects are pretty clear
>> regarding the need for mutual exclusion only when the object state is
>> being changed.
>> 
>> Best,
>> 
>> Jeff
>> 
>> 
>>>>>> -----Original Message-----
>>>>>> From: mpi3-hybridpm-bounces at lists.mpi-forum.org
>>>>>> [mailto:mpi3-hybridpm- bounces at lists.mpi-forum.org] On Behalf Of Jeff
>>>>>> Hammond
>>>>>> Sent: Monday, March 25, 2013 10:08 AM
>>>>>> To: mpi3-hybridpm at lists.mpi-forum.org
>>>>>> Subject: Re: [Mpi3-hybridpm] Threading homeworking / next telecon
>>>>>> 
>>>>>> I finally wrote up another ticket that was my attempt at a more
>>>>>> holistic proposal for thread-safety.  It's a bit raw, so please try
>>>>>> to extract the essence of it before poking at specific flaws.
>>>>>> 
>>>>>> https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/373
>>>>>> 
>>>>>> Jeff
>> 
>> -- 
>> Jeff Hammond
>> Argonne Leadership Computing Facility
>> University of Chicago Computation Institute
>> jhammond at alcf.anl.gov / (630) 252-5381
>> http://www.linkedin.com/in/jeffhammond
>> https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond
>> _______________________________________________
>> Mpi3-hybridpm mailing list
>> Mpi3-hybridpm at lists.mpi-forum.org
>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm
> 


--
  Brian W. Barrett
  Scalable System Software Group
  Sandia National Laboratories


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-hybridpm/attachments/20130325/2957249b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 454 bytes
Desc: not available
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-hybridpm/attachments/20130325/2957249b/attachment-0001.bin>