[Mpi3-hybridpm] Threading homeworking / next telecon

[Jeff Hammond]

>> but I realized that it also addresses one of the primary challenges
>> to upping the requirement of thread-support in the standard, since
>> MPI_THREAD_PER_OBJECT is one possible way to achieve
>> MPI_THREAD_MULTIPLE-like behavior without requiring locks in
>> performance-critical functions.
>>
> Yes, it could be. I would be happy if this indeed solved two problems. I'd be happy to assist you in fleshing out this proposal. Let me know how I can help.

My primary concerns are:

- To what extent is per-object locking going to lead to better
performance on the pertinent networks?  I know how MPICH+PAMI works,
but it is critical to get feedback from e.g. Intel, Cray, etc. about
the possible problems that may exist.  If you could drive the
discussion from the Infiniband perspective, that would be great.

- While MPI_Comm, MPI_Win and MPI_File are the most obvious examples,
at least to me, where per-object locking can be used to achieve
concurrency, I want to be sure that this makes sense.  Are there
aspects of the MPI standard that have nonlocal effects and therefore
corrupt the clean separation of state implied by
MPI_THREAD_PER_OBJECT?  If so, does this actually matter?  What I mean
is that, if a function isn't performance critical, it seems
sufficient, at least to me, to specify global-lock (global within the
process, of course) semantics.

- When functions take multiple objects as arguments, what are the
concurrency rules?  When I imagined this idea, it seems
straightforward to me, but I certainly did not consider all ~500 MPI
functions, particularly in parts of the standard that I don't
currently use.

- Is per-object locking actually the right model?  Maybe it is better
to talk in terms of a single-writer, multiple-reader requirement, i.e.
that the per-object serialization is only required when the object
state will be changed.  MPI_Comm is the most complicated situation
since I am assuming that message-queues will be per-communicator, and
thus any call that takes an MPI_Comm argument and drives progress will
potentially modify that communicator.   We need to make the semantics
of this very clear.  On the other hand, I think that MPI_Info,
MPI_Datatype, MPI_Group, and possibly other objects are pretty clear
regarding the need for mutual exclusion only when the object state is
being changed.

Best,

Jeff


>> >> -----Original Message-----
>> >> From: mpi3-hybridpm-bounces at lists.mpi-forum.org
>> >> [mailto:mpi3-hybridpm- bounces at lists.mpi-forum.org] On Behalf Of Jeff
>> >> Hammond
>> >> Sent: Monday, March 25, 2013 10:08 AM
>> >> To: mpi3-hybridpm at lists.mpi-forum.org
>> >> Subject: Re: [Mpi3-hybridpm] Threading homeworking / next telecon
>> >>
>> >> I finally wrote up another ticket that was my attempt at a more
>> >> holistic proposal for thread-safety.  It's a bit raw, so please try
>> >> to extract the essence of it before poking at specific flaws.
>> >>
>> >> https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/373
>> >>
>> >> Jeff

-- 
Jeff Hammond
Argonne Leadership Computing Facility
University of Chicago Computation Institute
jhammond at alcf.anl.gov / (630) 252-5381
http://www.linkedin.com/in/jeffhammond
https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond