[Mpi3-rma] Use cases for RMA

William Gropp wgropp at illinois.edu
Wed Mar 3 14:24:59 CST 2010


Thanks.  I've added some comments inline.

On Mar 3, 2010, at 1:23 PM, Manojkumar Krishnan wrote:

>
> Bill,
>
> Here are some of the MPI RMA requirements for Global Arrays (GA) and
> ARMCI. ARMCI is GA's runtime system.  GA/ARMCI exploits native
> network communication interfaces and system resources (such as shared
> memory) to achieve the best possible performance of the remote memory
> access/one-sided communication. GA/ARMCI relies *heavily* on optimized
> contiguous and non-contiguous RMA operations (get/put/acc).
>
> For GA/ARMCI and its applications, below are some specfic examples of
> operations that are hard to achieve in MPI-2 RMA.
>
> 1. Memory Allocation: (This might be an implementation issue) The  
> user or
> library implementors should be able to allocate memory (e.g. shared
> memory), and register with MPI. This is useful in case of Global
> Arrays/ARMCI, which use RMA across nodes and shared memory within  
> nodes.
> ARMCI allocates shared memory segment, and pins/registers with the  
> network.

MPI already provides MPI_Alloc_mem , though this is not necessarily  
shared memory usable by other processes.  But the MPI implementation  
could allocate this from a shared memory pool and perform one-sided  
operations using load-store.  Do you need something different?

>
> 2. Locks: Should be made optional to keep the RMA programming model
> simple. If the user doesnot require concurrency, then locks are
> unnecessary. Enforcing to use locks as default might introduce
> unnecessary bugs if not carefully programmed (esp. at the level of of
> extreme scale systems, it is hard to debug).
>

You need something to provide for local and remote completion.  Do you  
mean to have blocking RMA operations that (in MPI terms) essentially  
act as lock-RMA_operation-unlock?

> 3. Support for non-overlapping concurrent operations in a window.

MPI RMA already does this, in all of the active target modes, and in  
the passive target mode with the shared lock mode.  What do you need  
that isn't supported?

>
> 4. RMW Operation - Useful for implementing dynamic load balancing
> algorithms (e.g. task queues/work stealing, group/global counters,  
> etc).
>

Yes, this is a major gap.  Is fetch-and-increment enough, or do you  
need compare-and-swap, or a general RMI?

Bill

> The above are feature requirements rather than performance issues,  
> which
> are implementation specific.
>
> If you have any questions, I would be happy to explain the above in  
> detail.
>
> Thanks,
> -Manoj.
> ---------------------------------------------------------------
> Manojkumar Krishnan
> High Performance Computing Group
> Pacific Northwest National Laboratory
> Ph: (509) 372-4206   Fax: (509) 372-4720
> http://hpc.pnl.gov/people/manoj
> ---------------------------------------------------------------
>
> On Wed, 3 Mar 2010, William Gropp wrote:
>
>> I went through the mail that was sent out in response to our request
>> for use cases, and I must say it was underwhelming.  I've included a
>> short summary below; based on this, we aren't looking at the correct
>> needs.  I don't think that these are representative of *all* of the
>> needs of RMA, but without good use cases, I don't see how we can
>> justify any but the most limited extensions/changes to the current
>> design.  Please (a) let me know if I overlooked something and (b)  
>> send
>> me (and the list) additional use cases.  For example, we haven't
>> included any of the issues needed to implement PGAS languages, nor
>> have we addressed the existing SHMEM codes.  Do we simply say that a
>> high-quality implementation will permit interoperation with whatever
>> OpenSHMEM is?  And what do we do about the RMI issue that one of the
>> two use cases that we have raises?
>>
>> Basically, I received two detailed notes for the area of Quantum
>> Chemistry.  In brief:
>>
>> MPI-2 RMA is already adequate for most parts, as long as the
>> implementation makes progress (as it is required to) on passive
>> updates.  Requires get or accumulate; rarely requires put.
>> Dynamic load balancing (a related but separate issue) needs some sort
>> of RMW.  (Thanks to Jeff Hammond)
>>
>> More complex algorithms and data structures appear to require remote
>> method invocation (RMI).  Sparse tree updates provide one example.
>> (Thanks to Robert Harrison)
>>
>> Bill
>>
>>
>> William Gropp
>> Deputy Director for Research
>> Institute for Advanced Computing Applications and Technologies
>> Paul and Cynthia Saylor Professor of Computer Science
>> University of Illinois Urbana-Champaign
>>
>>
>>
>>
>> _______________________________________________
>> mpi3-rma mailing list
>> mpi3-rma at lists.mpi-forum.org
>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>
> _______________________________________________
> mpi3-rma mailing list
> mpi3-rma at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

William Gropp
Deputy Director for Research
Institute for Advanced Computing Applications and Technologies
Paul and Cynthia Saylor Professor of Computer Science
University of Illinois Urbana-Champaign







More information about the mpiwg-rma mailing list