[Mpi3-rma] mpi3-rma Digest, Vol 13, Issue 1

[Torsten Hoefler]

On Tue, Feb 02, 2010 at 08:07:18AM -0600, Quincey Koziol wrote:
> Hi all,
> 	I have two difficult problems in HDF5 that I need to solve, hopefully with some portion of new RMA-style work:
> 
> 	- HDF5 files have a single global resource: the space in the file
> 	itself.  I'm willing to concede that I may not be able to track
> 	individual pieces of free space in the file when operating in
> 	parallel, but I definitely need to be able to allocate new space at
> 	the end of the file in a "thread-safe" way, without setting aside any
> 	MPI process or thread to manage the "end of file" value.  I believe
> 	that an atomic fetch-and-increment would solve this problem, as long
> 	as it could increment by an arbitrary amount (not just '1') and
> 	didn't involve any other MPI processes besides the one requesting the
> 	space in the file.
that should work "out of the box" with MPI_Get_accumulate (or
MPI_Accumulate_get). You could even do this today with a somewhat more
complex protocol, i.e., Peterson's, Dekker's or Lamport's algorithms for
mutual exclusion, involving multiple round-trips :-(. You could also
implement AM semantics in top of P2P and Iprobes if you don't want an
additional thread.

> 	- HDF5 files also have multiple distributed (local, perhaps is a
> 	better word?) resources that need to be operated on in atomic ways
> 	(i.e. without involving other MPI processes).  For example, when an
> 	application wants to create a new object in an HDF5 file, the
> 	following operations need to occur (or something similar):
> 
> 	1*) <Allocate space for the object in the file>
Well, MPI will probably never have an "allocate memory" operation :-).
This seems to be a good use-case for user-defined accumulate operations,
or even better: active messages!

> 	2) <Write the information for the new object to the space allocated>
> 	3*) <Check if an object of the requested name exists in a group in
> 	the file, and if not, modify the group's data structure(s) to include
> 	a link to the new object>
Same as 1).

> 	Operations marked with a '*' need to happen atomically and without
> 	other processes being involved.  Step 3) seems analogous to a
> 	fetch-and-increment, in an abstract way, but I'm not certain how to
> 	solve it.  Would the "Remote Method Invocation" be able to take care
> 	of this sort of operation, which is essentially equivalent to
> 	inserting an object in a hash table?
Yes, for user-defined ops or AM's, you would probably need to do your
own locking with special locks (to insure message progression and avoid
deadlocks). I'd recommend to look at the GASNet documentation if you
really want to know why.

All the Best,
  Torsten

-- 
 bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ -----
Torsten Hoefler         | Research Associate
Blue Waters Directorate | University of Illinois
1205 W Clark Street     | Urbana, IL, 61801
NCSA Building           | +01 (217) 244-7736