[MPI3-IO] [EXTERNAL] New MPI-IO routines

Rob Latham robl at mcs.anl.gov
Thu May 24 16:01:12 CDT 2012

On Thu, May 24, 2012 at 03:53:04PM -0500, Mohamad Chaarawi wrote:
> Hi Dries,
> On 5/24/2012 3:28 PM, Dries Kimpe wrote:
> >Also, what (if anything) would prevent implementing this on top of MPI,
> >using existing the MPI_File_iread/iwrite routines?
> How would you do this efficiently and asynchronously?
> you need to set atomic mode to get sequential consistency (which is
> not pretty and I don't think it is well supported). Even with atomic
> mode set, how would you guarantee serial execution order?

Dries is asking how you could build a library on top of
MPI_File_iread/iwrite to provide these additional semantics.  Yes,
iread/iwrite are independent and so offer no chance to coordinate.  

- Jay's thread-like event loop requiring periodic poking

- If you set up an RMA window to buffer I/O, you run into trouble if
  the host processor goes off into a computational loop and the MPI
  library does not make background RMA progress.   Said otherwise,
  this one also requires periodic poking

- If you use something like zookeeper to hand out offsets, you have
  the problem of setting up another process  or set of processes. 

- If you go back in time and run on an Intel Paragon you can use that
  file systems native shared file pointer support to get the same

See, now you have a list of options, and you show them to someone like
Dries and he says either "you forgot about XYZ" or "oh, i see now, all
of those are gross"


Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA

More information about the mpiwg-io mailing list