[MPI3-IO] [EXTERNAL] New MPI-IO routines

[Rob Latham]

On Thu, May 24, 2012 at 03:53:04PM -0500, Mohamad Chaarawi wrote:
> Hi Dries,
> 
> On 5/24/2012 3:28 PM, Dries Kimpe wrote:
> >Also, what (if anything) would prevent implementing this on top of MPI,
> >using existing the MPI_File_iread/iwrite routines?
> 
> How would you do this efficiently and asynchronously?
> you need to set atomic mode to get sequential consistency (which is
> not pretty and I don't think it is well supported). Even with atomic
> mode set, how would you guarantee serial execution order?

Dries is asking how you could build a library on top of
MPI_File_iread/iwrite to provide these additional semantics.  Yes,
iread/iwrite are independent and so offer no chance to coordinate.  

- Jay's thread-like event loop requiring periodic poking

- If you set up an RMA window to buffer I/O, you run into trouble if
  the host processor goes off into a computational loop and the MPI
  library does not make background RMA progress.   Said otherwise,
  this one also requires periodic poking

- If you use something like zookeeper to hand out offsets, you have
  the problem of setting up another process  or set of processes. 

- If you go back in time and run on an Intel Paragon you can use that
  file systems native shared file pointer support to get the same
  effect.

See, now you have a list of options, and you show them to someone like
Dries and he says either "you forgot about XYZ" or "oh, i see now, all
of those are gross"

==rob

-- 
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA