[MPI3 Fortran] [Fwd: Library-based ASYNCHRONOUS I/O and SYNC MEMORY]

[Jeff Squyres]

On Sep 9, 2008, at 8:19 AM, Dan Nagle wrote:

> Look, MPI works with Fortran, and it has for years.
> There's no emergency.  We can take our time and
> do things right to make the Fortran interface
> to MPI routines easier to define, to maintain, and to use.

Sure; we've waited 12 years -- why not longer?  ;-)

>> Can you come up with an example of this dual buffer usage where  
>> performance would actually require the kind of precise control  
>> you're talking about?

Separation and asynchronocity between communication and computation is  
a well-known philosophy (I don't know the first machine that had  
dedicated communication co-processors, but it was many years ago --  
probably more than 15-20?).  I'm guessing that it's pretty much  
exactly the same arguments for why Fortran has a built-in asynchronous  
IO library.

The short version is that network communication -- even with modern,  
very low latency / high bandwidth networks -- takes orders of  
magnitude longer than local computation.  There are many MPI  
applications that actively benefit from asynchronous network  
communication (e.g., hardware- and/or software-driven message passing  
progress while the main computation thread(s) go off and do useful/non- 
message-passing-related work).

> Personally (and others may differ), I believe the main issue
> is to better document what happens with asynchronous modifications
> to actual arguments after the initiating routine has returned.
> Generally, this is surprising to a standards conforming program.

I guess I don't see why this is so surprising given the history of  
separation of communication and computation.  Non-blocking network  
communication, when the application can assume that progress is being  
made "in the background" is a very long-standing precept.  It can be  
very simply summarized as:

- you start the communication (e.g., a send or a receive)
- you go do other stuff, most likely not involving the buffer(s)  
involved with the ongoing communication
- you poll for completion of the communication later

The main important point is to not touch the involved buffers until  
the communication has been completed (e.g., if you're receiving, don't  
read from it until the receive is complete; if you're sending, don't  
write to it until the send is complete).  I'm waving my hands past a  
few details, but that's the gist of it.

One clarification that is going to be in the upcoming MPI-2.2 document  
(probably 6-12 months out?) is that multiple non-blocking sends may  
occur simultaneously from the same buffer.  That is, it's ok to have  
multiple simultaneous reads from a buffer during a send operation.

> If a compiler (even one at some time in the future)
> can use the information profitably, so much the better.
> But the main issue, to me at least, is to document,
> *in the program*, what is expected to happen.  Having this
> information in the source file helps the applications writer,
> the applications maintainer,  and any code checking tools
> brought to bear.


Hmm; I don't think I understand your point here.  Are you saying that  
a pragma or attribute or somesuch is better code documentation than a  
call to an API function that is documented to be non-blocking?

-- 
Jeff Squyres
Cisco Systems