[mpi3-coll] Nonblocking collectives standard draft

[Rajeev Thakur]

> At 01:39 PM 12/1/2008, Rajeev Thakur wrote:
> >MPI_Cancel cannot simply be ignored. For example, if there is an 
> >MPI_Isend followed by MPI_Cancel, and the matching receive is never 
> >called, the cancel cannot fail.
> 
> Are you sure that this is the way it works? It doesn't say so 
> explicitly in the spec and the notes for MPI_Cancel on 
> http://www-unix.mcs.anl.gov/mpi/www/ say "Cancel has only 
> been implemented for receive requests; it is a no-op for send 
> requests".

Those are some old web pages from somewhere :-). 

The standard says on pg 67 "If a send is marked for cancellation, then it
must be the case that either the send completes normally, in which case the
message sent was received at the destination process, or that the send is
successfully canceled, in which case no part of the message was received at
the destination. Then, any matching receive has to be satisfied by another
send." 

So if a send is cancelled, and the destination program doesn't ever call a
matching receive, the implementation has to cancel the send; it can't simply
say the cancel failed. Otherwise, MPI_Cancel would be the easiest call to
implement (along with MPI_Pcontrol) :-).

> However, if we do need to provide the above semantics for 
> MPI_Cancel, we will have to deal with a funny non-locality of 
> behavior. Suppose we have an MPI_Ibcast and one of the 
> receivers calls MPI_Cancel. What happens to the other 
> participants? Do they all have to call MPI_Cancel to not 
> participate or is any one calling it sufficient? 
> What about things like MPI_Gather and MPI_AlltoAll? We 
> certainly don't want to force non-all-to-all operations to 
> have to do an all-reduce at the end to check for cancellation.

Yup, those are the issues that come up. One could say that MPI_Cancel is not
allowed for nonblocking collectives.

Rajeev