[Mpi-forum] MPI_Mprobe workaround

[Jeff Hammond]

I was reading through
www.unixer.de/publications/img/mprobe-proposal-rev4.pdf and - just for
fun - wondered if there is a workaround, not because MPI_Mprobe
shouldn't be in the MPI-3 standard, but because some folks might want
a workaround for backwards compatibility if they are forced to use
MPI-2 somewhere.

I apologize in advance if something equivalent to what I say below was
discussed at the Forum and I was not present.

Torsten and coworkers say the following.
============================
For example, the following code can not be executed concurrently by
two threads in an MPI process, because a message could be found by the
MPI Probe in both threads, while only one of the threads could
successfully receive the message (the other will block):
MPI_Status status;
int value;
MPI_Probe(MPI_ANY_SOURCE, /*tag=*/0, MPI_COMM_WORLD, &status);
MPI_Recv(&value, 1, MPI_INT, status.MPI_SOURCE, /*tag=*/0,
MPI_COMM_WORLD, MPI_STATUS_IGNORE);

<snip>

There is no known workaround that addresses all of the problems with
MPI Probe and MPI Iprobe in multi-threaded MPI applications.
============================

Obviously, a fat mutex around this block solves the problem, but the
time spent in the mutex will scale with the message size.  I was
curious the following workaround is reasonable when MPI-2 must be
used.
============================
MPI_Status status;
MPI_Request request;
AppropriateMutex mutex;
int value;

ACQUIRE_MUTEX(&mutex);
MPI_Probe(MPI_ANY_SOURCE, /*tag=*/0, MPI_COMM_WORLD, &status);
/* ? */
MPI_Irecv(&value, 1, MPI_INT, status.MPI_SOURCE, /*tag=*/0,
MPI_COMM_WORLD, &request);
RELEASE_MUTEX(&mutex);
MPI_Wait(&request, MPI_STATUS_IGNORE);
============================

Assuming MPI_Irecv returns quickly and there is nothing in between the
two operations, then the time spent in the critical section defined by
the mutex will be rather short.  My solution may become inefficient if
one has to allocate memory to receive the message while holding the
mutex.  However, instances where memory allocation is slow should be
quite rare and there are workarounds when such scenarios exist.

Are there nuances regarding the use of MPI that I have missed?  Do the
real-world use cases have too much to do in the "/* ? */" to make this
viable?

Thanks,

Jeff

-- 
Jeff Hammond
Argonne Leadership Computing Facility
University of Chicago Computation Institute
jhammond at alcf.anl.gov / (630) 252-5381
http://www.linkedin.com/in/jeffhammond
https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond