[Mpi3-rma] Fwd: [Mpi-comments] Comments on MPI 3.0 draft 2 [part 4]

[Jeff Hammond]

Hi Jeremiah,

I am not a chapter author but a member of the RMA working group.
Thanks for your very thorough comments.  I will comment on the few
non-trivial ones that I am qualified to discuss.  I'm sure others will
comment once they've read your exhausted list and cross-checked with
the standard.

Best,

Jeff

> Why are the request-based one-sided functions named MPI_R* rather than
> MPI_I* like nonblocking point-to-point and collective functions?

The "I" in MPI_Isend is in reference to these functions returning
"immediately", not their use of requests.

MPI RMA communications operations are "nonblocking" (there was some
discussion about when they could block, but I believe that is only in
lock/unlock; I could be wrong) and therefore return "immediately"
whether or not they involve requests, so the "immediate" name
attribute is not appropriate as a distinguishing feature of the
request-based operations.

In any case, the MPI standard does not seek to come up with perfect
names.  There are plenty of functions that have unfortunate names.
Users are always welcome to define their own wrappers to MPI with
names they like :-)

> There are no request-based versions of fetch-and-op or compare-and-swap.

There was no use-case for this.  I had to fight pretty hard to get
request-based operations into the RMA chapter and I did so by pointing
to a very reasonable use-case.

If you have a use-case for request-based RMW operations, then I'm sure
the Forum would be happy to consider them for MPI-Next.  There's
nothing stopping us from defining these operations.

> Page 414 line 25: The "should" might need to be "must" to avoid issues
> similar to <URL:https://upc-bugs.lbl.gov/bugzilla/show_bug.cgi?id=495>.

I completely agree with the general point that the word "should" is
useless.  It is not binding, which is why legal documents use "shall"
instead.  I don't know if "must" has a legal meaning.

The MPI standard is, of course, not a legal document, but it never
hurts to use language in such a way that would stand up in court :-)

> Page 433: Is it possible/likely that an rget does not complete locally until
> after a flush, fence, or similar call?  That might be worth mentioning if
> true.

I am reading this as "rget does not complete locally until a flush,
fence, or similar call" since it is not clear to me what you mean by
"after".  I assume the meaning is "at" instead.

MPI requires the user to query completion explicitly before the local
buffer can be queried in a meaningful way (there is an exception for
MPI_Isend and read-only access, but that's not relevant here).
Polling on bytes is E.V.I.L. and not standard-compliant.

Perhaps I have misunderstood your comment.

> Page 487: I think MPI_INIT_THREAD should be moved to be near MPI_INIT in the
> standard, since many high-performance applications use threads and any
> threading at all (even OpenMP in non-MPI-using regions) is forbidden when
> MPI_INIT is used.

I agree with this 100%.  The reason things are the way they are is
presumably historical.  I find it really obnoxious to have these two
functions in different chapters.  I'd love to see them relocated in
MPI-Next.

> [post 3.0] In the future, it might make sense for MPI_INIT to choose a
> thread mode such as MPI_THREAD_FUNNELED (or for implementations to be
> allowed to choose one) as the default when, for example, OpenMP is enabled
> by compiler settings.  Many people are probably using MPI_INIT in OpenMP
> applications without knowing that MPI makes doing that erroneous.

This would require MPI implementations to interact with the compiler,
which is not something anyone I know desires.  Furthermore, it is
possible that an application will use MPI and OpenMP in a disjoint set
of source files.  How will the MPI implementation know that e.g. foo()
invokes OpenMP?  Do you want the linker to do something crazy with
symbol redefinition to make this happen?

I find many MPI users violated the standard when using OpenMP.  I
rebuke them whenever I observe this.  I've not experienced anyone
refusing to change their code to comply with the standard.

I'll note that some implementations, which I will not name here to
protect the guilty, support threads in a very poor way such that
MPI_THREAD_FUNNELED is implemented as MPI_THREAD_MULTIPLE, meaning
that it is better for uses to violate the MPI standard and take it on
faith the implementation won't explode when OpenMP's Pthreads are
cloned than to introduce the locking overhead of MPI_THREAD_MULTIPLE
when it is entirely unnecessary.


-- 
Jeff Hammond
Argonne Leadership Computing Facility
University of Chicago Computation Institute
jhammond at alcf.anl.gov / (630) 252-5381
http://www.linkedin.com/in/jeffhammond
https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond