[mpi3-coll] Nonblocking collectives standard draft

[Christian Siebert]

Hello Torsten,

excellent! However, there are some remarks and corrections from my (our) 
side:

p 1: "Non-Blocking" -> "Nonblocking" ;-)

btw. all line numbers within the MPI standard are not really line 
numbers (they have equal distances and as soon as there is another font 
size - like for the interface definitions - then you end up anywhere...)
   It's like "look at page X at _centimetre_ Y".

p 49, l 46: it is not enough to state "receive buffer must not be read"
   it should be forbidden to access the receive buffer in any way
   (imagine an implementation that uses the buffer to forwards sth.)

p 50, l 7: we should discuss MPI_REQUEST_FREE (again) - I talked to some
   colleagues and we see application scenarios where this might be
   useful (e.g., if only a subgroup participates in an irregular
   collective); because this is a local operation, we also don't
   see any problems by generally allowing MPI_REQUEST_FREE for
   nonblocking collective operations

p 50, l 13: "a simple local execution schedule" -> "local execution 
schedules"
   (alternatively "A nonblocking collective operation...";
    however, please omit this subjective "simple")

p 50, l 19: maybe we should be even more precise and add after 
"environments." something like "It is therefore erroneous to start a set 
of nbc in different logical orders on the participating ranks."
   The sentence in l 20 ("The behavior...") can and should be deleted.
   "Nbc operations and bc operations [+are totally unrelated and ]
    do not match each other."

p 52, l 41.5: the C++ interface name is for the blocking variant
   l 45: "non-blocking" -> "nonblocking"

several locations: "memory movement" should be replaced by "data movement"
   (but "movement" sounds bad anyway because you say
    "after completion" - maybe "data placement" would be
    more appropriate)

p 58, l 23.6 and 28.4: interface name for C++ & Fortran is for blocking var.

p 59, l 17.2: Fortran interface name...

p 61, l 31.8 & p 62, l 27.5: "after it completion" -> "after completion"?

p 63, l 11: "reduction-to-all" for MPI_REDUCE_SCATTER?

p 63, l 39-41 ...

btw. there seems to be a potential naming problem for beginners:
   "Exclusive Scan -> Exscan" && "Inclusive Scan -> Iscan"???

p 64, l 24-27 ...

p 68, l 46.3: "req[1]" -> "req"

p 69, l 20: "req[1]" -> "req[0]"

p 69, l 41: "in SPMD style programs" seems superflous to me
   but "double buffering" could also be added as an example ...

p 70, l 7: "t" -> "to"

Example 5.33 seems to be very dangerous for at least two reasons:
   1) sbuf and rbuf must be carefully initialized
      (using the same for both operations seems nonsense to me)
   2) this doesn't make it a "global" allreduce because
      (depending how sbuf and rbuf are initialized) all
      seem to calculate different results
      (0+0+1+2 vs. 0+1+1+2 vs. 1+2+0+2)
      This example is a good idea but needs some clarifications...

What do we want to have (either or):
   a) the same ordering over both blocking and nonblocking collectives
      (suggested by Example 5.29)
   b) blocking and nonblocking collectives are completely independent
      (suggested by "reserved tag-space" p 50, l 15)
Upon a decision, the related text needs some clarification.


All in all a big step towards the standardization. However, there seem 
to be quite a couple of mistakes in the copy-and-paste parts and I'm 
quite sure that I didn't catch all in my first go... so please be 
especially careful in those places.

Many thanks!

Best regards,
    Christian

Torsten Hoefler wrote:
> Hello,
> as promised in the last forum, I finished the draft of our standard
> proposal. I don't want to post it to the mailinglist, so I uploaded it
> to the wiki pages [1] and put a copy at [2] for convenience.
> 
> The document includes the whole collective chapter and the additions
> (some references to other chapters are thus broken). All changes are
> clearly marked:
> - Striked text has been removed and underlined text has been added. 
> - Section 5.12 is new (not underlined)
> - Examples 5.28-5.33 are new (not underlined)
> 
> Please read the additions and send comments to me!
> 
> I hope to be able to discuss the proposal during the next teleconference
> (which will be announced in a separate mail).
> 
> Thanks & All the Best,
>   Torsten
> 
> [1]: https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/NBColl
> [2]: http://www.unixer.de/sec/nbc-proposal-11-12-2008.pdf

-- 
Christian Siebert, Dipl.-Inf.               Research Associate

            NEC Laboratories Europe, NEC Europe Ltd.
        Rathausallee 10, D-53757 Sankt Augustin, Germany

Phone: +49 (0) 2241 - 92 52 44    Fax: +49 (0) 2241 - 92 52 99

  (Registered Office: 1 Victoria Road, London W3 6BL, 2832014)