[Mpi-21] Review of MPI-2.1 combined document

[Solt, David George]

I have a mix of suggestions and questions below  for chapter 3.  Feel free to ignore the questions and picky suggestions.  I hope maybe an experienced forum member can assist me with some of my questions next week.

I do have 2 strong suggestions that are not merge related.  I don't think these are on a ballot yet:

[[ strong suggestion ]] Page 99, line 31.  "A committed datatype can still be used as a argument in datatype constructors." should be "An uncommitted datatype can still be used as a argument in datatype constructors."

[[ strong suggestion ]] Page 63, line 43 requests(indices(i)) should be request_list(indices(i))

And the rest....

[[ Picky Suggestion ]] Page 27, line 20.  Example could be shorted using MPI_STATUS_IGNORE.

[[ question ]] Page 28, line 3.  Here we claim that the last 3 arguments "specify" the envelope. But in section 3.2.3, we claim that a message envelope consists of 4 parts.  Perhaps this line should read, "The last three parameters of the send operation, together with the rank of the caller, specify the envelope for the message sent.

[[ question ]] Page 28, 3.2, line 28.  I never understood why it says "Initial address".  Makes it sound like it could change during the course of the operation.  I like "starting" or "beginning"

[[ suggestion ]] Page 30, line 1.  Remove "MPI LONG LONG INT, for C integers declared to be of type long long;"  (this is discussed below at line Page 30, line 28.

[[ merge suggestion ]] Page 30, 3.2, line 16.  I would like to see MPI_WCHAR and wchar_t simply added to the C table (page 29, line 22).  I don't see any reason to add any additional flavor text or rational in the merged document.  Those who know what a wchar_t is, know why MPI_WCHAR is in the standard and the others do not care.

[[ merge suggestion ]] Page 30, 3.2, line 28.  The committee has included BOTH long long and unsigned long long, but the standard is only adopting MPI_UNSIGNED_LONG_LONG?  Why not MPI_LONG_LONG.  Also, if these are now accepted by ISO C9X and by the MPI standard, then simply include them in the table (page 29, line 22), there is no need to call them out as special in any way in the merged document.

[[ suggestion ]] page 33, line 1.  Reads awkwardly.  Perhaps parenthesis (unless source =MPI_ANY_SOURCE in the pattern) and (unless tag== MPI_ANY_TAG in the pattern) or eliminate these strings entirely since we already said that these are patching patterns for any source/tag.

[[ question ]] page 35, line 3.  I don't understand the motivation for the text: "Note that MPI STATUS IGNORE is not a special type of MPI STATUS object; rather, it is a special value for the argument.  In C one would expect it to be NULL, not the address of a special MPI STATUS."  I assume that the intent is to avoid the user from trying to reference fields of MPI_STATUS_IGNORE.  However, I think this is a very minor benefit and using NULL gives the compiler less opportunity to find type mismatch errors.  I assume that ((MPI_Status*) 0) would be ok, but the way this reads, it sounds like such a definition is contrary the standard.

[[ suggestion merge ]] Page 35, line 12.  "The functions that can be passed MPI STATUS IGNORE are all the various forms of MPI RECV, MPI TEST, and MPI WAIT, as well as MPI REQUEST GET STATUS."  is semi-redundant with Page 35, line 2: "... which when passed to a receive, wait, or test function, inform the implementation that the status fields are not to be filled in."  I would suggest that line 2 be changed to "which when passed to a receive, wait, test or the MPI_REQUEST_GET_STATUS function, inform the implementation that the status fields are not to be filled in." and then remove the sentence from line 12 completely.

[[ suggestion merge ]] Page 35, line 14.  "When an array is" change to "When an array of statuses is".  If the change above is made, then I think it will read easier to include "of statuses".

[[ picky suggestion ]] Page 40, line 9.  "used" to "uses"

[[ picky suggestion ]] Page 43, line 15.  "(most?)" is this actually part of the text or leftover editing?

[[ question ]] Page 47, line 3,17.  I don't understand why "initial" is used here.  I think "starting" or "beginning" would be better than "initial" for most of the function description, since "initial" implies it may change during the call.  However, in this case, I don't understand why any descriptor is used before "buffer address".

[[ suggestion ]] Page 49, line 18, 24.  "The send start call will return before the message was copied out of the send buffer."  This seems like this should be an implementation choice.  The MPI_Isend should be allowed to copy data out of the send buffer as long as it does not block on the receiver.  I think it should say, "The send start call can return..." Same on line 24.

[[ suggestion ]] Page 50, line 18.  "The use of nonblocking sends is advantageous in these cases [buffered & ready] only if data copying can be concurrent with computation."  I believe this statement is false.  An early call to MPI_Ibsend, for example, can avoid buffering entirely:

SENDER                          RECEIVER
------                          --------
MPI_Ibsend(..., req);
                                        MPI_Recv(...);
MPI_Wait(req);   (completes quickly without buffering because the message has already been transferred)

Compared to:

                                        MPI_Recv(...);
MPI_Bsend(...); (must buffer the message because the sender does not yet know if
                        a matching receive call has been made)

[[ merge suggestion ]] Page 53, line 29 (+ other occurrences).  "Section on Argument copying and register optimization"  (remove?, change to a heading?).  Although I understand the seriousness of this issue, I'm not sure polluting the document with multiple advice to users sections is the best way to go.  This issue needs to be very clearly addressed in the document as a fundamental fortran issue, but otherwise left alone.  Scattering multiple warnings throughout the document is a maintenance issue and only ensures we will miss some locations.

[[ picky suggestion ]] Page 55, line 29.  "Thus, a blocking Wait can be easily replaced by a nonblocking Test" changed to "Thus, a blocking Wait can be easily implemented using a nonblocking Test".   The intent is clearly a loop of nonblocking Test calls, not a direct replacement.

[[ question ]] Page 58, line 14.  What is the point to the paragraph.  It has already been stated that even if no completion routine has been called, that a matched non-blocking call must complete.  What is the point to calling out MPI_Test?

[[ picky suggestion ]] Page 58, line 26.  "enabled operations" is unclear.  Maybe "finished operations"?

[[ picky suggestion ]] Page 59, line 4.  The attempt to define the functionality of MPI_Waitany uses the functionality of MPI_Waitany in the definition.   I would not mind, since I think the sentence is still meaningful, but then on line 39, MPI_Testany is defined more fully using the phrase "for i=0 ,..., count-1".  The description of MPI_Testany defines the call in terms of the equivalent MPI_Test calls that we would expect would be used to implement the MPI_Testany call.  However, on line 4, the description of MPI_Waitany is given solely in terms of the end net effect of the call in comparison to the equivalent MPI_Wait call.  One description is in terms of implementation, the other in terms of net result.

[[ picky suggestion ]] Page 64, line 2.  "source rank, or MPI ANY SOURCE (integer)"  The use of MPI_ANY_SOURCE (same for TAG) is not called out in receive routines, but is here.  There may be some confusion among users if MPI_ANY_SOURCE is allowed in both cases when looking at just the prototypes.

[[ suggestion ]] Page 74, line 27.  The statement is true for sendrecv, but not for sendrecv_replace.  Its placement at this location would indicate it is in reference to sendrecv_replace.

[[ question ]] Page 74, line 1.  Why did we not use MPI_IN_PLACE for the receive buffer to MPI_Sendrecv and not have an MPI_Sendrecv_replace.  Is MPI_IN_PLACE allowed for MPI_Sendrecv? Should it be?

[[ merge suggestion ]] Page 76, line 39.  I don't know what the intended style is of the merged document, but I would prefer that it refrained from designating functions as "new".  Deprecated functions should be designated as such, but otherwise my preference is to avoid distinctions between "new" and "old".   I would title this section "Datatype Manipulation Functions".  Suggested text:

The following sections present type manipulation functions.  These functions use address sized arguments where ever an offset may be specified, allowing for the passing of offsets greater than 2^32.  Deprecated versions of these functions are listed in Section 2.6.1.

[[ merge suggestion ]] Page 79, line 43.  Do we want to list out deprecated functions?  I wonder if we could hide them all away in an appendix.

[[ merge suggestion ]] Page 81, line 7.  Why is this function not deprecated since it does not take integer sized displacements.

[[ merge suggestion ]] Page 83. line 27.  Some formatting problems

[[ merge suggestion ]] Page 84, line 7.  No MPI_Type_create_hindexed_block?

[[ merge suggestion ]] Page 84, line 31.  "It further generalizes the previous one in" -- "It further generalizes MPI_Type_create_indexed" or "It further generalizes indexed", since MPI_Type_create_hindexed_block was a step backwards in complexity.

[[ merge suggestion ]] Page 89, line 44.  Again, I would prefer a presentation in which MPI-2 is a full and complete specification and therefore reword this to:

Advice to users. For both Fortran and C arrays, the ordering of processes in the
process grid is assumed to be row-major. This is consistent with the ordering used in
virtual Cartesian process topologies. To create such virtual process topologies,
or to find the coordinates of a process in the process grid, etc., users may use
the corresponding process topology functions. (End of advice to users.)

[[ merge suggestion ]] Page 94, line 22.  Update example to use MPI_Get_address.  Not sure if there are other examples that use deprecated functions.  Maybe a full search of the deprecated functions is in order.

[[ merge suggestion ]] Page 95, line 1.  I would recommend this section be re-written to present MPI_Type_get_exent explicitly and not in terms of the deprecated functions.

[[ merge suggestion ]] Page 95, line 21.  Reference to section 3.12.2 is incorrect and the parenthetical statement should reference the 2.1 document.  At this point in the document, the lower bound of a datatype has not been presented.  Some reordering will need to take place as I believe this originally referred to what is now 3.12.9.

[[ merge suggestion ]] Page 96, line 24.  Again, my preference towards removing text such as this.

[[ question ]] To this point, there has been no description of what is a legal or illegal datatype.  Any combination of arguments that individually meet the requirements appear are assumed legal.  There has been no description of datatypes such as ((MPI_INT, 0), (MPI_INT, 0)) or worse yet, ((MPI_CHAR, 0), (MPI_INT, 1)), which has alignment problems.

[[ merge suggestion ]] Page 97, line 41.  Remove "(Readers should compare this with the definitions in Section 3.12.3 of the MPI-1 standard,
which describes the function MPI TYPE EXTENT.)" since they are now presented together.

[[ merge suggestion ]] Page 98, line 1.  Move content of this section to the beginning of 3.12.7

[[ merge suggestion ]] Page 99, line 1-27.  Move these definitions to bottom of page 95 (after MPI_Type_extent description) or follow my earlier proposal that we don't document any deprecated functions in the text.

[[ merge question ]] Page 100, line 6.  This gets to the heart of it... Are we describing both standards or just 2.0?  If the later, than I would change this to: "MPI TYPE COMMIT will accept a committed datatype; in this case, it is equivalent to a no-op."

[[ merge suggestion ]] Page 101, line 15.  use of "new".  I don't think key values or copy callback functions have been presented yet, at least not in this chapter.

[[ merge suggestion ]] Page 103, line 17, formatting problems.

[[ question ]] Page 104, line 27:  "The function MPI ADDRESS returns a valid address, when passed as argument a variable of the calling program."  Not sure what this is saying.  Same for line 29-30.

[[ suggestion ]] Page 105, line 11. "Note that in Fortran, Fortran INTEGERs may be too small to contain an address (e.g., 32 bit INTEGERs on a machine with 64bit pointers). Because of this, in Fortran, implementations may restrict the use of absolute addresses to only part of the process memory, and restrict the use of relative displacements to subranges of the process memory where they are constrained by the size of Fortran INTEGERs."  Since this new document has introduced MPI_ADDRESS_KIND for all arguments that would have taken an integer, do we still need to say this.

[[ merge suggestion ]] Page 105, line 19.  Examples should not use deprecated functions.

[[ suggestion ]] Page 120, line 28.  MPI_PACK_EXTERNAL_SIZE is never defined.

Thanks,
Dave Solt
HP