[Mpi-comments] Request: Canonical MPI_Pack_external with different data representations

[Martin Bernreuther]

Hello,

since MPI-2.0 there's a "Canonical MPI_PACK and MPI_UNPACK" (p. 78, section 4.16)
(also see e.g. https://www.mpi-forum.org/docs/mpi-2.1/mpi21-report-bw/node83.htm#Node83),
which didn't change much over time:
"
These functions read/write data to/from the buffer in the "external32" data format specified
in Section 13.5.2, and calculate the size needed for packing. Their first arguments specify
the data format, for future extensibility, but currently the only valid value of the datarep
argument is "external32."
"(taken from MPI-3.1, p. 138, section 4.3)

After more than 20 years, how about making use of the "future extensibility"
for the next standard to introduce more datarep arguments (like e.g. "native")?



With MPI_PACK for I/O there's a
"
Advice to users. The type MPI_PACKED is treated as bytes and is not converted.
The user should be aware that MPI_PACK has the option of placing a header in the
beginning of the pack buffer. (End of advice to users.)
"(MPI-31, p. 539)
A "header" might be undesirable for other programs to read and this seems
not to be an issue using MPI_Pack_external with MPI_BYTE, since there is no header:
"
Rationale. MPI_PACK_EXTERNAL specifies that there is no header on the message
and further specifies the exact format of the data. Since MPI_PACK may (and is
allowed to) use a header, the datatype MPI_PACKED cannot be used for data packed
with MPI_PACK_EXTERNAL. (End of rationale.)
"(MPI-31, p. 138)
Thus the canonical version seems to have an advantage for MPI-I/O here.
However if a user wants to write e.g. "little endian" on an actual x86 architecture,
this seems to be excluded by the actual standard.


Thank you,
	Martin Bernreuther

P.S.:
I also wonder how to determine the exact size of a pack buffer.
"
A call to MPI_PACK_SIZE(incount, datatype, comm, size) returns in size an upper bound
on the increment in position that is effected by a call to MPI_PACK(inbuf, incount, datatype,
outbuf, outcount, position, comm).[...]

Rationale. The call returns an upper bound, rather than an exact bound, since the
exact amount of space needed to pack the message may depend on the context (e.g.,
first message packed in a packing unit may take more space). (End of rationale.)
"(MPI3.1, p. 135)
There seems not to exist e.g. a routine to add up the sizes (INOUT size instead of position)
analogous to the Pack procedure. Is there a function to determine the extra size for the
first message (see Rationale).
How should the exact buffer size (and not only an upper bound) be determinated before packing?

--
________________________________________________________________________
  Dr.-Ing. Martin Bernreuther
  University of Stuttgart
  High Performance Computing Center (HLRS)
  Nobelstrasse 19 (Office: 1.307)
  70569 Stuttgart, Germany
  
  Phone: (++49-(0)711) 685-64542
  Fax:   (++49-(0)711) 685-54542, -87279, -65832
  E-Mail: bernreuther at hlrs.de
  URL: http://www.hlrs.de/people/bernreuther/