[MPI3 Fortran] Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION

Fri Oct 7 11:13:58 CDT 2011

I forgot to add the struct-alignment discussion which is valid for C and
BIND(C) Fortran! Now (Oct.7) done!

(This mail does not take reviews into account that were doen since
yesterday)
The atteched .txt file contains the same as this mail-text.

(Lines with n are new on Oct.7, 2011,
 lines with * are new on Oct.6, 2011, 
 lines with : are modified, 
 lines with x are removed)

________

mpi-report-F2008-2011-09-08-changeonlyplustickets.pdf
p13:23-31 read

  Rationale. Since the Fortran integer values are equivalent, applications can easily
  convert MPI handles between all three supported Fortran methods. For example, an
  integer communicator handle COMM can be converted directly into an exactly equivalent
  mpi_f08 communicator handle named comm_f08 by comm_f08%MPI_VAL=COMM, and
  vice versa. The use of the INTEGER defined handles and the BIND(C) derived type
  handles is different: Fortran 2003 (and later) define that BIND(C) derived types can
  be used within user defined common blocks, but it is up to the rules of the companion
  C compiler how many numerical storage unit are used for these BIND(C) derived type
  handles. 
  (End of rationale.)

but should read

  Rationale. Since the Fortran integer values are equivalent, applications can easily
  convert MPI handles between all three supported Fortran methods. For example, an
  integer communicator handle COMM can be converted directly into an exactly equivalent
  mpi_f08 communicator handle named comm_f08 by comm_f08%MPI_VAL=COMM, and
  vice versa. The use of the INTEGER defined handles and the BIND(C) derived type
  handles is different: Fortran 2003 (and later) define that BIND(C) derived types can
  be used within user defined common blocks, but it is up to the rules of the companion
: C compiler how many numerical storage units are used for these BIND(C) derived type
  handles.
n Most compilers use one unit for both, the INTEGER handles and the handles defined as 
n BIND(C) derived types. 
  (End of rationale.)

________

mpi-report-F2008-2011-09-08-changeonlyplustickets.pdf
p86:45-47 (=MPI-2.2 p78:45-47) read

  If type_i requires alignment to a byte address that is a multiple of k_i, then epsilon 
  is the least non-negative increment needed to round extent(Typemap) to the next multiple 
  of max_i k_i.
  The complete definition of extent is given on page 107.

but should read

  If type_i requires alignment to a byte address that is a multiple of k_i, then epsilon 
  is the least non-negative increment needed to round extent(Typemap) to the next multiple 
  of max_i k_i.
* In Fortran, whether the alignments k_i are computed according to the alignments 
* used by the compiler in common blocks, SEQUENCE derived types, or BIND(C) derived types,
* is implementation-dependent.
: The complete definition of extent is given in Section 4.1.6 on page 106.

________

mpi-report-F2008-2011-09-08-changeonlyplustickets.pdf
p87:7-11 (=MPI-2.2 p79:7-11) read

  Rationale. The definition of extent is motivated by the assumption that the amount
  of padding added at the end of each structure in an array of structures is the least
  needed to fulfill alignment constraints. More explicit control of the extent is provided
  in Section 4.1.6. Such explicit control is needed in cases where the assumption does
  not hold, for example, where union types are used. 
  (End of rationale.)

but should read

  Rationale. The definition of extent is motivated by the assumption that the amount
  of padding added at the end of each structure in an array of structures is the least
  needed to fulfill alignment constraints. More explicit control of the extent is provided
  in Section 4.1.6. Such explicit control is needed in cases where the assumption does
  not hold, for example, where union types are used. 
* In Fortran, structures can be expressed with several language features, e.g.,
* common blocks, SEQUENCE derived types, or BIND(C) derived types. The compiler may
* use different alignments, and therefore, it is recommended to use MPI_TYPE_CREATE_RESIZED
* for arrays of structures if an alignment may cause an alignment-gap at the end of 
* a structure as described in Section 4.1.6 on page 106 and in Section 16.2.15 on page 580. 
  (End of rationale.)

________

mpi-report-F2008-2011-09-08-changeonlyplustickets.pdf
p107:13-17 (=MPI-2.2 p96:42-45) read

  If type_i requires alignment to a byte address that is a multiple of k_i, then epsilon
  is the least non-negative increment needed to round extent(Typemap) to the next multiple
  of max_i k_i.
  The formal definitions given for the various datatype constructors apply now, 
  with the amended definition of extent.

but should read

  If type_i requires alignment to a byte address that is a multiple of k_i, then epsilon
  is the least non-negative increment needed to round extent(Typemap) to the next multiple
  of max_i k_i.
* In Fortran, whether the alignments k_i are computed according to the alignments 
* used by the compiler in common blocks, SEQUENCE derived types, or BIND(C) derived types,
* is implementation-dependent.
  The formal definitions given for the various datatype constructors apply now, 
  with the amended definition of extent.

* Rationale.
* Before Fortran 2003, MPI_TYPE_CREATE_STRUCT could be applied to Fortran common blocks and
* SEQUENCE derived types. With Fortran 2003, this list was extended by BIND(C) derived types
* and MPI implementors have implemented the alignments k_i different, i.e., some based on
* the alignments used in SEQUENCE derived types, and others according to BIND(C) derived types.
* (End of rationale.)

* Advice to implementors.
* In Fortran, it is generally recommended to use BIND(C) derived types instead of
* common blocks or SEQUENCE derived types. Therefore it is recommended to calculate the
* alignments k_i based on BIND(C) derived types.
* (End of advice to implementors.)

* Advice to users.
* Structures combining different basic datatypes should be defined so that there will
* be no gaps based on alignment rules, and if used as an array of structures, then
* also without such an alignment-gap at the end of the structure.
* In MPI communication, the content of such gaps would not be communicated.
* If alignment-gaps may occur, e.g., between an odd number of floats ore REAL before
* a double or DOUBLE PRECISION data, then such gaps should be added explicitely in 
* the structure and also to the MPI derived datatype handle, because the communication
* of a contiguous derived datatype may be significantly faster than the communication
* of one that is non-contiguous due such alignment-gaps. 
*
* Example:
*   Instead of
*
*     TYPE, BIND(C) :: my_data
*       REAL, DIMENSION(3) :: x
*       ! there may be a gap of the size of one REAL
*       ! if the alignment of a DOUBLE PRECISION is 
*       ! two times the size of a REAL
*       DOUBLE PRECISION: p
*     END TYPE
* 
* one should define
*
*     TYPE, BIND(C) :: my_data
*       REAL, DIMENSION(3) :: x
*       REAL :: gap1
*       DOUBLE PRECISION: p
*     END TYPE
*
* and also including gap1 in the matching MPI derived datatype.
n It is required that all processes in a communication add the same
n gaps, i.e., with defined the same basic datatype.
n
n In principle, a compiler may define an additional alignment rule 
n for structures, e.g., to use at least 4 or 8 byte alignment although
n the content may have a max_i k_i alignment less than this structure
n alignment. To keep an application portable, it is therefore
n recommended to always resize structure derived datatype handles
n if used in an array of structures, see the Example in Section
n 16.2.15 on page 580.
n (End of advice to users.)

__________

mpi-report-F2008-2011-09-08-changeonlyplustickets.pdf
p580:12,21,26,30,34,36:

  Add a forth structure entry: logical l after the double precision d

________

mpi-report-F2008-2011-09-08-changeonlyplustickets.pdf
p581:11-29 read

x TODO: The following text about the extent of derived types should be
x strongly checked by specialist on derived types!!! The correct variant should
x be chosen.
  To use a derived type in an array requires a correct extent of the datatype handle to
  take care of the alignment rules applied by the compiler. These alignment rules may imply
  that there are gaps between the elements of a derived type, and also between the array
  elements. 
x The alignment rules in Section 4.1 on page 85 and Section 4.1.6 on page 106
x apply only to
x VARIANT 1: SEQUENCE
x VARIANT 2: BIND(C)
x derived types. 
  The extent of an iteroperable derived type (i.e., defined with BIND(C)) and
  a SEQUENCE derived type with the same content may be different because C and Fortran
  may apply different alignment rules.
x VARIANT 1:
x Using the SEQUENCE attribute instead of BIND(C) in the declararion on mytype, one can
x directly use newtype to send the fooarr array.
x VARIANT 2:
x In the example, one can directly use newtype to send the fooarr array. The resized
x newarrtype datatype is only needed, if mytype is a SEQUENCE derived type.

but should read

  To use a derived type in an array requires a correct extent of the datatype handle to
  take care of the alignment rules applied by the compiler. These alignment rules may imply
  that there are gaps between the elements of a derived type, and also between the array
  elements.
  The extent of an iteroperable derived type (i.e., defined with BIND(C)) and
  a SEQUENCE derived type with the same content may be different because C and Fortran
  may apply different alignment rules.
n As recommended in the advice to users in Section 4.1.6 on page 106,
n one should add an additional sixth structure element with one numerical storage unit 
n at the end of this structure to guarantee that the array of stuctures is contiguous.
n Without such an additional element, the resizing is needed because the 
n alignment rules cannot guarantee that aliagnment of the stucture used by the compiler 
n and calculated by MPI is identical.
n With such an additional element, one should also keep the resizing due to the
n special alignment rules that can be used by the compiler for structures, 
n as also mentioned in this advice.

________________

Best regards
Rolf

-- 
Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de
High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530
University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832
Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner
Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: MPI-Fortran-Late-Changes-Alignment.txt
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-fortran/attachments/20111007/84b90acd/attachment-0001.txt>