[MPI3 Fortran] Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION

[Rolf Rabenseifner]

Dear all who contributed to this discussion,

for MPI-3.0, I got the strong conclusion that we 
should not try to define anything about alignment
in the combination of Fortran and MPI.

I'll try to choose careful wording and advices.

Thanks and best regards
Rolf

----- Original Message -----
> From: "Bill Long" <longb at cray.com>
> To: mpi3-fortran at lists.mpi-forum.org
> Sent: Thursday, September 29, 2011 2:37:13 PM
> Subject: Re: [MPI3 Fortran] Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION
> On 9/29/11 3:22 AM, Rolf Rabenseifner wrote:
> > I never talked about DOUBLE PRECISION size.
> > All platforms used a size of 8 bytes.
> >
> 
> I hope that assumption is not embedded in the MPI design, since I know
> of examples where default REAL is 8 bytes and DOUBLE PRECISION is 16
> bytes.
> 
> 
> > The background of the problem is:
> > In the Intel compiler on IA64 and IBM's xlf on Power,
> > the alignment of (8 byte) DOUBLE PRECISION is
> > - 4 byte within a SEQUENCE derived type
> >      (always the first number in my summary sheet)
> > - 8 byte within a BIND(C) derived type
> >      (always the second number in my summary sheet)
> 
> The rules for BIND(C) layouts are determined by the particular
> compiler
> that is identified as the "companion processor" for the Fortran
> compiler. Usually it is the C compiler from the same vendor. These
> rules do not need to match those for a SEQUENCE type.
> 
> >
> > And MPI has strong alignment rules on MPI-2.2 page 78 and 79.
> >
> > The problem is:
> > This alignment is in some MPI libraries 4 and others 8 byte
> > with those compilers (always the third number in my summary sheet).
> > And a standard is not a standard if it is not standardizing
> > such things. It would be only a problem-description.
> >
> 
> It depends on what kind of standard it is. If it is a standard for a
> higher level language, like Fortran, that is intended to be
> (relatively)
> machine hardware independent, then I would expect alignment
> specifications to be largely omitted as inappropriate and harmful to
> efficient implementations. If the standard is for calling sequence
> conventions, or for internal memory representations, the the
> expectations are different.
> 
> Cheers,
> Bill
> 
> 
> 
> 
> > Rolf
> >
> > ----- Original Message -----
> >> From: "Craig E Rasmussen"<rasmussn at lanl.gov>
> >> To: "MPI-3 Fortran working group"<mpi3-fortran at lists.mpi-forum.org>
> >> Sent: Wednesday, September 28, 2011 7:58:00 PM
> >> Subject: Re: [MPI3 Fortran] Serious problem/bug in MPI libraries
> >> with the alignment of MPI_DOUBLE_PRECISION
> >> Rolf,
> >>
> >>
> >> I don't know where you get the idea that a DOUBLE PRECISION
> >> variable
> >> should occupy 4 bytes. The standard says:
> >>
> >>
> >>
> >> A nonpointer scalar object of type double precision real or default
> >> complex occupies two contiguous numeric storage units;
> >>
> >>
> >>
> >> The derived types we worked out in Santorini were especially
> >> constructed so that double precisions numbers would be misaligned
> >> if
> >> the compiler was forced to do bad things via the SEQUENCE
> >> statement.
> >> Can you give me an idea which part of the tests we wrote indicate a
> >> double precision number is 4 bytes?
> >>
> >>
> >> Also, I checked the F2003 and F2008 standards and this wording (see
> >> above) has not been changed.
> >>
> >>
> >> -craig
> >>
> >>
> >>
> >>
> >>
> >>
> >> On Sep 28, 2011, at 11:00 AM, Rolf Rabenseifner wrote:
> >>
> >>
> >>
> >>
> >>
> >> A very good question. Instead of referring to SEQUENCE, it should
> >> say
> >>
> >>
> >> that MPI_DOUBLE_PRECISION would use the numeric storage sequence
> >> rules
> >>
> >>
> >> (16.4.3.1 in Fortran 2003) for alignment.
> >>
> >> Nick,
> >> please can you explain, what "16.4.3.1 in Fortran 2003" tells
> >> about alignments.
> >> In SEQUENCE derived types and COMMON blocks,
> >> when I understand this correctly, then DOUBLE PRECISION
> >> must have also one numerical storage unit as alignment,
> >> i.e., with REAL=4 bytes, DOUBLE PRECISION alignment must be
> >> also 4 byte, i.e., exactly what the Intel compiler is doing
> >> in a SEQUENCE derived type or COMMON block.
> >>
> >> The implication is that the compiler issues nice "warnings" about
> >> misaligned data.
> >>
> >> Do I understand correctly, that the other compilers with
> >> REAL-size=4 byte and DOUBLE PRECISION size and alignment = 8 byte,
> >> do it wrong, i.e.not according to Fortran 2003 or 2008?
> >>
> >> Best regards
> >> Rolf
> >>
> >> ----- Original Message -----
> >>
> >>
> >> From: "N.M. Maclaren"< nmm1 at cam.ac.uk>
> >>
> >>
> >> To: "Fab Tillier"< ftillier at microsoft.com>
> >>
> >>
> >> Cc: "Rolf Rabenseifner"< rabenseifner at hlrs.de>, "Jeff Squyres"<
> >> jsquyres at cisco.com>, "Shinji Sumimoto"
> >>
> >>
> >> < s-sumi at labs.fujitsu.com>, "Hubert Ritzdorf"<
> >> hubert.ritzdorf at emea.nec.com>, "Howard Pritchard"< howardp at cray.com
> >>> ,
> >>
> >>
> >> "Brian Smith"< smithbr at us.ibm.com>, "Charles J Archer"<
> >> archerc at us.ibm.com>, "Rajeev Thakur"< thakur at mcs.anl.gov>,
> >>
> >>
> >> "Bill Long"< longb at cray.com>, "Bill Gropp"< wgropp at uiuc.edu>,
> >> "Richard Graham"< rlgraham at ornl.gov>, "Iain Bason"
> >>
> >>
> >> < iain.bason at oracle.com>
> >>
> >>
> >> Sent: Wednesday, September 28, 2011 10:26:37 AM
> >>
> >>
> >> Subject: RE: Serious problem/bug in MPI libraries with the
> >> alignment
> >> of MPI_DOUBLE_PRECISION
> >>
> >>
> >> On Sep 27 2011, Fab Tillier wrote:
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> It seems that all the a == 4 results come from using the Intel
> >>
> >>
> >>
> >>
> >> Fortran
> >>
> >>
> >>
> >>
> >> compiler suite. Given that Microsoft doesn't ship a Fortran
> >>
> >>
> >>
> >>
> >> compiler,
> >>
> >>
> >>
> >>
> >> mandating a == 8 doesn't put us in a position to succeed.
> >> Therefore,
> >>
> >>
> >>
> >>
> >> I
> >>
> >>
> >>
> >>
> >> cannot support such a proposal.
> >>
> >>
> >>
> >>
> >>
> >> I agree that Microsoft should not be forgotten, but it is not a
> >> major
> >>
> >>
> >> platform for MPI use, and the tail should not wag the dog. We have
> >> the
> >>
> >>
> >> ghastly example of C99 long long where some people claimed
> >> Microsoft
> >>
> >>
> >> compatibility required an incompatible change to the standard;
> >> largely
> >>
> >>
> >> as a result, the official C standard is no longer used as a
> >> portable
> >>
> >>
> >> programming base by a good half of the C-using community.
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> however, shouldn't a DOUBLE PRECISION BIND(C) derived type map to
> >>
> >>
> >>
> >>
> >> MPI_DOUBLE, not MPI_DOUBLE_PRECISION?
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> Can't we simply document that MPI_DOUBLE_PRECISION (and any Fortran
> >>
> >>
> >>
> >>
> >> types
> >>
> >>
> >>
> >>
> >> that need similar treatment) apply only to SEQUENCE derived types,
> >>
> >>
> >>
> >>
> >> while
> >>
> >>
> >>
> >>
> >> BIND(C) derived types should use the C types?
> >>
> >>
> >>
> >>
> >>
> >> A very good question. Instead of referring to SEQUENCE, it should
> >> say
> >>
> >>
> >> that MPI_DOUBLE_PRECISION would use the numeric storage sequence
> >> rules
> >>
> >>
> >> (16.4.3.1 in Fortran 2003) for alignment.
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> Slightly related to this, can't we PLEASE deprecate those horrible
> >>
> >>
> >> MPI_INTEGER8 etc. types? They have been a portability disaster for
> >>
> >>
> >> half a century, even among Fortran IV compilers (where they
> >> started)!
> >>
> >>
> >> MPI has provided some perfectly good mechanisms to use Fortran's
> >>
> >>
> >> selectable KINDs. The reason that it is related is that there is no
> >>
> >>
> >> guarantee that a compiler will give them the same alignment as
> >> another
> >>
> >>
> >> type even if they use the same storage, because they are
> >> non-standard
> >>
> >>
> >> and therefore any specification is the compiler's choice.
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> Regards,
> >>
> >>
> >> Nick Maclaren.
> >>
> >> --
> >> Dr. Rolf Rabenseifner . . . . . . . . . .. email
> >> rabenseifner at hlrs.de
> >> High Performance Computing Center (HLRS) . phone
> >> ++49(0)711/685-65530
> >> University of Stuttgart . . . . . . . . .. fax ++49(0)711 /
> >> 685-65832
> >> Head of Dpmt Parallel Computing . . .
> >> www.hlrs.de/people/rabenseifner
> >> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)
> >> _______________________________________________
> >> mpi3-fortran mailing list
> >> mpi3-fortran at lists.mpi-forum.org
> >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran
> >>
> >>
> >> _______________________________________________
> >> mpi3-fortran mailing list
> >> mpi3-fortran at lists.mpi-forum.org
> >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran
> >
> 
> --
> Bill Long longb at cray.com
> Fortran Technical Support & voice: 651-605-9024
> Bioinformatics Software Development fax: 651-605-9142
> Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
> 
> 
> _______________________________________________
> mpi3-fortran mailing list
> mpi3-fortran at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran

-- 
Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de
High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530
University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832
Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner
Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)