[MPI3 Fortran] Fwd: Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION
Rolf Rabenseifner
rabenseifner at hlrs.de
Wed Sep 28 02:14:06 CDT 2011
Fab and all,
your idea seems optimal, but ... (see later)!!!!
In detail:
We have the problem that the Intel ***Fortran*** compiler
answers with two different values a=4 and b=8.
Both are Fortran values, a=4 for Fortran SEQUENCE derived
types and b=8 for Fortran BIND(C) derived types.
(Fortran non-sequence and non-bind(C) derived types
are not really supported by MPI).
The MPI Forum has to decide which value should be used
in the example on page 581 of
https://svn.mpi-forum.org/trac/mpi-forum-web/attachment/ticket/229/mpi-report-F2008-2011-09-08-changeonlyplustickets.pdf
i.e., whether Alternative 1 or 2 is the correct one.
Your proposal (I name it Alternative 3) is that
the user has to choose the mapped C basic type.
This is a non-trivial mapping.
For MPI_INTEGER it is MPI_INT or MPI_LONG?
For MPI_INTEGER8 it is MPI_LONG_LONG?
What are we doing with the unnamed predefined
datatypes, produced by MPI_TYPE_CREATE_F90_COMPLEX /
_REAL / _INTEGER?
But your proposal inspires me to propose an
Alternative 4.1 and 4.2:
We need a routine that converts a given Fortran basic
named or unnamed datatype with
4.1) by default "SEQUENCE alignment" into an
unnamed predefined datatype with "BIND(C) alignment";
this works together with Alternative 1 for page 581.
MPI_F2003_CONVERT_SEQ_TO_BIND_C(IN seq_datatype, OUT bind_c_datatype)
4.2) by default "BIND(C) alignment" into an
unnamed predefined datatype with "SEQUENCE alignment";
this works together with Alternative 2 for page 581.
MPI_F2003_CONVERT_BIND_C_TO_SEQ(IN bind_c_datatype, OUT seq_datatype)
Best regards
Rolf
----- Original Message -----
> From: "Fab Tillier" <ftillier at microsoft.com>
> To: "Rolf Rabenseifner" <rabenseifner at hlrs.de>, "Jeff Squyres" <jsquyres at cisco.com>, "Shinji Sumimoto"
> <s-sumi at labs.fujitsu.com>, "Hubert Ritzdorf" <hubert.ritzdorf at emea.nec.com>, "Howard Pritchard" <howardp at cray.com>,
> "Brian Smith" <smithbr at us.ibm.com>, "Charles J Archer" <archerc at us.ibm.com>, "Rajeev Thakur" <thakur at mcs.anl.gov>,
> "Bill Long" <longb at cray.com>, "Bill Gropp" <wgropp at uiuc.edu>, "Richard Graham" <rlgraham at ornl.gov>
> Cc: "N.M. Maclaren" <nmm1 at cam.ac.uk>, "Iain Bason" <iain.bason at oracle.com>
> Sent: Wednesday, September 28, 2011 12:35:53 AM
> Subject: RE: Serious problem/bug in MPI libraries with the alignment of MPI_DOUBLE_PRECISION
> Hi Rolf,
>
> It seems that all the a == 4 results come from using the Intel Fortran
> compiler suite. Given that Microsoft doesn't ship a Fortran compiler,
> mandating a == 8 doesn't put us in a position to succeed. Therefore, I
> cannot support such a proposal.
>
> however, shouldn't a DOUBLE PRECISION BIND(C) derived type map to
> MPI_DOUBLE, not MPI_DOUBLE_PRECISION?
>
Alternative 3:
> Can't we simply document that MPI_DOUBLE_PRECISION (and any Fortran
> types that need similar treatment) apply only to SEQUENCE derived
> types, while BIND(C) derived types should use the C types?
>
> -Fab
>
> Rolf Rabenseifner wrote on Tue, 27 Sep 2011 at 14:38:45
>
> > Dear all,
> >
> > thank you very much for the fast answers.
> > The result is maximal bad,
> > i.e., about 50% of the MPIs use SEQUENCE alignments and other
> > about 50% use BIND(C) based calculation of the alignment.
> >
> > Here the results (also attached as summary_out.txt).
> > Please read carefully my proposal after this summary sheet.
> >
> > In all protocols:
> > -----------------
> > Size of one Fortran REAL = 4
> > Size of one Fortran DOUBLE PRECISION = 8
> > Results:
> > --------
> >
> > a / b / c = Alignments of:
> > - DOUBLE PRECISION within a SEQUENCE derived type = a
> > - DOUBLE PRECISION within a BIND(C) derived type = b
> > - MPI_DOUBLE_PRECISION (= k_i in MPI-2.2 p.78:45) = c
> >
> > A) The 4 / 8 / 4 group = alignment of MPI_DOUBLE_PRECISION is
> > correct for (old-style) SEQUENCE derived types
> >
> > JeffSquyres_OpenMP_linux-x86-64-ifort.txt: Intel + OpenMPI on x86-64
> > Results: 4 / 8 / 4
> > FabTillier_Intel+MicrosoftMPI_out.txt: Intel + MS-MPI on ?
> > Results: 4 / 8 / 4
> >
> > B) The 4 / 8 / 8 group = alignment of MPI_DOUBLE_PRECISION
> > is correct for (modern) BIND(C) derived types
> >
> > Rajeev_mpich_out.ifort.x86-64.txt: Intel + mpich2 on x86-64
> > Results: 4 / 8 / 8
> > Rolf_asama+cray_output.txt: Intel + NEC-MPI? on asama
> > Results: 4 / 8 / 8
> > BillLong_Cray_5compiler_out.txt: Intel + CrayMPI on Cray XE6
> > Results: 4 / 8 / 8
> > BrianSmith-IBM_outlog-bgp-xl.txt: Intel + IBM-XL on BGP
> > Results: 4 / 8 / 8
> > BrianSmith-IBM_outlog-xlf.txt: xlf + IBM-XL on ?
> > Results: 4 / 8 / 8
> >
> > C) No problems because all DOUBLE PRECISION alignments
> > are identical = 8
> >
> > BillLong_Cray_5compiler_out.txt: PGI + CrayMPI on Cray XE6
> > Results: 8 / 8 / 8
> > BillLong_Cray_5compiler_out.txt: Cray + CrayMPI on Cray XE6
> > Results: 8 / 8 / 8
> > BillLong_Cray_5compiler_out.txt: gfortran + CrayMPI on Cray XE6
> > Results: 8 / 8 / 8
> > BillLong_Cray_5compiler_out.txt: pathscale + CrayMPI on Cray XE6
> > Results: 8 / 8 / 8
> > BrianSmith-IBM_outlog-gcc.txt: gfortran + IBM-XL on BGQ
> > Results: 8 / 8 / 8
> > JeffSquyres_OpenMP_linux-x86-64-gfortran.txt: gfortran + OpenMPI on x86-64
> > Results: 8 / 8 / 8
> > JeffSquyres_OpenMP_osx-x86-84-gfortran.txt: gfortran + OpenMPI on osx-x86-64
> > Results: 8 / 8 / 8
> > Rajeev_gnu_laptop_out.txt: gfortran + mpich2 on laptop
> > Results: 8 / 8 / 8
> > Rajeev_mpich_out.gfortran.x86-64.txt: gfortran + mpich2 on x86-64
> > Results: 8 / 8 / 8
> >
> > D) No problems because all DOUBLE PRECISION alignments are
> > identical = 4
> >
> > Rajeev_mpich_out.gfortran.ia32.txt: gfortran + mpich2 on ia32
> > Results: 4 / 4 / 4
> > Rajeev_mpich_out.ifort.ia32.txt: Intel + mpich2 on ia32
> > Results: 4 / 4 / 4
> >
> > Proposal:
> > ---------
> >
> > The goal of a standard is the possibility to write portable code.
> > Therefore the answer should be standardized and not implementation-
> > specific, or in other words, not defining it is the worst solution.
> >
> > I expect only a few Fortran codes are using arrays of derived types
> > in Fortran.
> > The oldest library is mpich.
> > Therefore I would propose to standardize Fortran alignments are
> > based
> > on BIND(C) which should be the same as in C.
> >
> > This would imply that IBM-XL and mpich2 and derivatives need no
> > change.
> > But it also implies that the younger OpenMPI and Microsoft-MPI
> > need a change. These implementation may add an environment switch
> > to keep the old behavior if this switch is set,
> > but I would expect that there is nearly no user
> > who needs or want to use this switch.
> >
> > Fab and Jeff, could you live with this proposal?
> >
> > Best regards
> > Rolf
> >
> > PS: Additional remarks:
> > Yes, some compiler detect that DOUBLE PR. on 4 byte alignment is
> > inefficient, but this is only an information. Other compilers
> > detect, that we use DOUBLE PRECISION in BIND(C) derived types
> > instead of using the C double through these new intrinsic C
> > types in
> > Fortran. This is also okay and all compiler work as expected,
> > they
> > use the C alignment.
> >
> >
> >> Rolf Rabenseifner wrote on Tue, 27 Sep 2011 at 09:14:33
> >>
> >>> In my output from Cray, the compilers
> >>> - gnu
> >>> - pgi
> >>> - cray
> >>> all use 8 byte double alignment in C and Fortran.
> >>> ==> no problem with those compilers on the tested platform.
> >>> Maybe we have differences between IA64 and 32 architectures
> >>> for the same compiler.
> >>>
> >>> Important is to check MPI with those compilers that have
> >>> Fortran 4 byte double precision alignment (e.g. Intel).
> >>>
> >>> Best regards
> >>> Rolf
> >>>
> >
> >>>> On Sep 27, 2011, at 10:21 AM, Rolf Rabenseifner wrote:
> >>>>
> >>>>> Dear all, (now with attachment)
> >>>>>
> >>>>> as far as I know, you represent some MPI libraries that are
> >>>>> not directly based on another one in the list:
> >>>>> - mpich2: Rajeev
> >>>>> - OpenMPI: Jeff
> >>>>> - IBM: Rich Treumann
> >>>>> - NEC: Hubert
> >>>>> - Fujitsu: Shinji Sumimoto
> >>>>> - Microsoft: Fab
> >>>>> (Which independent library is missing on this list?)
> >>>>> (Who of the list uses already mich2 or OpenMPI as bases
> >>>>> or does not provide a Fortran binding,
> >>>>> and can be therefore removed from this list?)
> >>>>>
> >>>>> The problem is simple and has at the end only three numbers:
> >>>>>
> >>>>> Alignments of:
> >>>>> - DOUBLE PRECISION within a SEQUENCE derived type = 4 (e.g.
> >>>>> with
> >>>>> Intel)
> >>>>> - DOUBLE PRECISION within a BIND(C) derived type = 8
> >>>>> - MPI_DOUBLE_PRECISION (= k_i in MPI-2.2 p.78:45) = 4 or 8
> >>>>> The details:
> >>>>> Which is the alignment of Fortran DOUBLE PRECISION according to
> >>>>> k_i on MPI-2.2, page 78 line 45 and page 96 line 42.
> >>>>>
> >>>>> Since 1977 (Fortran 77) in Fortran COMMON blocks and 1990
> >>>>> (Fortran
> >>>>> 90)
> >>>>> in Fortran SEQUENCE derived types (which have the same memory
> >>>>> layout
> >>>>> according to the SEQUENCE-rules), the MPI alignment and the
> >>>>> Fortran
> >>>>> alignment in such constructs should be the same, e.g., the Intel
> >>>>> compiler has alignment 4 !!!
> >>>>>
> >>>>> This may be significantly different to C, where Intel uses
> >>>>> alignment
> >>>>> 8!
> >>>>>
> >>>>> With Fortran 2003, we got the Fortran BIND(C) derived types.
> >>>>> Here, a Fortran DOUBLE PRECISION has with Intel's compiler
> >>>>> the alignment 8.
> >>>>>
> >>>>> As far as I understand, mpich2 and Cray produce following
> >>>>> results
> >>>>> with
> >>>>> an Intel compiler:
> >>>>>
> >>>>> Alignments of:
> >>>>> - DOUBLE PRECISION within a SEQUENCE derived type = 4
> >>>>> - DOUBLE PRECISION within a BIND(C) derived type = 8
> >>>>> - MPI_DOUBLE_PRECISION (= k_i in MPI-2.2 p.78:45) = 8
> >>>>>
> >>>>> This is definitely wrong compared to MPI-1.1 and MPI-2.0
> >>>>> but may be helpful for MPI-3.0 when we like to switch
> >>>>> to a definition that is based on BIND(C).
> >>>>>
> >>>>> Question:
> >>>>> What is the output of the attached test program when running
> >>>>> with
> >>>>> - your MPI library
> >>>>> - and your compiler
> >>>>> - and with Intel's compiler
> >>>>>
> >>>>> Please can you send me the output of such mpiruns?
> >>>>>
> >>>>> Thanks in advance and best regards
> >>>>> Rolf
> >>>>>
> >>>>> PS: We need not to care about 8/8/8.
> >>>>> Only 4/8/x causes problems:
> >>>>> If all those have x=8 then I'll propose to
> >>>>> adopt the MPI-3.0 to the reality.
> >>>>> If we have some implementations with 4/8/8 and
> >>>>> others with 4/8/4 then we have a serious problem
> >>>>> because only one answer can be correct,
> >>>>> and the question would be whether the historical
> >>>>> answer (x=4) or the modern answer (x=8)
> >>>>> should be chosen.
> >
> >
--
Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de
High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530
University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832
Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner
Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)
More information about the mpiwg-fortran
mailing list