[MPI3 Fortran] Fortran proposal w.r.t. BIND(C)/logical/linker symbols/etc. (take 2)

[Rolf Rabenseifner]

Dear Bill and all,

----- Original Message -----
> From: "Bill Long" <longb at cray.com>
> To: "Rolf Rabenseifner" <rabenseifner at hlrs.de>
> Cc: "MPI-3 Fortran working group" mpi-forum.org>
> Sent: Monday, June 3, 2013 3:03:00 PM
> Subject: Re: [MPI3 Fortran] Fortran proposal w.r.t. BIND(C)/logical/linker symbols/etc. (take 2)
> Hi Rolf,
> 
> C19 looks much better. Thanks. I have thought of a couple more
> observations, though maybe not needing added text changes.
> 
> 1) In the case of the mpif.h module, if option 3d is used, then the
> header file will have to include generic interface blocks of the form
> 
> interface MPI_Send
> subroutine mpi_send_fts ( <args> )
> args>
> end subroutine mpi_send_fts
> end interface
> 
> and similarly for the PMPI version. Without these the user calls to
> MPI_Send would not not result in the creation of these new linkage
> names. This seems like a substantial change from the current form of
> mpif.h modules, and substantially increases their size and effect on
> compile time. There would be some temptation to just ignore the 3d
> option in the case of the mpif.h file.

Yes, therefore MPI_SUBARRAYS_SUPPORTED==.TRUE. in mpif.h
is a high-quality option, it is not mandatory.


> 2) The requirement for the visibility of the legacy style linkage
> names will require interface blocks similar to (1) above in the
> tool-modified
> mpi_f08 module. This is doable, but a significant effort compared to
> the very small fraction of MPI users who need this. I would expect
> this to be quite low on the implementation priority list. Note that tools
> users who are able to intercept the C names (1a - 1d) would not be
> affected by an implementation delay of this sort.

Yes, this compromise is needed to keep some backward compatibility.

 
> 3) The C implementation allows for certain routines to be implemented
> as macros (2.6.4, pages 19-20). In particular, the type conversion
> routines. In that case, there is no external PMPI version. It would be
> reasonable to allow the same exemption in the case that these routines
> are implemented by Fortran module procedures (which, from the end
> result of compilation, would have an effect much like a C macro). I assume
> that the tools people would not be harmed if these could not be
> intercepted, as that is already the case for C anyway.

The functions in 17.2.4 do not have Fortran interfaces.
They exist only in C.

The macro-allowance for MPI_Wtime and Wtick is only for C.

For MPI_Wtime and MPI_Wtick in Fortran, the usual rules apply.
No macro exception.
 

> 4) The new Fortran interface specifies a set of defined operators.
> (for example 601:38-40). The functions that implement these defined
> operators would be private in the module where the types are defined.
> I assume these functions are exempt from the PMPI duplication
> requirements, since they do not have names specified in the document.

Yes.

> Observations 3 and 4 suggest possible extra wording in the paragraph
> requiring the PMPI routines.

Yes.
The current text is only the binding text. Additional advices to users
and implementors should be added later.

I hope the binding part is now okay and all can agree on it.

I also hope, that the tools people have some ideas what they really
want as additional information that they need not to examine the 
binary module and library.


> I will not be attending the meeting this week. Steve Oyanagi will be
> attending for Cray. He can always ask me questions, and I will be
> available to answer emails during the week. I could also be available
> to participate in a webex meeting if people think it would be helpful for
> a particular discussion.
> 
> Cheers,
> Bill

Best regards
Rolf
> 
> 
> 
> On 6/2/13 4:16 PM, Rolf Rabenseifner wrote:
> > Hi Bill and all,
> >
> > thank you for your detailed review and hints.
> > Answers below.
> >
> > On June 2, 2013 4:10 AM, Bill Long wrote:
> >> See comments below to be sure I understand things correctly. Also
> >> some
> >> corrections and questions, particularly about an apparent
> >> contradiction.
> >>
> >>
> >> On 6/1/13 2:49 PM, Rolf Rabenseifner wrote:
> >>> 17.1.5.2 External linkage names and calling conventions
> >>>
> >>> A Fortran call to an MPI routine shall result in a call to a
> >>> procedure
> >>> with one of the following external linkage names and calling
> >>> conventions.
> >>> MPI_Send is used as an example. Case is significant in the names.
> >>>
> >>>       External Linkage name Calling convention
> >>>       --------------------- ------------------
> >>>
> >>> 1a) MPI_Send C interface and arguments, as in A.2.
> >>>
> >>> 1b) MPI_Send_cdesc Same as (1) except that choice arguments
> >>>                                     are pointer to CFI_cdesc_t
> >>>                                     rather than
> >>>                                void *.
> >>>
> >>> 1c) MPI_Send_f08cb Same as (1) except that the callback dummy
> >>> argument
> >>>                                 is a Fortran callback according to
> >>>                                 the A.1.3
> >>>                                 Fortran 2008 Bindings with the
> >>>                                 mpi_f08 module.
> >>>
> >>> 1d) MPI_Send_fcb Same as (1) except that the callback dummy
> >>> argument
> >>>                                 is a Fortran callback according to
> >>>                                 the A.1.3
> >>>                                 Fortran Bindings with mpif.h or
> >>>                                 the
> >>>                                 mpi module.
> >>
> >> 1c and 1d are only for the limited number of routines that have a
> >> callback procedure as an argument. These are all C functions (at
> >> least
> >> in the abstract), so there is no issue about derived-type versus
> >> INTEGER
> >> handles. However, it is possible for the routine to have a choice
> >> argument? If the routine has a choice argument then 1c will have a
> >> pointer to descriptor argument for that argument and 1d will have a
> >> void
> >> * choice argument. Correct?
> >
> >
> > In 1c and 1d, the pointers to user-written C callbacks are
> > substituted by
> > pointers to user-written Fortran callbacks according to
> >   - 1c) mpi_f08, MPI-3.0 p678:11 - p680:34, and
> >   - 1d) mpi + mpif.h, p680:36 - p682:37.
> >
> > 1c) have Fortran derived type handles, 1d) has Fortran INTEGER
> > handles.
> > Only 1c) p678:18 and 1d) p680:42 are choice buffers, but both
> > defined
> > without TS 29113!!! See also p183:45 - p184:7 and especially
> > p187:11-25!!!
> >
> > I expect that we should be more precise:
> >
> > 1c) MPI_Send_f08cb Same as (1) except that the callback dummy
> > argument
> >                                is a "void *" pointer to a Fortran
> >                                callback according
> >                                to A.1.3, subsection "Fortran 2008
> >                                Bindings with the mpi_f08 module".
> >
> > 1d) MPI_Send_fcb Same as (1) except that the callback dummy argument
> >                                is a "void *" pointer to a Fortran
> >                                callback according
> >                                to A.1.3, subsection "Fortran
> >                                Bindings with mpif.h or the mpi
> >                                module".
> >
> >
> >
> >>> 2a) mpi_send_f08_ Fortran interface and arguments, as in A.3.
> >>>                                 In routines with a choice buffer,
> >>>                                 the
> >>>                                 choice buffer
> >>>                                 dummy argument is implemented with
> >>>                                 non-standard
> >>>                                 extensions like !$PRAGMA
> >>>                                 IGNORE_TKR.
> >>
> >> I would prefer that the "non-standard extensions ... TKR" be worded
> >> differently. It is reasonable for the interface to declare the
> >> choice
> >> argument TYPE(*),DIMENSION(*) if the TS is supported, and that
> >> would
> >> not be "non-standard". What is actually required (as opposed to
> >> implementation suggestions) is that the choice argument be declared
> >> in
> >> a way compatible with (void *).
> >
> > This part is for the case that TS 29113 is not available, see
> > MPI-3.0 p610:34-36 and p611:44-45 and the important sentence
> > p613:26-30 (please remember, that MPI_SUBARRAYS_SUPPORTED==.TRUE.
> > requires that 2b) is used for non-blocking routines with
> > choice buffer arguments).
> >
> > I can mention, that this should be identical to TYPE(*),DIMENSION(*)
> > of TS 29113.
> >
> >> The beginning of the description needs a change of "in A.3. In
> >> routines"
> >> -> "in A.3, except that in routines". A.3 specifies the descriptor
> >> arguments, and this text is making an exception to the A.3 text.
> >
> > Yes. changed.
> >
> >
> >> A.3 also specifies OPTIONAL for the ierror argument, implying TS
> >> support. Does 2a) also need an exception to remove the OPTIONAL
> >> attribute in the interface?
> >
> > No, because we removed all BIND(C) from the mpi_f08 Interfaces.
> > Without BIND(C), the OPTIONAL is available since Fortran 90,
> > i.e., no Problem.
> >
> >
> >>> 2b) mpi_send_f08ts_ Fortran interface and arguments, as in A.3,
> >>>                                 but only for routines with a
> >>>                                 choice
> >>>                                 buffer argument
> >>>                                 and this dummy argument is
> >>>                                 implemented
> >>>                                 with TYPE(*), DIMENSION(..).
> >>
> >> A.3 already specifies TYPE(*),DIMENSION(..) in the interfaces with
> >> choice arguments. So, the description here can end with the A.3 in
> >> the
> >> first line. Since the OPTIONAL attribute used throughout A.3
> >> requires
> >> basically all of the routines to be in this category, the "but only
> >> for
> >> routines" is confusing - at least to me.
> >
> > No. The idea is that MPI_COMM_RANK, etc. (i.e. routines without
> > buffer)
> > are always done in mpi_f08 according to 2a).
> > 2b) is needed only for TYPE(*), DIMENSION(..).
> >
> >
> >> Just from the names, I expected that mpi_send_f08_ would have
> >> {derived-type handles, void * choice arguments, and no OPTIONAL
> >> attribute} and mpi_send_f08ts_ would have {derived-type handles,
> >> descriptor for choice, and OPTIONAL arguments} - i.e. as in A.3.
> >
> > No.
> > 2a) and 2b) have {derived-type handles, and OPTIONAL ierror}.
> > 2a) mpi_send_f08_ has "void *" choice arguments.
> > 2b) mpi_send_f08ts_ has TYPE(*),DIMENSION(..) arguments according
> >      to TS 29113 (therefore the postfix "ts"
> >
> >
> >>> 3a) mpi_send_ Fortran interface and arguments, as in A.4.
> >>>                                 In routines with a choice buffer,
> >>>                                 the
> >>>                                 choice buffer
> >>>                                 dummy argument is implemented with
> >>>                                 non-standard
> >>>                                 extensions like !$PRAGMA
> >>>                                 IGNORE_TKR.
> >>>
> >>> 3b) mpi_send_fts_ Fortran interface and arguments, as in A.4,
> >>>                                 but only for routines with a
> >>>                                 choice
> >>>                                 buffer argument
> >>>                                 and this dummy argument is
> >>>                                 implemented
> >>>                                 with TYPE(*), DIMENSION(..).
> >>
> >> I see missing details here, similar to 2 above. Basically all of
> >> the
> >> same comments, except that the handles are declared INTEGER here.
> >
> > The major differences between 2a+b) and 3a+b) are:
> > - INTEGER handles in 3a+b).
> > - IERROR is not OPTIONAL in 3a+b).
> > - The callbacks are just EXTERNAL in 3a+b)
> >    and have different Abstract interface definitions,
> >    see all routines with callback function dummy arguments.
> >    (The predefined callbacks do not cause additional differences).
> > - Some differences for buffer address OUT dummy arguments,
> >    see MPI_ALLOC_MEM, MPI_WIN_ALLOCATE, MPI_WIN_ALLOCATE_SHARED,
> >    MPI_WIN_SHARED_QUERY, and MPI_GET_ADDRESS.
> > - Other differences in MPI_BUFFER_DETACH.
> >
> > I'm not sure whether this list is complete.
> >
> > Important is also that TYPE(*),DIMENSION(..) is also allowed
> > (and recommended in the mpi module, see p602:24-28)
> > in the mpi module and mpif.h . Therefore 3b) is needed.
> >
> >
> >>> For each external linkage name supported, a second entry point
> >>> with
> >>> the same calling characteristics shall be supplied, expect that
> >>> the
> >>> name is modified by prefixing with the letter "p" in the same case
> >>> as the leading "m". For example, PMPI_Send and pmpi_send_.
> >>>
> >>> [No exceptions according to p555:35-37.]
> >>>
> >>> The external linkage names as mpi_send_, or mpi_send_fts_
> >>> are examples for the external linkage name that the Fortran
> >>> compiler
> >>> chooses for the Fortran routine names MPI_Send, or MPI_Send_fts.
> >>
> >> I assume that the intent here is that these are examples and that
> >> the
> >> real requirement is that the external linkage names are the ones
> >> generated by the particular Fortran compiler for MPI_Send and
> >> MPI_Send_fts.
> >
> > Yes. I updated the text a little bit into:
> >
> >   The external linkage names as mpi_send_f08_, mpi_send_f08ts_,
> >   mpi_send_,
> >   and mpi_send_fts_ are examples for the external linkage name that
> >   the
> >   particular Fortran compiler chooses for the Fortran routine names
> >   MPI_Send_f08, MPI_Sendf08ts, MPI_Send, and MPI_Send_fts.
> >
> > I hope that this is a bit clearer.
> >
> >
> >>> Using the mpi_f08 module, a Fortran call to MPI_Send must be
> >>> implemented through a call one of the linker names defined in 1a)
> >>> - 2b);
> >>> if linker names in 1a) - 1d) are used, then an additional library
> >>> must be provided that uses only 2a) and 2b) in the mpi_f08 module.
> >>
> >> Actually, 1d is not for the mpi_f08 module. It should be omitted
> >> from
> >> the ranges in the paragraph above.
> >
> > Yes.
> >>
> >> I assume you don't mean "in the mpi_f08 module" here since you
> >> would
> >> not get routines with names like these in a module.
> >> And hopefully the
> >> "only" is also wrong since otherwise this would lead to a huge
> >> amount
> >> of unnecessary duplication. The obvious way to implement these
> >> routines is in a file outside the module, with
> >>
> >>    subroutine mpi_send_f08 ( <args> )
> >>      use mpi_f08
> >
> > Note: This mpi_f08 need not to be identical the the mpi_f08
> >        provided to the user!
> >
> >>      args>
> >>      call mpi_send (<args> )
> >>    end subroutine mpi_send_f08
> >>
> >> This ensures correct mpi_f08 behavior and allows the maintenance of
> >> the
> >> actual Fortran MPI_Send to occur in only one place. And, of course,
> >> avoid all of the duplicated work of providing two complete
> >> implementations - one in the old pre-interop style, and one using
> >> modules.
> >
> >
> > Sorry, I understood that the compromise is as follows:
> >
> >   Using the mpi_f08 module, a Fortran call to MPI_Send must be
> >   implemented
> >   through a call one of the linker names defined in 1a, 1b, 1c, 2a ,
> >   or 2b;
> >   if linker names in 1a - 1c are used, then an additional library
> >   together
> >   with an mpi_f08 module must be provided that guarantees that
> >   a Fortran call to an MPI routine results in a call to a procedure
> >   according to 2a or 2b.
> >
> > You can do your wrapper as described above, as long as the module
> > mpi_f08 you use here is identical to your optimized Version
> > and the mpi_f08 module that you Hand over to the user
> > in the not-optimized MPI is different:
> >
> > The non-optimized mpi_f08 module defines the mpi routines without
> > CONTAINS.
> >
> > This text does not say anything about internals, i.e., it is still
> > allowed that internally 1a-c is used as it may
> > be done with your wrappers if the optimized version uses 1a-c.
> >
> >
> >>> Using the mpi module or mpif.h, a Fortran call to MPI_Send must
> >>> be implemented through a call to one of the linker names defined
> >>> in
> >>> 3a) and 3b).
> >>>
> >>
> >> This is hopefully wrong, since it contradicts 1d which is
> >> specifically
> >> for the mpi module. The current wording seems to suggest that a
> >> module
> >> implementation using C interoperability is prohibited for the mpi
> >> module, which (hopefully) is not the case. This needs to read
> >> similar
> >> to the previous paragraph, such as
> >>
> >> "Using the mpi module, a Fortran call to MPI_Send must be
> >> implemented
> >> through one of the linker names 1a), 1d), 3a), or 3b). If 1a) and
> >> 1d)
> >> are used, then routines with external linkage names as specified in
> >> 3a)
> >> and 3b) must also be provided."
> >
> > Yes, there is no need to do the compromise only for the mpi_f08
> > module. The compromise is also good for the mpi module:
> >
> >    Using the mpi module, a Fortran call to MPI_Send must be
> >    implemented
> >    through a call one of the linker names defined in 1a, 1b, 1d, 3a,
> >    or 3b;
> >    if linker names in 1a, 1b, and 1d are used, then an additional
> >    library
> >    together with an mpi module must be provided that guarantees that
> >    a Fortran call to an MPI routine results in a call to a procedure
> >    according to 3a or 3b.
> >
> > It cannot be done for mpif.h, because it cannot contain a module.
> > Therefore, only 3a and 3b is possible:
> >
> >    Using mpif.h, a Fortran call to MPI_Send must be implemented
> >    through a call one of the linker names defined in 3a, or 3b.
> >
> >
> >> Do the tools people really need that second sentence? I was under
> >> the
> >> impression they would be happy with dealing only with the C
> >> functions.
> >
> > We must make also the users happy who used in the past some PMPI
> > Interfaces for some reasons!
> > This group is not identical to the "tools people".
> >
> > This is the background for this compromise.
> >
> >> [Although is this, and similarly for the mpi_f08 module case,
> >> actually
> >> true? It seems that this requirement is only relevant for the tools
> >> developers, and that their real need is that the PMPI_xxx versions
> >> be
> >> available.]
> >
> >
> > Both is needed: the possibility for interception and the PMPI
> > routines.
> >
> >> Cheers,
> >> Bill
> >
> >>> 17.1.5.3 Tools Support
> >
> > I changed my proposed Header file Name into pmpif.h
> >
> >>> 17.1.5.3 Execution Environments
> >
> > I removed the now empty section.
> >
> >
> > I hope the new version c19.comments.txt solves most of your issues
> > and serves also the issues of OpenMPI as defined in the compromise.
> >
> > I cannot come to the June Meeting but I hope that this text
> > is a good Input for the Fortran WG Meeting there.
> > Bill, do you come to the June meeting tomorrow?
> >
> > Best regards
> > Rolf
> >>
> >>
> >> --
> >> Bill Long longb at cray.com
> >> Fortran Technical Support & voice: 651-605-9024
> >> Bioinformatics Software Development fax: 651-605-9142
> >> Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
> >
> 
> --
> Bill Long longb at cray.com
> Fortran Technical Support & voice: 651-605-9024
> Bioinformatics Software Development fax: 651-605-9142
> Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101

-- 
Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de
High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530
University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832
Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner
Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)