[MPI3 Fortran] Fortran proposal w.r.t. BIND(C)/logical/linker symbols/etc. (take 2)

[Bill Long]

Hi Rolf,

C19 looks much better. Thanks.  I have thought of a couple more 
observations, though maybe not needing added text changes.

1) In the case of the mpif.h module, if option 3d is used, then the 
header file will have to include generic interface blocks of the form

    interface MPI_Send
        subroutine mpi_send_fts ( <args> )
        <declarations for args>
        end subroutine mpi_send_fts
    end interface

and similarly for the PMPI version.  Without these the user calls to 
MPI_Send would not not result in the creation of these new linkage 
names.  This seems like a substantial change from the current form of 
mpif.h modules, and substantially increases their size and effect on 
compile time.  There would be some temptation to just ignore the 3d 
option in the case of the mpif.h file.

2) The requirement for the visibility of the legacy style linkage names 
will require interface blocks similar to (1) above in the tool-modified 
mpi_f08 module.  This is doable, but a significant effort compared to 
the very small fraction of MPI users who need this.  I would expect this 
to be quite low on the implementation priority list.  Note that tools 
users who are able to intercept the C names (1a - 1d) would not be 
affected by an implementation delay of this sort.

3) The C implementation allows for certain routines to be implemented as 
macros (2.6.4, pages 19-20). In particular, the type conversion 
routines.  In that case, there is no external PMPI version.  It would be 
reasonable to allow the same exemption in the case that these routines 
are implemented by Fortran module procedures (which, from the end result 
of compilation, would have an effect much like a C macro).  I assume 
that the tools people would not be harmed if these could not be 
intercepted, as that is already the case for C anyway.

4) The new Fortran interface specifies a set of defined operators. (for 
example 601:38-40).  The functions that implement these defined 
operators would be private in the module where the types are defined.  I 
assume these functions are exempt from the PMPI duplication 
requirements, since they do not have names specified in the document.

Observations 3 and 4 suggest possible extra wording in the paragraph 
requiring the PMPI routines.

I will not be attending the meeting this week.  Steve Oyanagi will be 
attending for Cray.  He can always ask me questions, and I will be 
available to answer emails during the week. I could also be available to 
participate in a webex meeting if people think it would be helpful for a 
particular discussion.

Cheers,
Bill



On 6/2/13 4:16 PM, Rolf Rabenseifner wrote:
> Hi Bill and all,
>
> thank you for your detailed review and hints.
> Answers below.
>
> On June 2, 2013 4:10 AM, Bill Long wrote:
>> See comments below to be sure I understand things correctly. Also some
>> corrections and questions, particularly about an apparent
>> contradiction.
>>
>>
>> On 6/1/13 2:49 PM, Rolf Rabenseifner wrote:
>>> 17.1.5.2 External linkage names and calling conventions
>>>
>>> A Fortran call to an MPI routine shall result in a call to a
>>> procedure
>>> with one of the following external linkage names and calling
>>> conventions.
>>> MPI_Send is used as an example. Case is significant in the names.
>>>
>>>       External Linkage name Calling convention
>>>       --------------------- ------------------
>>>
>>> 1a) MPI_Send C interface and arguments, as in A.2.
>>>
>>> 1b) MPI_Send_cdesc Same as (1) except that choice arguments
>>>                                     are pointer to CFI_cdesc_t rather than
>>>                                void *.
>>>
>>> 1c) MPI_Send_f08cb Same as (1) except that the callback dummy
>>> argument
>>>                                 is a Fortran callback according to
>>>                                 the A.1.3
>>>                                 Fortran 2008 Bindings with the
>>>                                 mpi_f08 module.
>>>
>>> 1d) MPI_Send_fcb Same as (1) except that the callback dummy argument
>>>                                 is a Fortran callback according to
>>>                                 the A.1.3
>>>                                 Fortran Bindings with mpif.h or the
>>>                                 mpi module.
>>
>> 1c and 1d are only for the limited number of routines that have a
>> callback procedure as an argument. These are all C functions (at least
>> in the abstract), so there is no issue about derived-type versus
>> INTEGER
>> handles. However, it is possible for the routine to have a choice
>> argument? If the routine has a choice argument then 1c will have a
>> pointer to descriptor argument for that argument and 1d will have a
>> void
>> * choice argument. Correct?
>
>
> In 1c and 1d, the pointers to user-written C callbacks are substituted by
> pointers to user-written Fortran callbacks according to
>   - 1c) mpi_f08, MPI-3.0 p678:11 - p680:34, and
>   - 1d) mpi + mpif.h, p680:36 - p682:37.
>
> 1c) have Fortran derived type handles, 1d) has Fortran INTEGER handles.
> Only 1c) p678:18 and 1d) p680:42 are choice buffers, but both defined
> without TS 29113!!! See also p183:45 - p184:7 and especially p187:11-25!!!
>
> I expect that we should be more precise:
>
> 1c) MPI_Send_f08cb              Same as (1) except that the callback dummy argument
>                                is a "void *" pointer to a Fortran callback according
>                                to A.1.3, subsection "Fortran 2008 Bindings with the mpi_f08 module".
>
> 1d) MPI_Send_fcb              Same as (1) except that the callback dummy argument
>                                is a "void *" pointer to a Fortran callback according
>                                to A.1.3, subsection "Fortran Bindings with mpif.h or the mpi module".
>
>
>
>>> 2a) mpi_send_f08_ Fortran interface and arguments, as in A.3.
>>>                                 In routines with a choice buffer, the
>>>                                 choice buffer
>>>                                 dummy argument is implemented with
>>>                                 non-standard
>>>                                 extensions like !$PRAGMA IGNORE_TKR.
>>
>> I would prefer that the "non-standard extensions ... TKR" be worded
>> differently. It is reasonable for the interface to declare the choice
>> argument TYPE(*),DIMENSION(*) if the TS is supported, and that would
>> not be "non-standard". What is actually required (as opposed to
>> implementation suggestions) is that the choice argument be declared in
>> a way compatible with (void *).
>
> This part is for the case that TS 29113 is not available, see
> MPI-3.0 p610:34-36 and p611:44-45 and the important sentence
> p613:26-30 (please remember, that MPI_SUBARRAYS_SUPPORTED==.TRUE.
> requires that 2b) is used for non-blocking routines with
> choice buffer arguments).
>
> I can mention, that this should be identical to TYPE(*),DIMENSION(*)
> of TS 29113.
>
>> The beginning of the description needs a change of "in A.3. In
>> routines"
>> -> "in A.3, except that in routines". A.3 specifies the descriptor
>> arguments, and this text is making an exception to the A.3 text.
>
> Yes. changed.
>
>
>> A.3 also specifies OPTIONAL for the ierror argument, implying TS
>> support. Does 2a) also need an exception to remove the OPTIONAL
>> attribute in the interface?
>
> No, because we removed all BIND(C) from the mpi_f08 Interfaces.
> Without BIND(C), the OPTIONAL is available since Fortran 90,
> i.e., no Problem.
>
>
>>> 2b) mpi_send_f08ts_ Fortran interface and arguments, as in A.3,
>>>                                 but only for routines with a choice
>>>                                 buffer argument
>>>                                 and this dummy argument is
>>>                                 implemented
>>>                                 with TYPE(*), DIMENSION(..).
>>
>> A.3 already specifies TYPE(*),DIMENSION(..) in the interfaces with
>> choice arguments. So, the description here can end with the A.3 in the
>> first line. Since the OPTIONAL attribute used throughout A.3 requires
>> basically all of the routines to be in this category, the "but only
>> for
>> routines" is confusing - at least to me.
>
> No. The idea is that MPI_COMM_RANK, etc. (i.e. routines without buffer)
> are always done in mpi_f08 according to 2a).
> 2b) is needed only for TYPE(*), DIMENSION(..).
>
>
>> Just from the names, I expected that mpi_send_f08_ would have
>> {derived-type handles, void * choice arguments, and no OPTIONAL
>> attribute} and mpi_send_f08ts_ would have {derived-type handles,
>> descriptor for choice, and OPTIONAL arguments} - i.e. as in A.3.
>
> No.
> 2a) and 2b) have {derived-type handles, and OPTIONAL ierror}.
> 2a) mpi_send_f08_ has "void *" choice arguments.
> 2b) mpi_send_f08ts_ has TYPE(*),DIMENSION(..) arguments according
>      to TS 29113 (therefore the postfix "ts"
>
>
>>> 3a) mpi_send_ Fortran interface and arguments, as in A.4.
>>>                                 In routines with a choice buffer, the
>>>                                 choice buffer
>>>                                 dummy argument is implemented with
>>>                                 non-standard
>>>                                 extensions like !$PRAGMA IGNORE_TKR.
>>>
>>> 3b) mpi_send_fts_ Fortran interface and arguments, as in A.4,
>>>                                 but only for routines with a choice
>>>                                 buffer argument
>>>                                 and this dummy argument is
>>>                                 implemented
>>>                                 with TYPE(*), DIMENSION(..).
>>
>> I see missing details here, similar to 2 above. Basically all of the
>> same comments, except that the handles are declared INTEGER here.
>
> The major differences between 2a+b) and 3a+b) are:
> - INTEGER handles in 3a+b).
> - IERROR is not OPTIONAL in 3a+b).
> - The callbacks are just EXTERNAL in 3a+b)
>    and have different Abstract interface definitions,
>    see all routines with callback function dummy arguments.
>    (The predefined callbacks do not cause additional differences).
> - Some differences for buffer address OUT dummy arguments,
>    see MPI_ALLOC_MEM, MPI_WIN_ALLOCATE, MPI_WIN_ALLOCATE_SHARED,
>    MPI_WIN_SHARED_QUERY, and MPI_GET_ADDRESS.
> - Other differences in MPI_BUFFER_DETACH.
>
> I'm not sure whether this list is complete.
>
> Important is also that TYPE(*),DIMENSION(..) is also allowed
> (and recommended in the mpi module, see p602:24-28)
> in the mpi module and mpif.h . Therefore 3b) is needed.
>
>
>>> For each external linkage name supported, a second entry point with
>>> the same calling characteristics shall be supplied, expect that the
>>> name is modified by prefixing with the letter "p" in the same case
>>> as the leading "m". For example, PMPI_Send and pmpi_send_.
>>>
>>> [No exceptions according to p555:35-37.]
>>>
>>> The external linkage names as mpi_send_, or mpi_send_fts_
>>> are examples for the external linkage name that the Fortran compiler
>>> chooses for the Fortran routine names MPI_Send, or MPI_Send_fts.
>>
>> I assume that the intent here is that these are examples and that the
>> real requirement is that the external linkage names are the ones
>> generated by the particular Fortran compiler for MPI_Send and
>> MPI_Send_fts.
>
> Yes. I updated the text a little bit into:
>
>   The external linkage names as mpi_send_f08_, mpi_send_f08ts_, mpi_send_,
>   and mpi_send_fts_ are examples for the external linkage name that the
>   particular Fortran compiler chooses for the Fortran routine names
>   MPI_Send_f08, MPI_Sendf08ts, MPI_Send, and MPI_Send_fts.
>
> I hope that this is a bit clearer.
>
>
>>> Using the mpi_f08 module, a Fortran call to MPI_Send must be
>>> implemented through a call one of the linker names defined in 1a) - 2b);
>>> if linker names in 1a) - 1d) are used, then an additional library
>>> must be provided that uses only 2a) and 2b) in the mpi_f08 module.
>>
>> Actually, 1d is not for the mpi_f08 module. It should be omitted from
>> the ranges in the paragraph above.
>
> Yes.
>>
>> I assume you don't mean "in the mpi_f08 module" here since you would
>> not get routines with names like these in a module.
>> And hopefully the
>> "only" is also wrong since otherwise this would lead to a huge amount
>> of unnecessary duplication. The obvious way to implement these
>> routines is in a file outside the module, with
>>
>>    subroutine mpi_send_f08 ( <args> )
>>      use mpi_f08
>
> Note: This mpi_f08 need not to be identical the the mpi_f08
>        provided to the user!
>
>>      <declarations for args>
>>      call mpi_send (<args> )
>>    end subroutine mpi_send_f08
>>
>> This ensures correct mpi_f08 behavior and allows the maintenance of the
>> actual Fortran MPI_Send to occur in only one place. And, of course,
>> avoid all of the duplicated work of providing two complete
>> implementations - one in the old pre-interop style, and one using
>> modules.
>
>
> Sorry, I understood that the compromise is as follows:
>
>   Using the mpi_f08 module, a Fortran call to MPI_Send must be implemented
>   through a call one of the linker names defined in 1a, 1b, 1c, 2a , or 2b;
>   if linker names in 1a - 1c are used, then an additional library together
>   with an mpi_f08 module must be provided that guarantees that
>   a Fortran call to an MPI routine results in a call to a procedure
>   according to 2a or 2b.
>
> You can do your wrapper as described above, as long as the module
> mpi_f08 you use here is identical to your optimized Version
> and the mpi_f08 module that you Hand over to the user
> in the not-optimized MPI is different:
>
> The non-optimized mpi_f08 module defines the mpi routines without CONTAINS.
>
> This text does not say anything about internals, i.e., it is still
> allowed that internally 1a-c is used as it may
> be done with your wrappers if the optimized version uses 1a-c.
>
>
>>> Using the mpi module or mpif.h, a Fortran call to MPI_Send must
>>> be implemented through a call to one of the linker names defined in
>>> 3a) and 3b).
>>>
>>
>> This is hopefully wrong, since it contradicts 1d which is specifically
>> for the mpi module. The current wording seems to suggest that a module
>> implementation using C interoperability is prohibited for the mpi
>> module, which (hopefully) is not the case. This needs to read similar
>> to the previous paragraph, such as
>>
>> "Using the mpi module, a Fortran call to MPI_Send must be implemented
>> through one of the linker names 1a), 1d), 3a), or 3b). If 1a) and 1d)
>> are used, then routines with external linkage names as specified in
>> 3a)
>> and 3b) must also be provided."
>
> Yes, there is no need to do the compromise only for the mpi_f08
> module. The compromise is also good for the mpi module:
>
>    Using the mpi module, a Fortran call to MPI_Send must be implemented
>    through a call one of the linker names defined in 1a, 1b, 1d, 3a, or 3b;
>    if linker names in 1a, 1b, and 1d are used, then an additional library
>    together with an mpi module must be provided that guarantees that
>    a Fortran call to an MPI routine results in a call to a procedure
>    according to 3a or 3b.
>
> It cannot be done for mpif.h, because it cannot contain a module.
> Therefore, only 3a and 3b is possible:
>
>    Using mpif.h, a Fortran call to MPI_Send must be implemented
>    through a call one of the linker names defined in 3a, or 3b.
>
>
>> Do the tools people really need that second sentence? I was under the
>> impression they would be happy with dealing only with the C functions.
>
> We must make also the users happy who used in the past some PMPI
> Interfaces for some reasons!
> This group is not identical to the "tools people".
>
> This is the background for this compromise.
>
>> [Although is this, and similarly for the mpi_f08 module case, actually
>> true? It seems that this requirement is only relevant for the tools
>> developers, and that their real need is that the PMPI_xxx versions be
>> available.]
>
>
> Both is needed: the possibility for interception and the PMPI routines.
>
>> Cheers,
>> Bill
>
>>> 17.1.5.3 Tools Support
>
> I changed my proposed Header file Name into pmpif.h
>
>>> 17.1.5.3 Execution Environments
>
> I removed the now empty section.
>
>
> I hope the new version c19.comments.txt solves most of your issues
> and serves also the issues of OpenMPI as defined in the compromise.
>
> I cannot come to the June Meeting but I hope that this text
> is a good Input for the Fortran WG Meeting there.
> Bill, do you come to the June meeting tomorrow?
>
> Best regards
> Rolf
>>
>>
>> --
>> Bill Long longb at cray.com
>> Fortran Technical Support & voice: 651-605-9024
>> Bioinformatics Software Development fax: 651-605-9142
>> Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
>

-- 
Bill Long                                           longb at cray.com
Fortran Technical Support    &                 voice: 651-605-9024
Bioinformatics Software Development            fax:   651-605-9142
Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101