[Mpiwg-large-counts] A perfect trick in C with similar API as with Fortran overloading - instead of the many new _l functions in C

Mon Oct 21 10:11:20 CDT 2019

+1 do not break ABI in this way.

Thanks,
Ken

On 10/21/19 8:41 AM, Jeff Squyres (jsquyres) via mpiwg-large-counts wrote:
> I agree: breaking ABI in this way is truly evil.
> 
> Specifically: compile your app with implementation version X.  Then run your app with same implementation, but version Y (this is not uncommon for commercial ISV MPI apps).  Things fail in weird, non-obvious ways (or worse, they *don't* fail, but silently give you wrong results).
> 
> Speaking as someone who writes code for a living, changing the signature of an existing, well-established function is evil.  Changing it in ways that a compiler may not even warn you about (in forward- and backward-compatility scenarios) is TRULY evil.
> 
> 
> 
> 
>> On Oct 21, 2019, at 9:35 AM, Jeff Hammond via mpiwg-large-counts <mpiwg-large-counts at lists.mpi-forum.org> wrote:
>>
>> ABI is very much a user problem. We are not breaking ABI for this.
>>
>> What is being done now is five years in the making. I’m not sure why you didn’t comment until recently.
>>
>> Jeff
>>
>> Sent from my iPhone
>>
>>> On Oct 20, 2019, at 10:49 PM, Rolf Rabenseifner via mpiwg-large-counts <mpiwg-large-counts at lists.mpi-forum.org> wrote:
>>>
>>> Dear Tony and all,
>>>
>>>> the group has labored over each area... 1 sided has had the least attention so far... All of MPI is to be made 64 bit clean not parts.
>>>
>>> My proposal ia for whole MPI.
>>> Only for few routines we need some additional _x versions.
>>>
>>>> Partial answer to your earliest question in the note: Changing the meaning of existing APIs was disallowed some time ago.  Certainly, for in arguments of count , replacing int with MPI_Count is a partial solution.  But it changes the APIs ...
>>>
>>> The goal is backward compatibility.
>>> And this one is provided.
>>>
>>>> In C, a program written for the assumption of big count and compiled accidentally with an MPI-3 compliant MPI will silently build and fail at runtime ... rounding of integers ...
>>>
>>> Let's recommend to the users of the long coutnt feature that they should at a check of the version. In C, in can be tested at compile time with cpp.
>>>
>>>> and changing the AbI violates the ABI for tools and such  — two  possible reasons why not chosen ... but we will have to go refresh on all the reasons .
>>>
>>> Not a user problem ...
>>>
>>>> I am sure you will get extensive feedback on these questions from the whole group ? :-)
>>>
>>> And also by the plenum.
>>>
>>> Best regards
>>> Rolf
>>>
>>> ----- Tony Skjellum <skjellum at gmail.com> wrote:
>>>> Rolf, the group has labored over each area... 1 sided has had the least attention so far... All of MPI is to be made 64 bit clean not parts.
>>>>
>>>> Partial answer to your earliest question in the note: Changing the meaning of existing APIs was disallowed some time ago.  Certainly, for in arguments of count , replacing int with MPI_Count is a partial solution.  But it changes the APIs ...
>>>>
>>>> In C, a program written for the assumption of big count and compiled accidentally with an MPI-3 compliant MPI will silently build and fail at runtime ... rounding of integers ... and changing the AbI violates the ABI for tools and such  — two  possible reasons why not chosen ... but we will have to go refresh on all the reasons .
>>>>
>>>> I am sure you will get extensive feedback on these questions from the whole group ? :-)
>>>>
>>>> Tony
>>>>
>>>> Anthony Skjellum, PhD
>>>> 205-807-4968
>>>>
>>>>
>>>>> On Oct 20, 2019, at 2:25 PM, Rolf Rabenseifner via mpiwg-large-counts <mpiwg-large-counts at lists.mpi-forum.org> wrote:
>>>>>
>>>>> Dear all,
>>>>>
>>>>> a participant in my latest course asked me, whether we not
>>>>> substituting in by MPI_Count in all "int xxxxx" input arguments.
>>>>> The compiler automatically would cast to MPI_Count which may be a fast
>>>>> instruction. Then we need the new _x or _l functions only for those
>>>>> rare functions with MPI_Count as output or array argument.
>>>>>
>>>>> Was this ever discussed?
>>>>>
>>>>> Second question:
>>>>> Does the large Count roup plan, only to use MPI_Count for data,
>>>>> i.e., only in communication, I/O, and derived type routines?
>>>>>
>>>>> Is it correct, that we keep the number of
>>>>> - request handles in request arrays,
>>>>> - ranks an size in/of communicator and group handles,
>>>>> will stay int?
>>>>>
>>>>> How do we handle disp in 1-sided communication?
>>>>>
>>>>> With this approach, it seems that explicit MPI_Count versions of
>>>>> a subroutine is needed only for
>>>>> MPI_[I][NEIGHBOR_]{ALLTOALL|ALLGATHER}{V|W}
>>>>> MPI_[I]{GATHER|SCATTER}V
>>>>> MPI_[I]REDUCE_SCATTER
>>>>> (18 routines)
>>>>> MPI_WIN_SHARED_QUERY --> *disp
>>>>> (1 Routine)
>>>>> MPI_[UN]PACK
>>>>> MPI_PACK_SIZE _x/_l version should have MPI_Aint (same as for MPI_...PACK..._external)
>>>>> (3 routines)
>>>>> MPI_GET_COUNT
>>>>> MPI_GET_ELEMENTS (has already an _X version)
>>>>> MPI_TYPE_SIZE (has already an _X version)
>>>>>
>>>>> Which solution exists for MPI_CONVERSION_FN_NULL?
>>>>>
>>>>> All other 100 routines with "int count" and "int size" seems
>>>>> to be solved with directly changing it into "MPI_Count count" and "MPI_Count size".
>>>>>
>>>>> Best regards
>>>>> Rolf
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "mpiwg-large-counts" <mpiwg-large-counts at lists.mpi-forum.org>
>>>>>> To: "mpiwg-large-counts" <mpiwg-large-counts at lists.mpi-forum.org>
>>>>>> Cc: "Jeff Squyres" <jsquyres at cisco.com>
>>>>>> Sent: Saturday, October 19, 2019 10:26:23 PM
>>>>>> Subject: [Mpiwg-large-counts] Pythonizing instructions
>>>>>
>>>>>> As discussed in our last meeting, I've written a first take at instructions for
>>>>>> Chapter Authors for how to write Pythonized functions.
>>>>>>
>>>>>> These instructions are not yet intended for a wide audience -- they are intended
>>>>>> solely for the working group.  Let's iterate on them here in the WG and get
>>>>>> them to a good enough state before sharing them widely:
>>>>>>
>>>>>> https://github.com/mpiwg-large-count/large-count-issues/wiki/pythonizing
>>>>>>
>>>>>> Puri/Tony/Dan: please try to use these instructions to Pythonize a few bindings.
>>>>>> The PR for the Pythonization is
>>>>>> https://github.com/mpiwg-large-count/mpi-standard/pull/2.  All even-numbered
>>>>>> chapters are done, and a small number of odd-numbered chapters.  Grep for
>>>>>> `funcdef` in chapter *.tex files to find a chapter that hasn't been Pythonized
>>>>>> yet (funcdef is the LaTeX macro for hard-coded LaTeX macros).
>>>>>>
>>>>>> Send your feedback here and/or we'll discuss on Wednesday.
>>>>>>
>>>>>> --
>>>>>> Jeff Squyres
>>>>>> jsquyres at cisco.com
>>>>>>
>>>>>> _______________________________________________
>>>>>> mpiwg-large-counts mailing list
>>>>>> mpiwg-large-counts at lists.mpi-forum.org
>>>>>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-large-counts
>>>>>
>>>>> -- 
>>>>> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de .
>>>>> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 .
>>>>> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 .
>>>>> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner .
>>>>> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .
>>>>> _______________________________________________
>>>>> mpiwg-large-counts mailing list
>>>>> mpiwg-large-counts at lists.mpi-forum.org
>>>>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-large-counts
>>>
>>> -- 
>>> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de .
>>> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 .
>>> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 .
>>> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner .
>>> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .
>>> _______________________________________________
>>> mpiwg-large-counts mailing list
>>> mpiwg-large-counts at lists.mpi-forum.org
>>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-large-counts
>> _______________________________________________
>> mpiwg-large-counts mailing list
>> mpiwg-large-counts at lists.mpi-forum.org
>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-large-counts
> 
>