[MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point

Fri Jun 16 12:22:42 CDT 2017

> On Jun 16, 2017, at 7:19 PM, Rolf Rabenseifner <rabenseifner at hlrs.de> wrote:
> 
> Jeff is partially right:
> If the Fortran user uses selected_real_kind(p, r), then all is done.
> 
> On the other hand,
> I expect that the Fortran standardization body will do the same 
> as the C standardization body, namely to add a new language Basic type.

No. The basic type is REAL.  This is just another KIND of REAL.  This is not much of an issue for the standard.  It is a HUGE deal for implementations.

Cheers,
Bill

> 
> For this of course, we have to provide the corresponding MPI predefined datatype.
> Rolf
> 
> 
> ----- Original Message -----
>> From: "Jeff Hammond" <jeff.science at gmail.com>
>> To: "MPI WG Point To Point Communications working group" <mpiwg-p2p at lists.mpi-forum.org>
>> Cc: "Atsushi Hori" <ahori at riken.jp>, "Reinhold Bader" <Reinhold.Bader at lrz-muenchen.de>, "fortran standards email list
>> for J3" <j3 at j3-fortran.org>, "MPI-WG Fortran working group" <mpiwg-fortran at lists.mpi-forum.org>
>> Sent: Friday, June 16, 2017 5:11:24 PM
>> Subject: Re: [MPIWG Fortran] [mpiwg-p2p]  FP16 16-bit-floating-point
> 
>> No change to the MPI standard is required here. MPI_Type_create_f90_real is
>> already sufficient.
>> 
>> "This function returns a predefined MPI datatype that matches a REAL variable of
>> KIND selected_real_kind(p, r)." - I do not see why we need to enumerate a
>> predefined datatype name associated with float16 in Fortran for this statement
>> to be true. An implementation can return a datatype with the appropriate
>> properties without being enumerated like MPI_REAL.
>> 
>> "It is erroneous to supply values for p and r not supported by the compiler." -
>> Conversely, the user should be allowed to supply values that are supported by
>> the compiler (although the standard doesn't guarentee it today). MPI
>> implementations might not support every type the compiler does, as is already
>> the case for Fortran float128 (because MPI libraries are written in C, and do
>> not call back into Fortran to do math on these types, which may not be
>> identical to C extensions like __float128), but I consider that an
>> implementation defect.
>> 
>> Thus, I propose to merely add the text that allows users to rely on
>> MPI_Type_create_f90_real for float16 in Fortran, provided that the compiler
>> supports it. It will then be up to implementations to figure out how to do math
>> on these types internally.
>> 
>> While we are on this subject, we should be considering the more general
>> inadequacy of the current state of the MPI type system from the perspective of
>> reductions. See [ https://github.com/mpi-forum/mpi-issues/issues/18 |
>> https://github.com/mpi-forum/mpi-issues/issues/18 ] for details.
>> 
>> As for ISO C/C++, I can talk to Clark Nelson if a better candidate is not
>> identified, but Lawrence Crowl is leading the C++ effort and I suspect the C
>> folks will follow, although I will confirm this with Clark.
>> 
>> Jeff
>> 
>> On Fri, Jun 16, 2017 at 6:16 AM, Kawashima, Takahiro < [
>> mailto:t-kawashima at jp.fujitsu.com | t-kawashima at jp.fujitsu.com ] > wrote:
>> 
>> 
>> Hi,
>> 
>> I don't know the status of C standardization nor contact person
>> but the suggestion of 'short float' was seem to made in a WG.
>> 
>> [ http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf |
>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf ]
>> [ http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf |
>> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf ]
>> 
>> Regards,
>> Takahiro Kawashima,
>> Fujitsu
>> 
>>> Tony,
>>> 
>>> we try to find the type names used in C and Fortran before we define
>>> the new predefined MPI datatype handles for representing those
>>> language types.
>>> 
>>> Who knows, whom we can contact on the C standardization side?
>>> 
>>> Best regards
>>> Rolf
>>> 
>>> ----- Original Message -----
>>>> From: "Anthony Skjellum" < [ mailto:tony at runtimecomputing.com |
>>>> tony at runtimecomputing.com ] >
>>>> To: "MPI WG Point To Point Communications working group" < [
>>>> mailto:mpiwg-p2p at lists.mpi-forum.org | mpiwg-p2p at lists.mpi-forum.org ] >
>>>> Cc: "Reinhold Bader" < [ mailto:Reinhold.Bader at lrz-muenchen.de |
>>>> Reinhold.Bader at lrz-muenchen.de ] >, "Atsushi Hori" < [ mailto:ahori at riken.jp |
>>>> ahori at riken.jp ] >, "fortran standards email list
>>>> for J3" < [ mailto:j3 at j3-fortran.org | j3 at j3-fortran.org ] >, "MPI-WG Fortran
>>>> working group" < [ mailto:mpiwg-fortran at lists.mpi-forum.org |
>>>> mpiwg-fortran at lists.mpi-forum.org ] >
>>>> Sent: Friday, June 16, 2017 2:50:00 PM
>>>> Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point
>>> 
>>>> Rolf, apart from the Fortan angle, may I strongly encourage that we adopt this
>>>> datatype for MPI-4. It is part of real MPI codes already, but now we have to
>>>> fake transfer datatypes as 16-bit integers or bytes... which means not really
>>>> right if heterogeneous interoperability were to be maintained. And this is a
>>>> canonical IEEE type now, as you note. There is also the "2 half-float" datatype
>>>> (32-bits) at least in CUDA to be looked at; maybe ignored or maybe supported,
>>>> but it is there too. Key to this MPI users and applications use half floats and
>>>> 2 half-floats :-) in practice.
>>>> 
>>>> May I also suggest a forward look at the types in AVX-512, in case any of these
>>>> become particularly relevant in future.
>>>> At least 8-bit integers are conceptually new, apart from BYTE. [
>>>> [ https://en.wikipedia.org/wiki/AVX-512 | https://en.wikipedia.org/wiki/AVX-512
>>>> ] | [ https://en.wikipedia.org/wiki/AVX-512 |
>>>> https://en.wikipedia.org/wiki/AVX-512 ] ]
>>>> 
>>>> Regards,
>>>> Tony
>>>> 
>>>> 
>>>> On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [
>>>> mailto: [ mailto:rabenseifner at hlrs.de | rabenseifner at hlrs.de ] | [
>>>> mailto:rabenseifner at hlrs.de | rabenseifner at hlrs.de ] ] > wrote:
>>>> 
>>>> 
>>>> Hi all together,
>>>> 
>>>> I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16
>>>> datatype.
>>>> 
>>>> Questions to the Fortran atandardization body:
>>>> 
>>>> - Does the Fortran Standardization Body has already a proposal for
>>>> the new name in Fortran for this IEEE FP16?
>>>> 
>>>> - Will be REAL*2 identical to IEEE FP16?
>>>> 
>>>> Best regards
>>>> Rolf
>> _______________________________________________
>> mpiwg-p2p mailing list
>> [ mailto:mpiwg-p2p at lists.mpi-forum.org | mpiwg-p2p at lists.mpi-forum.org ]
>> [ https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p |
>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p ]
>> 
>> 
>> 
>> --
>> Jeff Hammond
>> [ mailto:jeff.science at gmail.com | jeff.science at gmail.com ]
>> [ http://jeffhammond.github.io/ | http://jeffhammond.github.io/ ]
>> 
>> _______________________________________________
>> mpiwg-fortran mailing list
>> mpiwg-fortran at lists.mpi-forum.org
>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran
> 
> -- 
> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de .
> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 .
> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 .
> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner .
> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .
> _______________________________________________
> mpiwg-fortran mailing list
> mpiwg-fortran at lists.mpi-forum.org
> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran

Bill Long                                                                       longb at cray.com
Principal Engineer, Fortran Technical Support &   voice:  651-605-9024
Bioinformatics Software Development                      fax:  651-605-9143
Cray Inc./ 2131 Lindau Lane/  Suite 1000/  Bloomington, MN  55425