[MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point

Fri Jun 16 12:35:26 CDT 2017

First, I want to remeber to Takahiro's answer aboout C.

>  http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf
>  http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf
They will probably add 
 - short float, and
 - float16_t, float32_t, float64_t
to the C Standard.

For Fortran, the MPI standard currently supports floating points through

- MPI_REAL  for REAL
- MPI_DOUBLE_PRECISION for  DOUBLE PRECISION 

and only when supported by the Compiler (i.e. optional)

- MPI_REAL2  for REAL*2
- MPI_REAL4  for REAL*4
- MPI_REAL8  for REAL*8
- MPI_REAL16 for REAL*16

The question is, whether Fortran will add something similar
as "short float".

Another question is, whether we never detected that Fortran has
officially byte- or bit-sized floats, like the REAL*8.

Rolf

----- Original Message -----
> From: "Kawashima, Takahiro" <t-kawashima at jp.fujitsu.com>
> To: "MPI WG Point To Point Communications working group" <mpiwg-p2p at lists.mpi-forum.org>
> Cc: "Rolf Rabenseifner" <rabenseifner at hlrs.de>, "MPI-WG Fortran working group" <mpiwg-fortran at lists.mpi-forum.org>,
> "Anthony Skjellum" <tony at runtimecomputing.com>, "Reinhold Bader" <Reinhold.Bader at lrz-muenchen.de>, "Atsushi Hori"
> <ahori at riken.jp>, "fortran standards email list for J3" <j3 at j3-fortran.org>
> Sent: Friday, June 16, 2017 3:16:57 PM
> Subject: Re: [mpiwg-p2p] [MPIWG Fortran]  FP16 16-bit-floating-point

> Hi,
> 
> I don't know the status of C standardization nor contact person
> but the suggestion of 'short float' was seem to made in a WG.
> 
>  http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf
>  http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf
> 
> Regards,
> Takahiro Kawashima,
> Fujitsu

----- Original Message -----
> From: "longb" <longb at cray.com>
> To: "MPI-WG Fortran working group" <mpiwg-fortran at lists.mpi-forum.org>
> Cc: "Atsushi Hori" <ahori at riken.jp>, "Reinhold Bader" <Reinhold.Bader at lrz-muenchen.de>, "fortran standards email list
> for J3" <j3 at j3-fortran.org>
> Sent: Friday, June 16, 2017 7:22:42 PM
> Subject: Re: [MPIWG Fortran] [mpiwg-p2p]  FP16 16-bit-floating-point

>> On Jun 16, 2017, at 7:19 PM, Rolf Rabenseifner <rabenseifner at hlrs.de> wrote:
>> 
>> Jeff is partially right:
>> If the Fortran user uses selected_real_kind(p, r), then all is done.
>> 
>> On the other hand,
>> I expect that the Fortran standardization body will do the same
>> as the C standardization body, namely to add a new language Basic type.
> 
> No. The basic type is REAL.  This is just another KIND of REAL.  This is not
> much of an issue for the standard.  It is a HUGE deal for implementations.
> 
> Cheers,
> Bill
> 
> 
>> 
>> For this of course, we have to provide the corresponding MPI predefined
>> datatype.
>> Rolf
>> 
>> 
>> ----- Original Message -----
>>> From: "Jeff Hammond" <jeff.science at gmail.com>
>>> To: "MPI WG Point To Point Communications working group"
>>> <mpiwg-p2p at lists.mpi-forum.org>
>>> Cc: "Atsushi Hori" <ahori at riken.jp>, "Reinhold Bader"
>>> <Reinhold.Bader at lrz-muenchen.de>, "fortran standards email list
>>> for J3" <j3 at j3-fortran.org>, "MPI-WG Fortran working group"
>>> <mpiwg-fortran at lists.mpi-forum.org>
>>> Sent: Friday, June 16, 2017 5:11:24 PM
>>> Subject: Re: [MPIWG Fortran] [mpiwg-p2p]  FP16 16-bit-floating-point
>> 
>>> No change to the MPI standard is required here. MPI_Type_create_f90_real is
>>> already sufficient.
>>> 
>>> "This function returns a predefined MPI datatype that matches a REAL variable of
>>> KIND selected_real_kind(p, r)." - I do not see why we need to enumerate a
>>> predefined datatype name associated with float16 in Fortran for this statement
>>> to be true. An implementation can return a datatype with the appropriate
>>> properties without being enumerated like MPI_REAL.
>>> 
>>> "It is erroneous to supply values for p and r not supported by the compiler." -
>>> Conversely, the user should be allowed to supply values that are supported by
>>> the compiler (although the standard doesn't guarentee it today). MPI
>>> implementations might not support every type the compiler does, as is already
>>> the case for Fortran float128 (because MPI libraries are written in C, and do
>>> not call back into Fortran to do math on these types, which may not be
>>> identical to C extensions like __float128), but I consider that an
>>> implementation defect.
>>> 
>>> Thus, I propose to merely add the text that allows users to rely on
>>> MPI_Type_create_f90_real for float16 in Fortran, provided that the compiler
>>> supports it. It will then be up to implementations to figure out how to do math
>>> on these types internally.
>>> 
>>> While we are on this subject, we should be considering the more general
>>> inadequacy of the current state of the MPI type system from the perspective of
>>> reductions. See [ https://github.com/mpi-forum/mpi-issues/issues/18 |
>>> https://github.com/mpi-forum/mpi-issues/issues/18 ] for details.
>>> 
>>> As for ISO C/C++, I can talk to Clark Nelson if a better candidate is not
>>> identified, but Lawrence Crowl is leading the C++ effort and I suspect the C
>>> folks will follow, although I will confirm this with Clark.
>>> 
>>> Jeff
>>> 
>>> On Fri, Jun 16, 2017 at 6:16 AM, Kawashima, Takahiro < [
>>> mailto:t-kawashima at jp.fujitsu.com | t-kawashima at jp.fujitsu.com ] > wrote:
>>> 
>>> 
>>> Hi,
>>> 
>>> I don't know the status of C standardization nor contact person
>>> but the suggestion of 'short float' was seem to made in a WG.
>>> 
>>> [ http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf |
>>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf ]
>>> [ http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf |
>>> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf ]
>>> 
>>> Regards,
>>> Takahiro Kawashima,
>>> Fujitsu
>>> 
>>>> Tony,
>>>> 
>>>> we try to find the type names used in C and Fortran before we define
>>>> the new predefined MPI datatype handles for representing those
>>>> language types.
>>>> 
>>>> Who knows, whom we can contact on the C standardization side?
>>>> 
>>>> Best regards
>>>> Rolf
>>>> 
>>>> ----- Original Message -----
>>>>> From: "Anthony Skjellum" < [ mailto:tony at runtimecomputing.com |
>>>>> tony at runtimecomputing.com ] >
>>>>> To: "MPI WG Point To Point Communications working group" < [
>>>>> mailto:mpiwg-p2p at lists.mpi-forum.org | mpiwg-p2p at lists.mpi-forum.org ] >
>>>>> Cc: "Reinhold Bader" < [ mailto:Reinhold.Bader at lrz-muenchen.de |
>>>>> Reinhold.Bader at lrz-muenchen.de ] >, "Atsushi Hori" < [ mailto:ahori at riken.jp |
>>>>> ahori at riken.jp ] >, "fortran standards email list
>>>>> for J3" < [ mailto:j3 at j3-fortran.org | j3 at j3-fortran.org ] >, "MPI-WG Fortran
>>>>> working group" < [ mailto:mpiwg-fortran at lists.mpi-forum.org |
>>>>> mpiwg-fortran at lists.mpi-forum.org ] >
>>>>> Sent: Friday, June 16, 2017 2:50:00 PM
>>>>> Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point
>>>> 
>>>>> Rolf, apart from the Fortan angle, may I strongly encourage that we adopt this
>>>>> datatype for MPI-4. It is part of real MPI codes already, but now we have to
>>>>> fake transfer datatypes as 16-bit integers or bytes... which means not really
>>>>> right if heterogeneous interoperability were to be maintained. And this is a
>>>>> canonical IEEE type now, as you note. There is also the "2 half-float" datatype
>>>>> (32-bits) at least in CUDA to be looked at; maybe ignored or maybe supported,
>>>>> but it is there too. Key to this MPI users and applications use half floats and
>>>>> 2 half-floats :-) in practice.
>>>>> 
>>>>> May I also suggest a forward look at the types in AVX-512, in case any of these
>>>>> become particularly relevant in future.
>>>>> At least 8-bit integers are conceptually new, apart from BYTE. [
>>>>> [ https://en.wikipedia.org/wiki/AVX-512 | https://en.wikipedia.org/wiki/AVX-512
>>>>> ] | [ https://en.wikipedia.org/wiki/AVX-512 |
>>>>> https://en.wikipedia.org/wiki/AVX-512 ] ]
>>>>> 
>>>>> Regards,
>>>>> Tony
>>>>> 
>>>>> 
>>>>> On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [
>>>>> mailto: [ mailto:rabenseifner at hlrs.de | rabenseifner at hlrs.de ] | [
>>>>> mailto:rabenseifner at hlrs.de | rabenseifner at hlrs.de ] ] > wrote:
>>>>> 
>>>>> 
>>>>> Hi all together,
>>>>> 
>>>>> I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16
>>>>> datatype.
>>>>> 
>>>>> Questions to the Fortran atandardization body:
>>>>> 
>>>>> - Does the Fortran Standardization Body has already a proposal for
>>>>> the new name in Fortran for this IEEE FP16?
>>>>> 
>>>>> - Will be REAL*2 identical to IEEE FP16?
>>>>> 
>>>>> Best regards
>>>>> Rolf
>>> _______________________________________________
>>> mpiwg-p2p mailing list
>>> [ mailto:mpiwg-p2p at lists.mpi-forum.org | mpiwg-p2p at lists.mpi-forum.org ]
>>> [ https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p |
>>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p ]
>>> 
>>> 
>>> 
>>> --
>>> Jeff Hammond
>>> [ mailto:jeff.science at gmail.com | jeff.science at gmail.com ]
>>> [ http://jeffhammond.github.io/ | http://jeffhammond.github.io/ ]
>>> 
>>> _______________________________________________
>>> mpiwg-fortran mailing list
>>> mpiwg-fortran at lists.mpi-forum.org
>>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran
>> 
>> --
>> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de .
>> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 .
>> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 .
>> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner .
>> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .
>> _______________________________________________
>> mpiwg-fortran mailing list
>> mpiwg-fortran at lists.mpi-forum.org
>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran
> 
> Bill Long
> longb at cray.com
> Principal Engineer, Fortran Technical Support &   voice:  651-605-9024
> Bioinformatics Software Development                      fax:  651-605-9143
> Cray Inc./ 2131 Lindau Lane/  Suite 1000/  Bloomington, MN  55425
> 
> 
> _______________________________________________
> mpiwg-fortran mailing list
> mpiwg-fortran at lists.mpi-forum.org
> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran

-- 
Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de .
High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 .
University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 .
Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner .
Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .