[mpiwg-p2p] [MPIWG Fortran] FP16 16-bit-floating-point

Fri Jun 16 12:19:44 CDT 2017

Jeff is partially right:
If the Fortran user uses selected_real_kind(p, r), then all is done.

On the other hand,
I expect that the Fortran standardization body will do the same 
as the C standardization body, namely to add a new language Basic type.

For this of course, we have to provide the corresponding MPI predefined datatype.
Rolf

----- Original Message -----
> From: "Jeff Hammond" <jeff.science at gmail.com>
> To: "MPI WG Point To Point Communications working group" <mpiwg-p2p at lists.mpi-forum.org>
> Cc: "Atsushi Hori" <ahori at riken.jp>, "Reinhold Bader" <Reinhold.Bader at lrz-muenchen.de>, "fortran standards email list
> for J3" <j3 at j3-fortran.org>, "MPI-WG Fortran working group" <mpiwg-fortran at lists.mpi-forum.org>
> Sent: Friday, June 16, 2017 5:11:24 PM
> Subject: Re: [MPIWG Fortran] [mpiwg-p2p]  FP16 16-bit-floating-point

> No change to the MPI standard is required here. MPI_Type_create_f90_real is
> already sufficient.
> 
> "This function returns a predefined MPI datatype that matches a REAL variable of
> KIND selected_real_kind(p, r)." - I do not see why we need to enumerate a
> predefined datatype name associated with float16 in Fortran for this statement
> to be true. An implementation can return a datatype with the appropriate
> properties without being enumerated like MPI_REAL.
> 
> "It is erroneous to supply values for p and r not supported by the compiler." -
> Conversely, the user should be allowed to supply values that are supported by
> the compiler (although the standard doesn't guarentee it today). MPI
> implementations might not support every type the compiler does, as is already
> the case for Fortran float128 (because MPI libraries are written in C, and do
> not call back into Fortran to do math on these types, which may not be
> identical to C extensions like __float128), but I consider that an
> implementation defect.
> 
> Thus, I propose to merely add the text that allows users to rely on
> MPI_Type_create_f90_real for float16 in Fortran, provided that the compiler
> supports it. It will then be up to implementations to figure out how to do math
> on these types internally.
> 
> While we are on this subject, we should be considering the more general
> inadequacy of the current state of the MPI type system from the perspective of
> reductions. See [ https://github.com/mpi-forum/mpi-issues/issues/18 |
> https://github.com/mpi-forum/mpi-issues/issues/18 ] for details.
> 
> As for ISO C/C++, I can talk to Clark Nelson if a better candidate is not
> identified, but Lawrence Crowl is leading the C++ effort and I suspect the C
> folks will follow, although I will confirm this with Clark.
> 
> Jeff
> 
> On Fri, Jun 16, 2017 at 6:16 AM, Kawashima, Takahiro < [
> mailto:t-kawashima at jp.fujitsu.com | t-kawashima at jp.fujitsu.com ] > wrote:
> 
> 
> Hi,
> 
> I don't know the status of C standardization nor contact person
> but the suggestion of 'short float' was seem to made in a WG.
> 
> [ http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf |
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf ]
> [ http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf |
> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf ]
> 
> Regards,
> Takahiro Kawashima,
> Fujitsu
> 
>> Tony,
>> 
>> we try to find the type names used in C and Fortran before we define
>> the new predefined MPI datatype handles for representing those
>> language types.
>> 
>> Who knows, whom we can contact on the C standardization side?
>> 
>> Best regards
>> Rolf
>> 
>> ----- Original Message -----
>> > From: "Anthony Skjellum" < [ mailto:tony at runtimecomputing.com |
>> > tony at runtimecomputing.com ] >
>> > To: "MPI WG Point To Point Communications working group" < [
>> > mailto:mpiwg-p2p at lists.mpi-forum.org | mpiwg-p2p at lists.mpi-forum.org ] >
>> > Cc: "Reinhold Bader" < [ mailto:Reinhold.Bader at lrz-muenchen.de |
>> > Reinhold.Bader at lrz-muenchen.de ] >, "Atsushi Hori" < [ mailto:ahori at riken.jp |
>> > ahori at riken.jp ] >, "fortran standards email list
>> > for J3" < [ mailto:j3 at j3-fortran.org | j3 at j3-fortran.org ] >, "MPI-WG Fortran
>> > working group" < [ mailto:mpiwg-fortran at lists.mpi-forum.org |
>> > mpiwg-fortran at lists.mpi-forum.org ] >
>> > Sent: Friday, June 16, 2017 2:50:00 PM
>> > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point
>> 
>> > Rolf, apart from the Fortan angle, may I strongly encourage that we adopt this
>> > datatype for MPI-4. It is part of real MPI codes already, but now we have to
>> > fake transfer datatypes as 16-bit integers or bytes... which means not really
>> > right if heterogeneous interoperability were to be maintained. And this is a
>> > canonical IEEE type now, as you note. There is also the "2 half-float" datatype
>> > (32-bits) at least in CUDA to be looked at; maybe ignored or maybe supported,
>> > but it is there too. Key to this MPI users and applications use half floats and
>> > 2 half-floats :-) in practice.
>> > 
>> > May I also suggest a forward look at the types in AVX-512, in case any of these
>> > become particularly relevant in future.
>> > At least 8-bit integers are conceptually new, apart from BYTE. [
>> > [ https://en.wikipedia.org/wiki/AVX-512 | https://en.wikipedia.org/wiki/AVX-512
>> > ] | [ https://en.wikipedia.org/wiki/AVX-512 |
>> > https://en.wikipedia.org/wiki/AVX-512 ] ]
>> > 
>> > Regards,
>> > Tony
>> > 
>> > 
>> > On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [
>> > mailto: [ mailto:rabenseifner at hlrs.de | rabenseifner at hlrs.de ] | [
>> > mailto:rabenseifner at hlrs.de | rabenseifner at hlrs.de ] ] > wrote:
>> > 
>> > 
>> > Hi all together,
>> > 
>> > I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16
>> > datatype.
>> > 
>> > Questions to the Fortran atandardization body:
>> > 
>> > - Does the Fortran Standardization Body has already a proposal for
>> > the new name in Fortran for this IEEE FP16?
>> > 
>> > - Will be REAL*2 identical to IEEE FP16?
>> > 
>> > Best regards
>> > Rolf
> _______________________________________________
> mpiwg-p2p mailing list
> [ mailto:mpiwg-p2p at lists.mpi-forum.org | mpiwg-p2p at lists.mpi-forum.org ]
> [ https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p |
> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p ]
> 
> 
> 
> --
> Jeff Hammond
> [ mailto:jeff.science at gmail.com | jeff.science at gmail.com ]
> [ http://jeffhammond.github.io/ | http://jeffhammond.github.io/ ]
> 
> _______________________________________________
> mpiwg-fortran mailing list
> mpiwg-fortran at lists.mpi-forum.org
> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-fortran

-- 
Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de .
High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 .
University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 .
Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner .
Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .