[mpiwg-p2p] [MPIWG Fortran] FP16 16-bit-floating-point

Fri Jun 16 13:04:50 CDT 2017

Atsushi-san, (now only in the P2P-WG)

>  http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf
>  http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf

Therefore for the C-side, I would propose for
(oc course voted in only after the C standardization gets confirmed)

MPI-3.1 table on page 673

 Named Predefined Datatypes | C types
 ---------------------------+------------
 MPI_SHORT_FLOAT            | short float
 MPI_FLOAT16_T              | float16_t
 MPI_FLOAT32_T              | float32_t
 MPI_FLOAT64_T              | float64_t

(We must check, whether they also include float128_t into the C Standard,
 or whether they only standardize short float without all ..._T)

MPI-3.1 table on page 675 lines 1-9

 Datatypes for reduction functions (C)
 -------------------------------------
 MPI_SHORT_FLOAT_INT

(we never did the MAX/MINLOC reductions also for fixed-size types)

MPI-3.1 Table 13.2 on page 540 ***and***
MPI-3.1 Table 3.2 on page 26

 Type            | Length
 ----------------+-------
 MPI_SHORT_FLOAT |   2
 MPI_FLOAT16_T   |   2
 MPI_FLOAT32_T   |   4
 MPI_FLOAT64_T   |   8

And adding the following to "Floating point:"
on page 176, line 45:

- before MPI_FLOAT:
     MPI_SHORT_FLOAT,
- after MPI_DOUBLE:
     MPI_FLOAT16_T, MPI_FLOAT32_T, MPI_FLOAT64_T,

By the way, the table on page 540 should be done
with normal Latex and the correct macros for constants
to guarantee that this Location is also visible in the 
constants-index page 823ff.

You may have detected further Locations that are relevant.

And of course, the appropriate change-log index is also needed.

Best regards
Rolf

----- Original Message -----
> From: "Kawashima, Takahiro" <t-kawashima at jp.fujitsu.com>
> To: "MPI WG Point To Point Communications working group" <mpiwg-p2p at lists.mpi-forum.org>
> Cc: "Rolf Rabenseifner" <rabenseifner at hlrs.de>, "MPI-WG Fortran working group" <mpiwg-fortran at lists.mpi-forum.org>,
> "Anthony Skjellum" <tony at runtimecomputing.com>, "Reinhold Bader" <Reinhold.Bader at lrz-muenchen.de>, "Atsushi Hori"
> <ahori at riken.jp>, "fortran standards email list for J3" <j3 at j3-fortran.org>
> Sent: Friday, June 16, 2017 3:16:57 PM
> Subject: Re: [mpiwg-p2p] [MPIWG Fortran]  FP16 16-bit-floating-point

> Hi,
> 
> I don't know the status of C standardization nor contact person
> but the suggestion of 'short float' was seem to made in a WG.
> 
>  http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf
>  http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf
> 
> Regards,
> Takahiro Kawashima,
> Fujitsu
> 
>> Tony,
>> 
>> we try to find the type names used in C and Fortran before we define
>> the new predefined MPI datatype handles for representing those
>> language types.
>> 
>> Who knows, whom we can contact on the C standardization side?
>> 
>> Best regards
>> Rolf
>> 
>> ----- Original Message -----
>> > From: "Anthony Skjellum" <tony at runtimecomputing.com>
>> > To: "MPI WG Point To Point Communications working group"
>> > <mpiwg-p2p at lists.mpi-forum.org>
>> > Cc: "Reinhold Bader" <Reinhold.Bader at lrz-muenchen.de>, "Atsushi Hori"
>> > <ahori at riken.jp>, "fortran standards email list
>> > for J3" <j3 at j3-fortran.org>, "MPI-WG Fortran working group"
>> > <mpiwg-fortran at lists.mpi-forum.org>
>> > Sent: Friday, June 16, 2017 2:50:00 PM
>> > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point
>> 
>> > Rolf, apart from the Fortan angle, may I strongly encourage that we adopt this
>> > datatype for MPI-4. It is part of real MPI codes already, but now we have to
>> > fake transfer datatypes as 16-bit integers or bytes... which means not really
>> > right if heterogeneous interoperability were to be maintained. And this is a
>> > canonical IEEE type now, as you note. There is also the "2 half-float" datatype
>> > (32-bits) at least in CUDA to be looked at; maybe ignored or maybe supported,
>> > but it is there too. Key to this MPI users and applications use half floats and
>> > 2 half-floats :-) in practice.
>> > 
>> > May I also suggest a forward look at the types in AVX-512, in case any of these
>> > become particularly relevant in future.
>> > At least 8-bit integers are conceptually new, apart from BYTE. [
>> > https://en.wikipedia.org/wiki/AVX-512 | https://en.wikipedia.org/wiki/AVX-512 ]
>> > 
>> > Regards,
>> > Tony
>> > 
>> > 
>> > On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [
>> > mailto:rabenseifner at hlrs.de | rabenseifner at hlrs.de ] > wrote:
>> > 
>> > 
>> > Hi all together,
>> > 
>> > I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16
>> > datatype.
>> > 
>> > Questions to the Fortran atandardization body:
>> > 
>> > - Does the Fortran Standardization Body has already a proposal for
>> > the new name in Fortran for this IEEE FP16?
>> > 
>> > - Will be REAL*2 identical to IEEE FP16?
>> > 
>> > Best regards
> > > Rolf

-- 
Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de .
High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 .
University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 .
Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner .
Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .