[mpiwg-p2p] [MPIWG Fortran] FP16 16-bit-floating-point

Jeff Hammond jeff.science at gmail.com
Fri Jun 16 14:34:21 CDT 2017


> (we never did the MAX/MINLOC reductions also for fixed-size types)

Yes, because it leads to an absurd explosion of enumerations, which is why
I proposed a utility routine to construct pair types that behave like
built-ins (
https://github.com/mpi-forum/mpi-issues/issues/18#issuecomment-155661199).

While it's embarrassing that MPI has failed to keep up with ISO language
built-in types, I console myself with the fact that I can implement all of
this in BigMPI or MPICH in a day or two and Open-MPI with more effort.

I don't know where float128_t sits w.r.t. ISO standardization in C or C++
but users have been asking for __float128 support for a long time to no
avail.

Jeff

On Fri, Jun 16, 2017 at 11:04 AM, Rolf Rabenseifner <rabenseifner at hlrs.de>
wrote:

> Atsushi-san, (now only in the P2P-WG)
>
> >  http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf
> >  http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf
>
> Therefore for the C-side, I would propose for
> (oc course voted in only after the C standardization gets confirmed)
>
> MPI-3.1 table on page 673
>
>  Named Predefined Datatypes | C types
>  ---------------------------+------------
>  MPI_SHORT_FLOAT            | short float
>  MPI_FLOAT16_T              | float16_t
>  MPI_FLOAT32_T              | float32_t
>  MPI_FLOAT64_T              | float64_t
>
> (We must check, whether they also include float128_t into the C Standard,
>  or whether they only standardize short float without all ..._T)
>
> MPI-3.1 table on page 675 lines 1-9
>
>  Datatypes for reduction functions (C)
>  -------------------------------------
>  MPI_SHORT_FLOAT_INT
>
> (we never did the MAX/MINLOC reductions also for fixed-size types)
>
> MPI-3.1 Table 13.2 on page 540 ***and***
> MPI-3.1 Table 3.2 on page 26
>
>  Type            | Length
>  ----------------+-------
>  MPI_SHORT_FLOAT |   2
>  MPI_FLOAT16_T   |   2
>  MPI_FLOAT32_T   |   4
>  MPI_FLOAT64_T   |   8
>
> And adding the following to "Floating point:"
> on page 176, line 45:
>
> - before MPI_FLOAT:
>      MPI_SHORT_FLOAT,
> - after MPI_DOUBLE:
>      MPI_FLOAT16_T, MPI_FLOAT32_T, MPI_FLOAT64_T,
>
> By the way, the table on page 540 should be done
> with normal Latex and the correct macros for constants
> to guarantee that this Location is also visible in the
> constants-index page 823ff.
>
> You may have detected further Locations that are relevant.
>
> And of course, the appropriate change-log index is also needed.
>
> Best regards
> Rolf
>
> ----- Original Message -----
> > From: "Kawashima, Takahiro" <t-kawashima at jp.fujitsu.com>
> > To: "MPI WG Point To Point Communications working group" <
> mpiwg-p2p at lists.mpi-forum.org>
> > Cc: "Rolf Rabenseifner" <rabenseifner at hlrs.de>, "MPI-WG Fortran working
> group" <mpiwg-fortran at lists.mpi-forum.org>,
> > "Anthony Skjellum" <tony at runtimecomputing.com>, "Reinhold Bader" <
> Reinhold.Bader at lrz-muenchen.de>, "Atsushi Hori"
> > <ahori at riken.jp>, "fortran standards email list for J3" <
> j3 at j3-fortran.org>
> > Sent: Friday, June 16, 2017 3:16:57 PM
> > Subject: Re: [mpiwg-p2p] [MPIWG Fortran]  FP16 16-bit-floating-point
>
> > Hi,
> >
> > I don't know the status of C standardization nor contact person
> > but the suggestion of 'short float' was seem to made in a WG.
> >
> >  http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0192r0.pdf
> >  http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2016.pdf
> >
> > Regards,
> > Takahiro Kawashima,
> > Fujitsu
> >
> >> Tony,
> >>
> >> we try to find the type names used in C and Fortran before we define
> >> the new predefined MPI datatype handles for representing those
> >> language types.
> >>
> >> Who knows, whom we can contact on the C standardization side?
> >>
> >> Best regards
> >> Rolf
> >>
> >> ----- Original Message -----
> >> > From: "Anthony Skjellum" <tony at runtimecomputing.com>
> >> > To: "MPI WG Point To Point Communications working group"
> >> > <mpiwg-p2p at lists.mpi-forum.org>
> >> > Cc: "Reinhold Bader" <Reinhold.Bader at lrz-muenchen.de>, "Atsushi Hori"
> >> > <ahori at riken.jp>, "fortran standards email list
> >> > for J3" <j3 at j3-fortran.org>, "MPI-WG Fortran working group"
> >> > <mpiwg-fortran at lists.mpi-forum.org>
> >> > Sent: Friday, June 16, 2017 2:50:00 PM
> >> > Subject: Re: [MPIWG Fortran] [mpiwg-p2p] FP16 16-bit-floating-point
> >>
> >> > Rolf, apart from the Fortan angle, may I strongly encourage that we
> adopt this
> >> > datatype for MPI-4. It is part of real MPI codes already, but now we
> have to
> >> > fake transfer datatypes as 16-bit integers or bytes... which means
> not really
> >> > right if heterogeneous interoperability were to be maintained. And
> this is a
> >> > canonical IEEE type now, as you note. There is also the "2
> half-float" datatype
> >> > (32-bits) at least in CUDA to be looked at; maybe ignored or maybe
> supported,
> >> > but it is there too. Key to this MPI users and applications use half
> floats and
> >> > 2 half-floats :-) in practice.
> >> >
> >> > May I also suggest a forward look at the types in AVX-512, in case
> any of these
> >> > become particularly relevant in future.
> >> > At least 8-bit integers are conceptually new, apart from BYTE. [
> >> > https://en.wikipedia.org/wiki/AVX-512 |
> https://en.wikipedia.org/wiki/AVX-512 ]
> >> >
> >> > Regards,
> >> > Tony
> >> >
> >> >
> >> > On Fri, Jun 16, 2017 at 6:52 AM, Rolf Rabenseifner < [
> >> > mailto:rabenseifner at hlrs.de | rabenseifner at hlrs.de ] > wrote:
> >> >
> >> >
> >> > Hi all together,
> >> >
> >> > I'm sitting in the MPI Forum Meeting and we discuss the new IEEE FP16
> >> > datatype.
> >> >
> >> > Questions to the Fortran atandardization body:
> >> >
> >> > - Does the Fortran Standardization Body has already a proposal for
> >> > the new name in Fortran for this IEEE FP16?
> >> >
> >> > - Will be REAL*2 identical to IEEE FP16?
> >> >
> >> > Best regards
> > > > Rolf
>
> --
> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de .
> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 .
> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 .
> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner .
> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .
> _______________________________________________
> mpiwg-p2p mailing list
> mpiwg-p2p at lists.mpi-forum.org
> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-p2p
>



-- 
Jeff Hammond
jeff.science at gmail.com
http://jeffhammond.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-p2p/attachments/20170616/29021ee1/attachment-0001.html>


More information about the mpiwg-p2p mailing list