[Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements
HOLMES Daniel
d.holmes at epcc.ed.ac.uk
Tue Nov 5 15:15:53 CST 2019
Hi Rolf,
Thanks. I’ve copied your response into the the issue to keep it all in one place (so we don’t have to search email to find it).
Cheers,
Dan.
—
Dr Daniel Holmes PhD
Architect (HPC Research)
d.holmes at epcc.ed.ac.uk<mailto:d.holmes at epcc.ed.ac.uk>
Phone: +44 (0) 131 651 3465
Mobile: +44 (0) 7940 524 088
Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT
—
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
—
On 5 Nov 2019, at 17:58, Rolf Rabenseifner <rabenseifner at hlrs.de<mailto:rabenseifner at hlrs.de>> wrote:
Sorry, I now detected that you wanted to point me also to
----- Jeff Hammond <jeff.science at gmail.com<mailto:jeff.science at gmail.com><mailto:jeff.science at gmail.com>>
wrote:
Rolf:
Have you looked at
https://github.com/mpiwg-large-count/large-count-issues/issues/6 ?
where I found your text. Thank you for remembering me.
About your choices:
1.change the name of MPI_Type_get_extext_x to MPI_Type_get_extext_l to fit in with all the other _l functions for large count. This is a bad idea because the MPI_Count variant is not (just) embiggening the MPI_Aint variant, it is providing a different role (target files, not absolute memory).
Agreed, bad idea.
2.add a new MPI_Type_get_extext_l that uses MPI_Offset. This is superfluous because we already have the _x variant with MPI_Count.
Agreed, bad idea, because there is no "int Count" Argument.
3.leave it well alone. This seems like the best approach.
Partially agreed, because for the _x Versions in the derived datatype chapter,
I would always use MPI_Count and not MPI_Offset, because one never know
for which other purpose derived datatypes may be used in the future..
The concept is not restricted to Memory and I/O.
I/O is only an example why we need to provide these _x Versions.
Therefore I prefere:
4. leave it. It is the starting point for the other _X versions.
Small diff between these two _x routines and the new ones is
that for each new one, it does not have a visible Fortran _X,
only the overloaded one,
and each existing one must get the additional overloaded interface.
As far as I understand Fortran, there will be then only one backend routine
with two different interfaces, because the backend is the same _X
routine for the explicit and for the overloaded MPI_Count routine.
Best regards
Rolf
----- Original Message -----
From: "mpiwg-large-counts" <mpiwg-large-counts at lists.mpi-forum.org<mailto:mpiwg-large-counts at lists.mpi-forum.org>>
To: "HOLMES Daniel" <d.holmes at epcc.ed.ac.uk<mailto:d.holmes at epcc.ed.ac.uk>>
Cc: "Rolf Rabenseifner" <rabenseifner at hlrs.de<mailto:rabenseifner at hlrs.de>>, "mpiwg-large-counts" <mpiwg-large-counts at lists.mpi-forum.org<mailto:mpiwg-large-counts at lists.mpi-forum.org>>
Sent: Tuesday, November 5, 2019 6:10:25 PM
Subject: Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements
Hi Dan,
our mailers do not want to work together.
I cannot find your "lengthy comment on that issue" in this email.
As you can see below, it is flattened.
Originally , it was with huge indents, but without any ">".
Text looked then líke
d
d
r
e
s
s
e
s
In the telcon, we completely aggreed that the large count versions MPI...._l
should keep all MPI_Aint and should add MPI_Aint where it was missing
by being wrong (MPI_Alltoall_w and MPI_(Un)pack.
We completely agreed that all the MPI_Aint discussion is done.
MPI_Aint is an signed integer mis-used to also store absoulte
adresses whatever there bits may mean. And we do not change this
nor we introduce any new such strange type.
And we agreed that there should be for the few derived datatype routines
with MPI_Aint an additional set with MPI_Count (or MPI_Offset, which would
be wierd) instead of MPI_Aint.
These MPI_Count versions do not allow the use of absolute addresses.
These MPI_Count versions are needed for the case that sizeof(MPI_Aint)
is less than sizeof(MPI_Count), may de 4:8 or 8.12 or 8:16,
depending on the memory size per MPI process and the filesystem size.
I only proposed in my latest email, that exactly two of them already exist:
MPI_Type_get_extent_x and MPI_Type_get_true_extent_x
and we should keep them and take the postfix _x for this set of routines
instead of throwing them away and reinvernting them again.
For ... I would say,
- MPI_TYPE_SIZE_X
the _l Version should have MPI_Aint size because it is a byte size.
- MPI_GET_ELEMENTS_X and MPI_STATUS_SET_ELEMENTS_X
in my opinion, we should keep _X, and the _l version
is then a duplicate of it.
Best regards
Rolf
----- Original Message -----
From: "HOLMES Daniel" <d.holmes at epcc.ed.ac.uk<mailto:d.holmes at epcc.ed.ac.uk>>
To: "Rolf Rabenseifner" <rabenseifner at hlrs.de<mailto:rabenseifner at hlrs.de>>
Cc: "mpiwg-large-counts" <mpiwg-large-counts at lists.mpi-forum.org<mailto:mpiwg-large-counts at lists.mpi-forum.org>>
Sent: Tuesday, November 5, 2019 3:18:13 PM
Subject: Re: [Mpiwg-large-counts] Large Count - the principles for counts,
sizes, and byte and nonbyte displacements
Hi Rolf (et al),
I wrote a lengthy comment on that issue to capture my current understanding of
your “really wrong” assertion.
Broadly, we agree - I just wanted to write down the reasoning and nuances of
that outcome.
Cheers,
Dan.
—
Dr Daniel Holmes PhD
Architect (HPC Research)
d.holmes at epcc.ed.ac.uk<mailto:d.holmes at epcc.ed.ac.uk><mailto:d.holmes at epcc.ed.ac.uk>
Phone: +44 (0) 131 651 3465
Mobile: +44 (0) 7940 524 088
Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT
—
The University of Edinburgh is a charitable body, registered in Scotland, with
registration number SC005336.
—
On 2 Nov 2019, at 09:13, Rolf Rabenseifner via mpiwg-large-counts
<mpiwg-large-counts at lists.mpi-forum.org<mailto:mpiwg-large-counts at lists.mpi-forum.org><mailto:mpiwg-large-counts at lists.mpi-forum.org>>
wrote:
After the Telcon it seems that this ticket is really wrong.
Some or all of the routines may/should be kept.
And it these routines arean essential part of the future large count concept.
Thank you very much for pointing us to this ticket.
Rolf
----- Jeff Hammond <jeff.science at gmail.com<mailto:jeff.science at gmail.com><mailto:jeff.science at gmail.com>>
wrote:
Rolf:
Have you looked at
https://github.com/mpiwg-large-count/large-count-issues/issues/6?
Jeff
On Fri, Nov 1, 2019 at 1:00 AM Rolf Rabenseifner <rabenseifner at hlrs.de>
wrote:
A small comment on the result of our telcon:
- Postfix _l for int -> MPI_Count
- Postfix _x for additionally
MPI_Aint -> MPI_Count
I.e., the additional routines in the derived datatype chapter.
Two of them already exist
MPI_Type_get_(true)extent_x
In Fortran we will have then for the
same routine two aliases:
- the overload one without _x
- and the explicit one with _x
For both ones, the internal function name is the same, with _x.
Best regards
Rolf
----- Jeff Hammond <jeff.science at gmail.com> wrote:
On Thu, Oct 31, 2019 at 7:48 AM Rolf Rabenseifner <rabenseifner at hlrs.de>
wrote:
Dear all,
here my summary as input for our telcon today.
In principle, it is a very simple question:
with large Counts, do we
- keep all MPI_Aint
- or do we substitute MPI_Aint by MPI_Count?
I haven't been involved as much lately but did we not use MPI_Count for
count and element displacements in the large count proposal? We need to
use MPI_Aint for offsets into memory because that is what this type is
for.
Jeff
In principle, the MPI Forum answered this question already
for MPI-3.0 in 2012 with a clear YES:
int MPI_Type_get_extent(MPI_Datatype datatype,
MPI_Aint *lb, MPI_Aint *extent)
int MPI_Type_get_extent_x(MPI_Datatype datatype,
MPI_Count *lb, MPI_Count *extent)
About Jeff H. question:
If we limit the API to not support MPI_Count
means that an MPI implementation has not really such quality options
when using I/O fileviews, because the API is restricted to
MPI_Aint (which should be implemented based on the, e.g.,
64bit memory system).
About Jim's comment:
Apologies, it's been a while since I looked at the I/O interfaces.
If
I/O
only needs relative displacements that have normal integer
semantics,
then
I don't see why MPI_Count would not work for this purpose. If you
have
an
MPI_Aint that contains a relative displacement, it also has normal
integer
semantics and can be converted to an MPI_Count.
Yes, but this automatically implies that the datatypes must also
be able to handle MPI_Count.
The only case we really
need to look out for is when an integer type contains an absolute
address.
In those cases, the quantity in the variable cannot be treated as a
normal
integer and we need special routines to work with it.
Yes, this happens when we extend MPI_Aint in the derived datatype
routines
to MPI_Count.
But in principle, this is not a big Problem, as you all could see in
the previous emails:
- We must do for MPI_Count the same as we did for MPI_Aint,
i.e., we'll have long versions of the routines
MPI_Get_address, MPI_Aint_diff, MPI_Aint_add
- And we must ensure that the type cast from MPI_Aint to
MPI_Count works, which is a small new advice to implementors
for MPI_Det_address.
Therefore again my 4 questions:
- Should the new large count routines be prepared for
more than 10 or 20 Exabyte files where we need 64/65 or
or 65/66 unsigned/signed integers for relative byte
displacements or byte counts?
If yes, then all MPI_Aint arguments must be substituted by MPI_Count.
(In other words, do we want to be prepared for another 25 years of
MPI?
:-)
As stated above, the MPI-Forum already decided 2012 with a YES.
- Should we allow that these new routines are also used for memory
description,
where we typically need only the large MPI_Count "count" arguments?
(or should we provide two different new routines for each routine
that
currently has int Count/... and MPI_Aint disp/... arguments)
I expect, that nobody wants to have two different large versions of
for example MPI_Type_create_struct.
- Should we allow a mix of old and new routines, especially for
memory-based
usage, that old-style MPI_Get_address is used to retrieve an absolute
address and then, e.g., new style MPI_Type_create_struct with
MPI_Count blocklength and displacements is used?
I expect that forbidding such a mix would be a problem for Software
development.
Often old-style modules must work together with new-style modules.
- Do we want to require for this type cast of MPI_Aint addr into
MPI_Count
that it is allowed to do this cast with a normal assignment, rather
than
a special MPI function?
I expect yes, because for must usage of MPI_Aint and MPI_Count,
it is for relative displacements or byte counts, i.e. for normal
integers and therefore automatic type cast between MPI_Aint
and MPI_Count is a must.
With yes to all four questions, the proposed solution above is
the easiest way.
Hope to see/hear you today in our telcon.
Best regards
Rolf
--
Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de<mailto:rabenseifner at hlrs.de> .
High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 .
University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 .
Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner<http://www.hlrs.de/people/rabenseifner> .
Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-large-counts/attachments/20191105/2ac2d228/attachment-0001.html>
More information about the mpiwg-large-counts
mailing list