<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Jim --
<div class=""><br class="">
</div>
<div class="">1. This is the first email (from Rolf) that started this journey:</div>
<div class=""><br class="">
</div>
<div class=""><a href="https://lists.mpi-forum.org/pipermail/mpiwg-large-counts/2019-September/000023.html" class="">https://lists.mpi-forum.org/pipermail/mpiwg-large-counts/2019-September/000023.html</a></div>
<div class=""><br class="">
</div>
<div class="">2. Then there were some off-list mails / replies (because the BigCount WG didn't know that this mailing lists existed -- oops!). This is the first re-entry of Rolf's email with some threaded replies on to this list:</div>
<div class=""><br class="">
</div>
<div class=""><a href="https://lists.mpi-forum.org/pipermail/mpiwg-large-counts/2019-October/000025.html" class="">https://lists.mpi-forum.org/pipermail/mpiwg-large-counts/2019-October/000025.html</a></div>
<div class=""><br class="">
</div>
<div class="">3. That thread ultimately led us here.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Oct 25, 2019, at 2:45 PM, Dinan, James <<a href="mailto:james.dinan@intel.com" class="">james.dinan@intel.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 20px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Jeff, thanks so much for opening up these old wounds. I’m not sure I have enough context to contribute to the discussion. Where can I read up on the issue with MPI_Aint?</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I’m glad to hear that C signed integers will finally have a well-defined representation.</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
~Jim.</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(181, 196, 223); padding: 3pt 0in 0in;" class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class=""><span style="font-size: 12pt;" class="">From:<span class="Apple-converted-space"> </span></span></b><span style="font-size: 12pt;" class="">Jeff Hammond <<a href="mailto:jeff.science@gmail.com" style="color: purple; text-decoration: underline;" class="">jeff.science@gmail.com</a>><br class="">
<b class="">Date:<span class="Apple-converted-space"> </span></b>Thursday, October 24, 2019 at 7:03 PM<br class="">
<b class="">To:<span class="Apple-converted-space"> </span></b>"Jeff Squyres (jsquyres)" <<a href="mailto:jsquyres@cisco.com" style="color: purple; text-decoration: underline;" class="">jsquyres@cisco.com</a>><br class="">
<b class="">Cc:<span class="Apple-converted-space"> </span></b>MPI BigCount Working Group <<a href="mailto:mpiwg-large-counts@lists.mpi-forum.org" style="color: purple; text-decoration: underline;" class="">mpiwg-large-counts@lists.mpi-forum.org</a>>, "Dinan,
James" <<a href="mailto:james.dinan@intel.com" style="color: purple; text-decoration: underline;" class="">james.dinan@intel.com</a>><br class="">
<b class="">Subject:<span class="Apple-converted-space"> </span></b>Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements<o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Jim (cc) suffered the most in MPI 3.0 days because of AINT_DIFF and AINT_SUM, so maybe he wants to create this ticket.</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Jeff</div>
</div>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div class="">
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Thu, Oct 24, 2019 at 2:41 PM Jeff Squyres (jsquyres) <<a href="mailto:jsquyres@cisco.com" style="color: purple; text-decoration: underline;" class="">jsquyres@cisco.com</a>> wrote:</div>
</div>
<blockquote style="border-style: none none none solid; border-left-width: 1pt; border-left-color: rgb(204, 204, 204); padding: 0in 0in 0in 6pt; margin: 5pt 0in 5pt 4.8pt;" class="" type="cite">
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Not opposed to ditching segmented addressing at all. We'd need a ticket for this ASAP, though.</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
This whole conversation is predicated on:</div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
- MPI supposedly supports segmented addressing</div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
- MPI_Aint is not sufficient for modern segmented addressing (i.e., representing an address that may not be in main RAM and is not mapped in to the current process' linear address space)</div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
If we no longer care about segmented addressing, that makes a whole bunch of BigCount stuff a LOT easier. E.g., MPI_Aint can basically be a non-segment-supporting address integer. AINT_DIFF and AINT_SUM can go away, too.</div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div class="">
<p class="MsoNormal" style="margin: 0in 0in 12pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<o:p class=""> </o:p></p>
<blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class="" type="cite">
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Oct 24, 2019, at 5:35 PM, Jeff Hammond via mpiwg-large-counts <<a href="mailto:mpiwg-large-counts@lists.mpi-forum.org" target="_blank" style="color: purple; text-decoration: underline;" class="">mpiwg-large-counts@lists.mpi-forum.org</a>> wrote:</div>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div class="">
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Rolf:</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<br class="">
Before anybody spends any time analyzing how we handle segmented addressing, I want you to provide an example of a platform where this is relevant. What system can you boot today that needs this and what MPI libraries have expressed an interest in supporting it?</div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
For anyone who didn't hear, ISO C and C++ have finally committed to twos-complement integers (<a href="http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0907r1.html" target="_blank" style="color: purple; text-decoration: underline;" class="">http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0907r1.html</a>, <a href="http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2218.htm" target="_blank" style="color: purple; text-decoration: underline;" class="">http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2218.htm</a>)
because modern programmers should not be limited by hardware designs from the 1960s. We should similarly not waste our time on obsolete features like segmentation.</div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Jeff</div>
</div>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div class="">
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Thu, Oct 24, 2019 at 10:13 AM Rolf Rabenseifner via mpiwg-large-counts <<a href="mailto:mpiwg-large-counts@lists.mpi-forum.org" target="_blank" style="color: purple; text-decoration: underline;" class="">mpiwg-large-counts@lists.mpi-forum.org</a>> wrote:</div>
</div>
<blockquote style="border-style: none none none solid; border-left-width: 1pt; border-left-color: rgb(204, 204, 204); padding: 0in 0in 0in 6pt; margin: 5pt 0in 5pt 4.8pt;" class="" type="cite">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
> I think that changes the conversation entirely, right?<br class="">
<br class="">
Not the first part, the state-of-current-MPI.<br class="">
<br class="">
It may change something for the future, or a new interface may be needed.<br class="">
<br class="">
Please, can you describe how MPI_Get_address can work with the<span class="Apple-converted-space"> </span><br class="">
different variables from different memory segments.<br class="">
<br class="">
Or whether a completely new function or a set of functions is needed.<br class="">
<br class="">
If we can still express variables from all memory segments as<span class="Apple-converted-space"> </span><br class="">
input to MPI_Get_address, there may be still a way to flatten<br class="">
the result of some internal address-iquiry into a flattened<br class="">
signed integer with the same behavior as MPI_Aint today.<br class="">
<br class="">
If this is impossible, then new way of thinking and solution<span class="Apple-converted-space"> </span><br class="">
may be needed.<br class="">
<br class="">
I really want to see examples for all current stuff as you<br class="">
mentioned in your last email.<br class="">
<br class="">
Best regards<br class="">
Rolf<br class="">
<br class="">
----- Original Message -----<br class="">
> From: "Jeff Squyres" <<a href="mailto:jsquyres@cisco.com" target="_blank" style="color: purple; text-decoration: underline;" class="">jsquyres@cisco.com</a>><br class="">
> To: "Rolf Rabenseifner" <<a href="mailto:rabenseifner@hlrs.de" target="_blank" style="color: purple; text-decoration: underline;" class="">rabenseifner@hlrs.de</a>><br class="">
> Cc: "mpiwg-large-counts" <<a href="mailto:mpiwg-large-counts@lists.mpi-forum.org" target="_blank" style="color: purple; text-decoration: underline;" class="">mpiwg-large-counts@lists.mpi-forum.org</a>><br class="">
> Sent: Thursday, October 24, 2019 5:27:31 PM<br class="">
> Subject: Re: [Mpiwg-large-counts] Large Count - the principles for counts, sizes, and byte and nonbyte displacements<br class="">
<br class="">
> On Oct 24, 2019, at 11:15 AM, Rolf Rabenseifner<br class="">
> <<a href="mailto:rabenseifner@hlrs.de" target="_blank" style="color: purple; text-decoration: underline;" class="">rabenseifner@hlrs.de</a><mailto:<a href="mailto:rabenseifner@hlrs.de" target="_blank" style="color: purple; text-decoration: underline;" class="">rabenseifner@hlrs.de</a>>>
wrote:<br class="">
><span class="Apple-converted-space"> </span><br class="">
> For me, it looked like that there was some misunderstanding<br class="">
> of the concept that absolute and relative addresses<br class="">
> and number of bytes that can be stored in MPI_Aint.<br class="">
><span class="Apple-converted-space"> </span><br class="">
> ...with the caveat that MPI_Aint -- as it is right now -- does not support<br class="">
> modern segmented memory systems (i.e., where you need more than a small number<br class="">
> of bits to indicate the segment where the memory lives).<br class="">
><span class="Apple-converted-space"> </span><br class="">
> I think that changes the conversation entirely, right?<br class="">
><span class="Apple-converted-space"> </span><br class="">
> --<br class="">
> Jeff Squyres<br class="">
><span class="Apple-converted-space"> </span><a href="mailto:jsquyres@cisco.com" target="_blank" style="color: purple; text-decoration: underline;" class="">jsquyres@cisco.com</a><mailto:<a href="mailto:jsquyres@cisco.com" target="_blank" style="color: purple; text-decoration: underline;" class="">jsquyres@cisco.com</a>><br class="">
<br class="">
--<span class="Apple-converted-space"> </span><br class="">
Dr. Rolf Rabenseifner . . . . . . . . . .. email<span class="Apple-converted-space"> </span><a href="mailto:rabenseifner@hlrs.de" target="_blank" style="color: purple; text-decoration: underline;" class="">rabenseifner@hlrs.de</a><span class="Apple-converted-space"> </span>.<br class="">
High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 .<br class="">
University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 .<br class="">
Head of Dpmt Parallel Computing . . .<span class="Apple-converted-space"> </span><a href="http://www.hlrs.de/people/rabenseifner" target="_blank" style="color: purple; text-decoration: underline;" class="">www.hlrs.de/people/rabenseifner</a><span class="Apple-converted-space"> </span>.<br class="">
Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .<br class="">
_______________________________________________<br class="">
mpiwg-large-counts mailing list<br class="">
<a href="mailto:mpiwg-large-counts@lists.mpi-forum.org" target="_blank" style="color: purple; text-decoration: underline;" class="">mpiwg-large-counts@lists.mpi-forum.org</a><br class="">
<a href="https://lists.mpi-forum.org/mailman/listinfo/mpiwg-large-counts" target="_blank" style="color: purple; text-decoration: underline;" class="">https://lists.mpi-forum.org/mailman/listinfo/mpiwg-large-counts</a></div>
</blockquote>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<br clear="all" class="">
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
--</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Jeff Hammond<br class="">
<a href="mailto:jeff.science@gmail.com" target="_blank" style="color: purple; text-decoration: underline;" class="">jeff.science@gmail.com</a><br class="">
<a href="http://jeffhammond.github.io/" target="_blank" style="color: purple; text-decoration: underline;" class="">http://jeffhammond.github.io/</a></div>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
_______________________________________________<br class="">
mpiwg-large-counts mailing list<br class="">
<a href="mailto:mpiwg-large-counts@lists.mpi-forum.org" target="_blank" style="color: purple; text-decoration: underline;" class="">mpiwg-large-counts@lists.mpi-forum.org</a><br class="">
<a href="https://lists.mpi-forum.org/mailman/listinfo/mpiwg-large-counts" target="_blank" style="color: purple; text-decoration: underline;" class="">https://lists.mpi-forum.org/mailman/listinfo/mpiwg-large-counts</a></div>
</div>
</blockquote>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<br class="">
-- <br class="">
Jeff Squyres<br class="">
<a href="mailto:jsquyres@cisco.com" target="_blank" style="color: purple; text-decoration: underline;" class="">jsquyres@cisco.com</a></div>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
</div>
</blockquote>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<br clear="all" class="">
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
--</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Jeff Hammond<br class="">
<a href="mailto:jeff.science@gmail.com" target="_blank" style="color: purple; text-decoration: underline;" class="">jeff.science@gmail.com</a><br class="">
<a href="http://jeffhammond.github.io/" target="_blank" style="color: purple; text-decoration: underline;" class="">http://jeffhammond.github.io/</a></div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
<div class=""><span class=""><br class="">
-- <br class="">
</span><span class="">Jeff Squyres<br class="">
</span><span class=""><a href="mailto:jsquyres@cisco.com" class="">jsquyres@cisco.com</a></span>
</div>
<br class="">
</div>
</body>
</html>