<div dir="ltr"><div>Just implement MPI allocation sanity once and forget about it :-)</div><div><br></div><div>#include <mpi.h></div><div>void * MPIX_Malloc(MPI_Aint size, MPI_Info info, int * status)</div><div>{</div><div> void * ptr = NULL;</div><div> *status = MPI_Alloc_mem(size, info, &ptr);</div><div> return ptr;</div><div>}</div><div><br></div><div>int MPIX_Free(void * ptr)</div><div>{</div><div> return MPI_Free_mem(ptr);</div><div>}</div><div><br></div><div><br></div><div>Jeff</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Oct 14, 2020 at 12:53 PM Skjellum, Anthony via mpi-forum <<a href="mailto:mpi-forum@lists.mpi-forum.org">mpi-forum@lists.mpi-forum.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Bill, I thank you much for your input too.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
POSIX has no problem making you do the cast in C <span id="gmail-m_1054232308686694120🙂">🙂 </span></div>
<pre style="font-family:monospace;padding:0px 2em 0px 0px;color:rgb(68,68,68);background-color:rgb(255,255,255)"><b style="font-weight:bold">int posix_memalign(void **</b><i style="font-style:italic">memptr</i><b style="font-weight:bold">, size_t</b> <i style="font-style:italic">alignment</i><b style="font-weight:bold">, size_t</b> <i style="font-style:italic">size</i><b style="font-weight:bold">);</b></pre>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
I get why its that way now in MPI, I still don't like it <span id="gmail-m_1054232308686694120🙂">🙂</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Tony</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
cf,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<div style="margin:0px;font-size:12pt;font-family:Calibri,Arial,Helvetica,sans-serif">
<a href="https://linux.die.net/man/3/posix_memalign" style="margin:0px" target="_blank">https://linux.die.net/man/3/posix_memalign</a><br>
</div>
<div style="margin:0px;font-size:14px">
<div style="margin:16px 0px;max-width:800px;min-width:424px">
<table style="padding:12px 36px 12px 12px;width:424px;border-width:1px;border-style:solid;border-color:rgb(200,200,200);border-radius:2px">
<tbody>
<tr style="border-spacing:0px">
<td style="width:368px">
<div style="margin:0px 8px 12px 0px;font-weight:300;font-size:21px;font-family:wf_segoe-ui_light,"Segoe UI Light","Segoe WP Light","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif">
<a href="https://linux.die.net/man/3/posix_memalign" style="margin:0px" target="_blank">posix_memalign(3): allocate aligned memory - Linux man page</a></div>
<div style="margin:0px 8px 12px 0px;font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif;color:rgb(102,102,102);max-height:100px;overflow:hidden">
The function posix_memalign() allocates size bytes and places the address of the allocated memory in *memptr. The address of the allocated memory will be a ...</div>
<div style="margin:0px;font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif;color:rgb(166,166,166)">
<a href="http://linux.die.net" target="_blank">linux.die.net</a></div>
</td>
</tr>
</tbody>
</table>
<div title="Remove link preview" style="margin:0px;height:32px;text-align:center">
<span style="margin:auto;font-weight:300;font-size:17px;line-height:32px;font-family:controlIcons;speak:none;display:block"></span></div>
<div title="Remove link preview" style="margin:0px;height:32px;text-align:center">
<span style="margin:auto;font-weight:300;font-size:17px;line-height:32px;font-family:controlIcons;speak:none;display:block"></span></div>
<div title="Remove link preview" style="margin:0px;height:32px;text-align:center">
<span style="margin:auto;font-weight:300;font-size:17px;line-height:32px;font-family:controlIcons;speak:none;display:block"></span></div>
</div>
</div>
<pre style="font-size:14px;background-color:rgb(255,255,255);font-family:monospace;padding:0px 2em 0px 0px;color:rgb(68,68,68)"><b style="font-weight:bold"><br></b></pre>
<br>
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div id="gmail-m_1054232308686694120Signature">
<div>
<div id="gmail-m_1054232308686694120divtagdefaultwrapper" dir="ltr" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif">
<p style="margin-top:0px;margin-bottom:0px">Anthony Skjellum, PhD</p>
<p style="margin-top:0px;margin-bottom:0px">Professor of Computer Science and Chair of Excellence</p>
<p style="margin-top:0px;margin-bottom:0px">Director, SimCenter</p>
<p style="margin-top:0px;margin-bottom:0px">University of Tennessee at Chattanooga (UTC)</p>
<p style="margin-top:0px;margin-bottom:0px"><a href="mailto:tony-skjellum@utc.edu" target="_blank">tony-skjellum@utc.edu</a> [or <a href="mailto:skjellum@gmail.com" target="_blank">skjellum@gmail.com</a>]</p>
<p style="margin-top:0px;margin-bottom:0px">cell: 205-807-4968</p>
<p style="margin-top:0px;margin-bottom:0px"><br>
</p>
</div>
</div>
</div>
<div id="gmail-m_1054232308686694120appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_1054232308686694120divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> William Gropp <<a href="mailto:wgropp@illinois.edu" target="_blank">wgropp@illinois.edu</a>><br>
<b>Sent:</b> Wednesday, October 14, 2020 3:48 PM<br>
<b>To:</b> Main MPI Forum mailing list <<a href="mailto:mpi-forum@lists.mpi-forum.org" target="_blank">mpi-forum@lists.mpi-forum.org</a>><br>
<b>Cc:</b> Rolf Rabenseifner <<a href="mailto:rabenseifner@hlrs.de" target="_blank">rabenseifner@hlrs.de</a>>; Skjellum, Anthony <<a href="mailto:Tony-Skjellum@utc.edu" target="_blank">Tony-Skjellum@utc.edu</a>><br>
<b>Subject:</b> Re: [Mpi-forum] Question about MPI_Alloc_mem</font>
<div> </div>
</div>
<div style="overflow-wrap: break-word;">My previous email (probably crossing paths with Tony’s reply) addressed most of this. The one thing that I want to add is that the API choice made in MPI for the C binding, both in the
MPI-1 attribute get functions and here, has nothing to do with the Fortran binding. These were made to match C’s limitations on anonymous pointers, and to avoid requiring pointer casts.
<div>
<div><br>
</div>
<div>Bill</div>
<div><br>
<div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
<div style="overflow-wrap: break-word;">
<div style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">
William Gropp<br>
Director and Chief Scientist, NCSA<br>
Thomas M. Siebel Chair in Computer Science<br>
University of Illinois Urbana-Champaign</div>
<br>
</div>
</div>
<br>
</div>
<br>
</div>
<br>
<br>
</div>
<div><br>
<blockquote type="cite">
<div>On Oct 14, 2020, at 2:42 PM, Skjellum, Anthony via mpi-forum <<a href="mailto:mpi-forum@lists.mpi-forum.org" target="_blank">mpi-forum@lists.mpi-forum.org</a>> wrote:</div>
<br>
<div>
<div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
Rolf, the rationale is not clear at all to me: "to facilitate type casting" is self-understood or a canonical term of art. I've been programming in C for 40 years... and I never would write the API as we have done it there. </div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
For pointers, out arguments are literally truthful: void **ptr is an out argument of a void pointer, and void *ptr is an in argument of a void pointer.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
{Years ago, before there was void *, we would have written char **ptr for a char out argument (and cast it to whatever type we wanted), and char *ptr for an in point argument. }</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
The fact that a void ** can stand in the place of a void * is clearly a weakened typing in the language, because void * can point at anything (unknown). Since a void * is a pointer, it can also hold a pointer to a pointer to void, sure. But the intent of
the second * is to remind you that you have an out argument. And you *must* pass a pointer to a pointer for the API to work in C.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
Somehow this is helping the Fortran and C_PTR API of Fortran, as it states. So the C API is made this way for convenience of Fortran.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
The rationale shows the argument with the & to pass the pointer to the API by reference. But, the rationale is not necessarily normative. A reasonable person reads the standard, sees void * and passes the baseptr without the &. </div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
Seems like a bad API to me.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
Tony</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div id="gmail-m_1054232308686694120x_Signature">
<div>
<div id="gmail-m_1054232308686694120x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt;font-family:Calibri,Helvetica,sans-serif">
<div style="margin-top:0px;margin-bottom:0px">Anthony Skjellum, PhD</div>
<div style="margin-top:0px;margin-bottom:0px">Professor of Computer Science and Chair of Excellence</div>
<div style="margin-top:0px;margin-bottom:0px">Director, SimCenter</div>
<div style="margin-top:0px;margin-bottom:0px">University of Tennessee at Chattanooga (UTC)</div>
<div style="margin-top:0px;margin-bottom:0px"><a href="mailto:tony-skjellum@utc.edu" target="_blank">tony-skjellum@utc.edu</a><span> </span> [or<span> </span><a href="mailto:skjellum@gmail.com" target="_blank">skjellum@gmail.com</a>]</div>
<div style="margin-top:0px;margin-bottom:0px">cell: 205-807-4968</div>
<div style="margin-top:0px;margin-bottom:0px"><br>
</div>
</div>
</div>
</div>
</div>
<div id="gmail-m_1054232308686694120x_appendonsend" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
</div>
<hr style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;display:inline-block;width:662.469px">
<span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline"></span>
<div id="gmail-m_1054232308686694120x_divRplyFwdMsg" dir="ltr" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<font face="Calibri, sans-serif" style="font-size:11pt"><b>From:</b><span> </span>mpi-forum <<a href="mailto:mpi-forum-bounces@lists.mpi-forum.org" target="_blank">mpi-forum-bounces@lists.mpi-forum.org</a>> on behalf
of Rolf Rabenseifner via mpi-forum <<a href="mailto:mpi-forum@lists.mpi-forum.org" target="_blank">mpi-forum@lists.mpi-forum.org</a>><br>
<b>Sent:</b><span> </span>Wednesday, October 14, 2020 1:32 PM<br>
<b>To:</b><span> </span>Main MPI Forum mailing list <<a href="mailto:mpi-forum@lists.mpi-forum.org" target="_blank">mpi-forum@lists.mpi-forum.org</a>><br>
<b>Cc:</b><span> </span>Rolf Rabenseifner <<a href="mailto:rabenseifner@hlrs.de" target="_blank">rabenseifner@hlrs.de</a>><br>
<b>Subject:</b><span> </span>Re: [Mpi-forum] Question about MPI_Alloc_mem</font>
<div> </div>
</div>
<div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<font size="2"><span style="font-size:11pt">
<div>Dear Tony,<span> </span><br>
<br>
please read MPI-3.1 page 338, lines 36-41.<br>
Do these lines resolve your question?<br>
<br>
Best regards<br>
Rolf<span> </span><br>
<br>
----- Original Message -----<br>
> From: "Main MPI Forum mailing list" <<a href="mailto:mpi-forum@lists.mpi-forum.org" target="_blank">mpi-forum@lists.mpi-forum.org</a>><br>
> To: "Main MPI Forum mailing list" <<a href="mailto:mpi-forum@lists.mpi-forum.org" target="_blank">mpi-forum@lists.mpi-forum.org</a>><br>
> Cc: "Anthony Skjellum" <<a href="mailto:skjellum@gmail.com" target="_blank">skjellum@gmail.com</a>><br>
> Sent: Wednesday, October 14, 2020 7:01:54 PM<br>
> Subject: [Mpi-forum] Question about MPI_Alloc_mem<br>
<br>
> Folks, I know we have had this function for a long time, and I've implemented<br>
> ports of MPI that actually use it (e.g., with pre-pinned memory). But, I am<br>
> trying to understand the logic for why baseptr is passed by value, instead of<br>
> by reference. In C, everything is by value, so the last argument in normal C<br>
> programs would be void **baseptr.<br>
><span> </span><br>
> The standard has:<br>
> int MPI_Alloc_mem(MPI_Aint size, MPI_Info info, void *baseptr);<br>
> Now, MPICC/GCC takes this with<br>
> void *memory = (void *)0;<br>
> int error = MPI_Alloc_mem(1024, MPI_INFO_NULL, &memory);<br>
> and you get the memory allocated properly.<br>
> What is more, this is incorrect:<br>
> int error = MPI_Alloc_mem(1024, MPI_INFO_NULL, memory);<br>
> although it compiles fine because that is, indeed, the API.<br>
> Why would we have the pointer going in by value, when it is coming out<br>
> as an OUT argument?<br>
> Isn't this plain wrong? void **baseptr means ---> I am passing you the<br>
> address of a void pointer.<br>
> Every viable implementation must do *(void **)baseptr = ...<br>
> when providing the "malloc'd/special" memory. So... why did we fudge<br>
> the C API. Is there some tie-in with the Fortran API?<br>
> Thanks in advance,<br>
> Tony Skjellum<br>
><span> </span><br>
><span> </span><br>
> --<br>
> Anthony Skjellum, PhD<br>
> [<span> </span><a href="mailto:skjellum@gmail.com" target="_blank">mailto:skjellum@gmail.com</a><span> </span>|<span> </span><a href="mailto:skjellum@gmail.com" target="_blank">skjellum@gmail.com</a><span> </span>]<br>
> Cell: +1-205-807-4968<br>
><span> </span><br>
><span> </span><br>
><span> </span><br>
> _______________________________________________<br>
> mpi-forum mailing list<br>
><span> </span><a href="mailto:mpi-forum@lists.mpi-forum.org" target="_blank">mpi-forum@lists.mpi-forum.org</a><br>
><span> </span><a href="https://lists.mpi-forum.org/mailman/listinfo/mpi-forum" target="_blank">https://lists.mpi-forum.org/mailman/listinfo/mpi-forum</a><br>
<br>
--<span> </span><br>
Dr. Rolf Rabenseifner . . . . . . . . . .. email<span> </span><a href="mailto:rabenseifner@hlrs.de" target="_blank">rabenseifner@hlrs.de</a><span> </span>.<br>
High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 .<br>
University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 .<br>
Head of Dpmt Parallel Computing . . .<span> </span><a href="http://www.hlrs.de/people/rabenseifner" target="_blank">www.hlrs.de/people/rabenseifner</a><span> </span>.<br>
Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) .<br>
_______________________________________________<br>
mpi-forum mailing list<br>
<a href="mailto:mpi-forum@lists.mpi-forum.org" target="_blank">mpi-forum@lists.mpi-forum.org</a><br>
<a href="https://lists.mpi-forum.org/mailman/listinfo/mpi-forum" target="_blank">https://lists.mpi-forum.org/mailman/listinfo/mpi-forum</a><br>
</div>
</span></font></div>
<span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">_______________________________________________</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">mpi-forum
mailing list</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<a href="mailto:mpi-forum@lists.mpi-forum.org" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank">mpi-forum@lists.mpi-forum.org</a><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none">
<a href="https://lists.mpi-forum.org/mailman/listinfo/mpi-forum" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank">https://lists.mpi-forum.org/mailman/listinfo/mpi-forum</a></div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
_______________________________________________<br>
mpi-forum mailing list<br>
<a href="mailto:mpi-forum@lists.mpi-forum.org" target="_blank">mpi-forum@lists.mpi-forum.org</a><br>
<a href="https://lists.mpi-forum.org/mailman/listinfo/mpi-forum" rel="noreferrer" target="_blank">https://lists.mpi-forum.org/mailman/listinfo/mpi-forum</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature">Jeff Hammond<br><a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a><br><a href="http://jeffhammond.github.io/" target="_blank">http://jeffhammond.github.io/</a></div></div>