<div dir="ltr"><div class="gmail_quote"><div>Thanks to everyone who joined our nearly 2 hour meeting today. There were 12-15 people in attendance (I can produce the details if compelled).</div><div><br></div><div>We made the following decisions, which are of course not final in any sense, but will be reflected in the latest proposal once I have time to update it, and is the basis of the implementation work that Hui and I are doing.</div><div><br></div><div>0. MPI handles should be incomplete struct pointers. This guarantees type safety. Compilers will notice if users pass MPI_Comm instead of MPI_Group, for example. Some users and the Open-MPI developers have made this a high priority. A corollary to this is that the name of the struct becomes part of the ABI.</div><div><br></div><div>For example, we will specify:</div><div><br></div><div>typedef struct MPI_ABI_Comm * MPI_Comm;</div><div><br></div><div>and <i>all</i> of this will become part of the ABI. Rust, for example, needs to know the struct name.</div><div><br></div><div>1. MPI predefined handle constants should be constant integer expressions, not addresses to a named location in the implementation. While the debate was heated, the poll was unanimous in favor of this choice.</div><div><br></div><div>For example, this means that we have:</div><div><br></div><div>#define MPI_INT (MPI_Datatype)0x3;</div><div><br></div><div>This is legal in C and C++, and relies on implementation defined behavior that is already assumed in MPI (e.g. MPI_IN_PLACE) and known to work everywhere.</div><div><br></div><div>The reasons for this decision are:</div><div><br></div><div>a. Using link-time constant addresses as constant initializers is not completely portable. For example, it does not work on Windows, and Open-MPI does not support Windows because of this.</div><div><br></div><div>b. We can do efficient encoding of predefined handles if they are integer values. More on this below...</div><div><br></div><div>c. Compile-time constants will make it easier to standardize a Fortran ABI that uses the exact same handle values as C so there is zero overhead from Fortran (F08/F18). Fortran does not allow link-time resolution of initializer expressions, which implies overhead for implementations that do not have compile-time constants for the C handle constants (e.g. Open-MPI).</div><div><br></div><div>d. It makes it easier to define/use predefined handle constants from binary, which was raised as a use case from a tool perspective (e.g. Dynist and Papi).</div><div><br></div><div>The arguments against this decision were:</div><div><br></div><div>e. We will need to state and reserve the integer values used for this. In practice, this is portable because of the zero page, but is otherwise implementable because implementations can just verify that allocated handles do not conflict with the predefined ones.</div><div><br></div><div>2. MPI_<HANDLE>_NULL should not be 0 (i.e. NULL).</div><div><br></div><div>The arguments in favor of this were:</div><div><br></div><div>f. It makes it clear when uninitialized handles are used, as this will always be erroneous.</div><div><br></div><div>g. MPI_COMM_NULL has a name and is thus not the same thing as NULL would imply.</div><div><br></div><div>The arguments in against this decision were:</div><div><br></div><div>h. Zero initialization is common and some consider it convenient to have this cause handles to be automatically initialized to MPI_<HANDLE>_NULL.</div><div><br></div><div>3. We will attempt to Huffman encode predefined handles into a unique set of values, rather than have each set of handles be a sequence of integers.</div><div><br></div><div>Arguments in favor:</div><div><br></div><div>i. Unique numbers for every handle help debugging.</div><div><br></div><div>j. We can encode useful information in handles, especially for datatypes, which allows optimizations similar to what MPICH does today.</div><div><br></div><div>Arguments against:</div><div><br></div><div>k. A sequence of integers means that a translation layer can use an array to translate from application handles to implementation handles</div><div><br></div><div>4. Non-decision: we discussed but made no decision about symbol visibility. We will address this later.<br></div><div><br></div><div>In the next meeting, to be scheduled later, I will propose values for a bunch of constants and we can argue about them. In cases where MPICH and Open-MPI today agree, I will propose that we use those values. Most of this activity will happen on GitHub and the meeting will be to argue about any controversies.</div><div><br></div><div>A copy of these minutes will be uploaded to GitHub shortly.</div><div><br></div><div>Jeff</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
><br>
> On 2/20/23 03:55, Jeff Hammond wrote:<br>
> > For the meeting tomorrow, I would like you all to consider the<br>
> following:<br>
> ><br>
> > There is consensus regarding the following to get type safety:<br>
> > typedef struct MPI_ABI_Handle * MPI_Handle;<br>
> ><br>
> > We have two choices for predefined handles:<br>
> ><br>
> > Option 1:<br>
> > #define MPI_INT (MPI_Datatype)0x3<br>
> ><br>
> > Option 2:<br>
> > #define MPI_INT (MPI_Datatype)&MPI_ABI_INT<br>
> ><br>
> > Option 1 relies on implementation defined behavior regarding the<br>
> > casting of integers to pointers, but it is safe. Gonzalo and I<br>
> had a<br>
> > long discussion of this, and the only issue is that<br>
> (intptr_t)MPI_INT<br>
> > == 0x3 is not guaranteed to be true, but (intptr_t)MPI_INT ==<br>
> > (intptr_t)(void*)0x3 is. Both MPICH and Open-MPI already rely<br>
> on this<br>
> > technique for MPI_IN_PLACE, among others.<br>
> ><br>
> > The advantage of Option 1 - assuming we use a sensible set of<br>
> > integer values - is that ABI compatibility layers can do<br>
> translation<br>
> > via a table. This is most useful for datatypes, where there are<br>
> ~100<br>
> > predefined handles, and to a lesser extent ops. For all other<br>
> > handles, there are no more than 3 predefined handles (IIRC).<br>
> ><br>
> > Option 2 allows run-time resolution of predefined handles, which I<br>
> > thought was good until I implemented it in<br>
> > <a href="https://github.com/jeffhammond/mukautuva" rel="noreferrer" target="_blank">https://github.com/jeffhammond/mukautuva</a>. While it works and<br>
> means I<br>
> > can avoid predefined handle translations in the forward<br>
> direction, I<br>
> > still have it in the backwards direction, and it's not exactly<br>
> simple<br>
> > to resolve all the symbols. I think Hui also implemented this<br>
> in his<br>
> > compatibility layer, although I don't know if he hates it as<br>
> much as I do.<br>
> ><br>
> > The second question is whether to #define MPI_HANDLE_NULL<br>
> > (MPI_Handle*)0. Lisandro argues for this, and it would make a<br>
> lot of<br>
> > things easy.<br>
> ><br>
> > I hope that we can decide these two questions today. For the<br>
> second<br>
> > meeting, we will address predefined integer constants.<br>
> ><br>
> > Jeff<br>
> ><br>
> > --<br>
> > Jeff Hammond<br>
> > <a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a><br>
> > <a href="http://jeffhammond.github.io/" rel="noreferrer" target="_blank">http://jeffhammond.github.io/</a><br>
> ><br>
><br>
> -- <br>
> mpiwg-abi mailing list<br>
> <a href="mailto:mpiwg-abi@lists.mpi-forum.org" target="_blank">mpiwg-abi@lists.mpi-forum.org</a><br>
> <a href="https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi" rel="noreferrer" target="_blank">https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi</a><br>
> -- <br>
> mpiwg-abi mailing list<br>
> <a href="mailto:mpiwg-abi@lists.mpi-forum.org" target="_blank">mpiwg-abi@lists.mpi-forum.org</a><br>
> <a href="https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi" rel="noreferrer" target="_blank">https://lists.mpi-forum.org/mailman/listinfo/mpiwg-abi</a><br>
><br>
><br>
><br>
> -- <br>
> Jeff Hammond<br>
> <a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a><br>
> <a href="http://jeffhammond.github.io/" rel="noreferrer" target="_blank">http://jeffhammond.github.io/</a><br>
<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature">Jeff Hammond<br><a href="mailto:jeff.science@gmail.com" target="_blank">jeff.science@gmail.com</a><br><a href="http://jeffhammond.github.io/" target="_blank">http://jeffhammond.github.io/</a></div></div>