<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:12.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Hi Martin,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">MPI is not PVM. We do not wait to see which/how many processes start and join the group/process set before deciding on the membership of the group/process set. The names and the
membership of all (built-in/predefined) process sets are known a priori without coordination during the initialisation procedure call(s). Deviation from that membership (e.g. a process fails to start or fails to join up with the other processes) is a fault,
which will cause a failure (e.g. a collective operation cannot complete), which will manifest as an error. The process set still exists and a group can still be formed from it; the communicator creation procedure that uses that group will raise an error.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">For scenarios/implementations where additional process sets “appear” during the execution, those new process sets might not appear until all involved processes will see the same
new set name (depending on what the implementation can support); that might mean every involved process will have to have done some progress after the process set was created internally before any process will expose it to the user via MPI calls. That delay
must never happen for the built-in/predefined process sets, so we have no conflict or difficulty.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Best wishes,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Dan.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt">From:</span></b><span lang="EN-US" style="font-size:11.0pt"> Martin Schulz <schulzm@in.tum.de>
<br>
<b>Sent:</b> 04 January 2023 19:43<br>
<b>To:</b> MPI Sessions working group <mpiwg-sessions@lists.mpi-forum.org>; Holmes, Daniel John <daniel.john.holmes@intel.com><br>
<b>Cc:</b> Pritchard Jr., Howard <howardp@lanl.gov><br>
<b>Subject:</b> Re: [mpiwg-sessions] [EXTERNAL] RE: MPI_Session_init semantics question/poll<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US">Hi all,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US">I agree with this interpretation – I always thought that was the original intent; non-local work should be able to be push off to the first communicator creation.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US">The question about it being an operation and/or a local call is interesting, though – I tend to also see it the same as Dan, but is there a scenario in implementations
that may require some kind of progress in other MPI processes (e.g., to internally synchronize on process sets)? If so, would we have to classify at least some calls (perhaps only the query of the process sets) as (local) operations so we can mandate progress?
Or maybe “have to” is to harsh, but it would implementations to be more efficient?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US">Martin<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">--<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Prof. Dr. Martin Schulz, Chair of Computer Architecture and Parallel Systems<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Department of Informatics, TU-Munich, Boltzmannstraße 3, D-85748 Garching<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Member of the Board of Directors at the Leibniz Supercomputing Centre (LRZ)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Email: <a href="mailto:schulzm@in.tum.de">
schulzm@in.tum.de</a><o:p></o:p></span></p>
</div>
</div>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="color:black">From: </span></b><span lang="EN-US" style="color:black">mpiwg-sessions <<a href="mailto:mpiwg-sessions-bounces@lists.mpi-forum.org">mpiwg-sessions-bounces@lists.mpi-forum.org</a>> on behalf of "Pritchard
Jr., Howard via mpiwg-sessions" <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org">mpiwg-sessions@lists.mpi-forum.org</a>><br>
<b>Reply to: </b>MPI Sessions working group <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org">mpiwg-sessions@lists.mpi-forum.org</a>><br>
<b>Date: </b>Wednesday, 4. January 2023 at 09:30<br>
<b>To: </b>"Holmes, Daniel John" <<a href="mailto:daniel.john.holmes@intel.com">daniel.john.holmes@intel.com</a>>, MPI Sessions working group <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org">mpiwg-sessions@lists.mpi-forum.org</a>><br>
<b>Cc: </b>"Pritchard Jr., Howard" <<a href="mailto:howardp@lanl.gov">howardp@lanl.gov</a>><br>
<b>Subject: </b>Re: [mpiwg-sessions] [EXTERNAL] RE: MPI_Session_init semantics question/poll<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">HI Dan,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">Yes that was my interpretation as well.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">We can discuss at our next meeting 1/9/23 if there’s time.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">Howard<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="color:black">From: </span></b><span style="color:black">"Holmes, Daniel John" <<a href="mailto:daniel.john.holmes@intel.com">daniel.john.holmes@intel.com</a>><br>
<b>Date: </b>Wednesday, January 4, 2023 at 12:05 PM<br>
<b>To: </b>MPI Sessions working group <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org">mpiwg-sessions@lists.mpi-forum.org</a>><br>
<b>Cc: </b>"Pritchard Jr., Howard" <<a href="mailto:howardp@lanl.gov">howardp@lanl.gov</a>><br>
<b>Subject: </b>[EXTERNAL] RE: MPI_Session_init semantics question/poll<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt">Hi Howard,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">It was always intended that MPI_Session_init was a local procedure. In fact, “initialise a session” is not even an MPI operation, so it doesn’t make sense for it to be expressed via a nonlocal procedure.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Further, it was intended that the nonlocal portion of the work done by MPI_Init that is eventually needed in the pure sessions pattern would be done during the first nonlocal procedure call in that pattern,
as follows:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">MPI_Session_init // local – PMIx fence prohibited<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">MPI_Group_from_pset // local – PMIx fence prohibited<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">MPI_Comm_create_from_group // nonlocal – PMIx fence permitted, if needed<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">The nonlocal work should be unnecessary until the first nonlocal procedure call, so this should all work out fine (modulo some refactoring/debugging).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Best wishes,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Dan.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt">From:</span></b><span lang="EN-US" style="font-size:11.0pt"> mpiwg-sessions <<a href="mailto:mpiwg-sessions-bounces@lists.mpi-forum.org">mpiwg-sessions-bounces@lists.mpi-forum.org</a>>
<b>On Behalf Of </b>Pritchard Jr., Howard via mpiwg-sessions<br>
<b>Sent:</b> 04 January 2023 18:32<br>
<b>To:</b> MPI Sessions working group <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org">mpiwg-sessions@lists.mpi-forum.org</a>><br>
<b>Cc:</b> Pritchard Jr., Howard <<a href="mailto:howardp@lanl.gov">howardp@lanl.gov</a>><br>
<b>Subject:</b> [mpiwg-sessions] MPI_Session_init semantics question/poll<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">Hi All,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">First, Happy New Year!<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">I’ve got a question about the semantics of MPI_Session_init. In particular, I’d be interested in knowing people’s opinion on whether this function is nonlocal or local.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">We don’t have any text in the current version of the standard that states whether or not MPI_Session_init is a nonlocal operation.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">I’m considering options for handling this issue:
<a href="https://urldefense.com/v3/__https:/github.com/open-mpi/ompi/issues/11166__;!!Bt8fGhp8LhKGRg!CKPfJnVxgJ8KyXfu93oiW-q0IPGmpAtrBZo2vO6bAElAdqtSv6Xv6G48O6Hk2sxr3csENDhZPwUW0mA8_fi98l7TQUw$">
https://github.com/open-mpi/ompi/issues/11166</a> . It turns out that the way to properly resolve this issue depends on whether or not MPI_Session_init has local or nonlocal semantics.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">I had been working under the assumption that we had intended session initialization to be a local function, but considering how to resolve issue 11166 made me begin to question this assumption.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">Thanks for any ideas,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt">Howard<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt"><o:p> </o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Arial",sans-serif;color:#0B1A8D"><br>
—</span><span lang="EN-US" style="font-size:11.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Arial",sans-serif;color:#0B1A8D"><o:p> </o:p></span></p>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="375" style="width:281.25pt;border-collapse:collapse">
<tbody>
<tr style="height:104.45pt">
<td width="78" valign="top" style="width:58.4pt;padding:0cm 5.4pt 0cm 5.4pt;height:104.45pt">
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Arial",sans-serif"><img border="0" width="55" height="55" style="width:.5729in;height:.5729in" id="Picture_x0020_5" src="cid:image001.png@01D92077.DD059D30" alt="signature_61897647"></span><span style="font-size:9.0pt;font-family:"Arial",sans-serif"><o:p></o:p></span></p>
</td>
<td width="297" valign="top" style="width:223.1pt;padding:0cm 5.4pt 0cm 5.4pt;height:104.45pt">
<p class="MsoNormal"><b><span style="font-family:"Arial",sans-serif;color:#0B1A8C">Howard Pritchard<o:p></o:p></span></b></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#545961">Research Scientist<o:p></o:p></span></b></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#545961">HPC-ENV<o:p></o:p></span></b></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#0B1A8C">Los Alamos National Laboratory<o:p></o:p></span></b></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#0B1A8C"><a href="mailto:howardp@lanl.gov">howardp@lanl.gov</a><o:p></o:p></span></b></p>
<p class="MsoNormal"><b><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#0B1A8C"><o:p> </o:p></span></b></p>
<p class="MsoNormal"><a href="https://urldefense.com/v3/__https:/www.instagram.com/losalamosnatlab/__;!!Bt8fGhp8LhKGRg!CKPfJnVxgJ8KyXfu93oiW-q0IPGmpAtrBZo2vO6bAElAdqtSv6Xv6G48O6Hk2sxr3csENDhZPwUW0mA8_fi9Rgwox5A$"><b><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#0B1A8C;text-decoration:none"><img border="0" width="27" height="27" style="width:.2812in;height:.2812in" id="Picture_x0020_4" src="cid:image002.png@01D92077.DD059D30" alt="signature_1293224934"></span></b></a><a href="https://urldefense.com/v3/__https:/twitter.com/LosAlamosNatLab__;!!Bt8fGhp8LhKGRg!CKPfJnVxgJ8KyXfu93oiW-q0IPGmpAtrBZo2vO6bAElAdqtSv6Xv6G48O6Hk2sxr3csENDhZPwUW0mA8_fi9vR2-KGc$"><b><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#0B1A8C;text-decoration:none"><img border="0" width="27" height="27" style="width:.2812in;height:.2812in" id="Picture_x0020_3" src="cid:image003.png@01D92077.DD059D30" alt="signature_2498822630"></span></b></a><a href="https://urldefense.com/v3/__https:/www.linkedin.com/company/los-alamos-national-laboratory/__;!!Bt8fGhp8LhKGRg!CKPfJnVxgJ8KyXfu93oiW-q0IPGmpAtrBZo2vO6bAElAdqtSv6Xv6G48O6Hk2sxr3csENDhZPwUW0mA8_fi9_F2cjUc$"><b><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#0B1A8C;text-decoration:none"><img border="0" width="27" height="27" style="width:.2812in;height:.2812in" id="Picture_x0020_2" src="cid:image004.png@01D92077.DD059D30" alt="signature_1283032776"></span></b></a><a href="https://urldefense.com/v3/__https:/www.facebook.com/LosAlamosNationalLab/__;!!Bt8fGhp8LhKGRg!CKPfJnVxgJ8KyXfu93oiW-q0IPGmpAtrBZo2vO6bAElAdqtSv6Xv6G48O6Hk2sxr3csENDhZPwUW0mA8_fi95RavtTU$"><b><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#0B1A8C;text-decoration:none"><img border="0" width="27" height="27" style="width:.2812in;height:.2812in" id="Picture_x0020_1" src="cid:image005.png@01D92077.DD059D30" alt="signature_3959178607"></span></b></a><b><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#0B1A8C"><o:p></o:p></span></b></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
</body>
</html>