<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hi Dan,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I think this is a good idea. I’m not sure we entirely missed it though for sessions 1.0. I vaguely recall discussions about something like this in the context of mpi_exec – but it was quite a while ago. Maybe some problems/limitations
were found with this process set approach to capturing info to support spawn functionality. We should probably double check the old slide deck.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">This would be a 4.1 proposal right?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Howard<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-left:.5in"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">HOLMES Daniel <d.holmes@epcc.ed.ac.uk><br>
<b>Date: </b>Friday, January 22, 2021 at 5:06 AM<br>
<b>To: </b>"Pritchard Jr., Howard" <howardp@lanl.gov>, Martin Schulz <schulzm@in.tum.de><br>
<b>Cc: </b>MPI Sessions working group <mpiwg-sessions@lists.mpi-forum.org><br>
<b>Subject: </b>[EXTERNAL] Better handling of dynamic mixed with sessions without world<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<p class="MsoNormal" style="margin-left:.5in">Hi all, <o:p></o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">I just reviewed PR 506 from Rolf. It adds predefined communicator names to Appendix A. One of those is “MPI_COMM_PARENT”. It is a correct change and only indirectly relevant because it is what set off an interesting
train of thought, which I summarise below.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">This suggests that we missed a trick when proposing Sessions v1.0 - we should have mandated the existence and meaning for an additional process set name: "<a href="mpi://MPI_COMM_PARENT">mpi://MPI_COMM_PARENT</a>”
(note the absence of “GET” in that name to match the default string name assigned by MPI to the communicator returned by the MPI_COMM_GET_PARENT procedure).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">This process set shall contain all the processes that would be in the communicator returned by MPI_COMM_GET_PARENT, either zero processes (if you would get MPI_COMM_NULL) or the union of the local and remote groups
(if you would get an inter-communicator).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">With this addition, MPI processes that are spawned can regain all of the functionality of the Dynamic Model without calling MPI_INIT[_THREAD], i.e. they can create the inter-communicator that they would have got
from MPI_COMM_GET_PARENT (which requires a prior call to MPI_INIT because World Model [ED: please check]).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">```pseudo-code<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Create a session<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Create an MPI_Group, groupParent, from the process set named <span style="color:black">"</span><a href="mpi://MPI_COMM_PARENT">mpi://MPI_COMM_PARENT</a><span style="color:black">”</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in">Create an MPI_Group, groupWorld, from the process set named <span style="color:black">“<a href="mpi://MPI_COMM_WORLD">mpi://MPI_COMM_WORLD</a>”</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="color:black">If (groupParent if MPI_GROUP_NULL) then</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="color:black"> This process was not spawned by other MPI processes</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="color:black">Else</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="color:black"> Create groupLocal from the intersection of groupWorld and groupParent</span><o:p></o:p></p>
</div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black"> Create groupRemote from the subtraction of groupWorld from groupParent<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black"> Create commParent using MPI_COMM_CREATE_FROM_GROUPS with groupLocal and groupRemote<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black">```<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black">No usage of the World Model, but we now have (a duplicate of) the inter-communicator that would have been returned by MPI_COMM_GET_PARENT if we had
been permitted to call it.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="color:black">For each MPI process, there is a unique “parent” process set by this definition. Multiple components doing this pseudo-code would get duplicate communicators with their own life-cycle.
This contrasts with the current way whereby freeing or disconnecting any communicator handle provided by MPI_COMM_GET_PARENT makes all other such handles stale, which is another instance of global state in MPI baked into the standardised interface and another anathema
to any attempt at isolation.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black">Of course, there is already no reason why the parent processes could not have used the Sessions Model to call MPI_COMM_SPAWN (passing in a `comm` that
was derived from the Sessions Model), so this completes the picture of how the Dynamic Model can be bolted on successfully to either the World Model (refer to MPI-3.1 and prior) or the Sessions Model (refer to MPI-4.1, which will include this addition) or
a mixture (refer to MPI-4.0, without this addition).<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black">[EDIT: if we decide that MPI_COMM_GET_PARENT can already be called without/before/after the World Model, then we should add it to Table 11.1 and write
some text about it in §11.8.2. In that scenario, the above provides an implementation route for that procedure using only the Sessions Model underneath - in a similar way to the observation that it is possible to implement MPI_INIT[_THREAD] using only the
Sessions Model underneath.]<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black">Cheers,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black">Dan.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black">—<br>
Dr Daniel Holmes PhD<br>
Architect (HPC Research)<br>
<a href="mailto:d.holmes@epcc.ed.ac.uk">d.holmes@epcc.ed.ac.uk</a><br>
Phone: +44 (0) 131 651 3465<br>
Mobile: +44 (0) 7940 524 088<br>
Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black">—<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black">The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:9.0pt;font-family:Helvetica;color:black">—<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p>
</div>
</div>
</body>
</html>