<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><div>Just so we are on the same page: a process set is some collection of processes that have agreed to call themselves "foo" - yes? Kind of like arbitrary named groupings.</div><div><br></div><div>In PMIx-land, we call those "groups" - which is immaterial, but I wanted to ensure anyone looking thru the standard knew the right place to find it. I'll follow your 'set" terminology here.</div><div><br></div><div>Definition of a set can be initiated from two sources:</div><div><br></div><div>* external to the application - e.g., someone using a tool to establish the definition</div><div><br></div><div>* from the application itself. PMIx supports two methods here: one that is driven by a single process (that invites others to join), and the other being a collective.</div><div><br></div><div>Looking solely at the application-initiated side, we have the possibility that a proc may augment the set membership when it joins. Essentially, the situation is where a process knows of some other procs that need to be members of the set, but aren't necessarily known to all set members just yet. When this proc joins, it has to add those procs to the set (via an attribute).</div><div><br></div><div>The bottom line is that application procs don't actually know the final set membership until they are handed it at completion of the construct operation. Thus, the operation is always a global one across the nodes where member procs are executing.</div><div><br></div><div>So the event notifying that a process set has been newly defined isn't generated until the operation completes (since the event includes the membership, which we don't know until it does complete). It is guaranteed visible to all processes within its scope (see the Standard for scope rules), not just the ones involved in the set as they really don't need it (the operation return tells them the set construct is complete). Receipt of the event is ansync - no guarantee of ordering.</div><div><br></div><div>If the app is doing the single-process driven method for constructing the set, then there are additional events involved (notifying that you have been invited, notifying the leader that you accepted/rejected the invite, etc). No real way for another process to interject itself into that handshake exchange.</div><div><br></div><div>I am well aware of Howard's scenario as I raised the symptoms of it to him over in OMPI-land a little whiule ago. I'm not sure, though, if what I'm answering really pertains to what he's trying to solve. I'll try toop around with him to see if I'm just dragging you all down a whirlpool - could well be that he hit something I don't know about :-)</div><div><br></div><div>Ralph</div><div><br></div><div><br><blockquote type="cite"><div>On Jan 5, 2023, at 6:33 PM, Martin Schulz <schulzm@in.tum.de> wrote:</div><br class="Apple-interchange-newline"><div><meta charset="UTF-8"><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" style="">Hi Ralph,</span><span style=""><o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" style=""> </span><span style=""><o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" style="">What does the definition of a new process set mean? Is this just local or then guaranteed visible across all involved processes? When is that coordination and/or synchronization happen and is that visible at any time to the user?</span><span style=""><o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" style=""> </span><span style=""><o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" style="">I guess this goes back to Howard’s original question on when an MPI implementations is required to add PMIx fences (not necessarily for a particular implementation for a portable usage of the API across any implementation).</span><span style=""><o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" style=""> </span><span style=""><o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" style="">Thanks,</span><span style=""><o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" style=""> </span><span style=""><o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" style="">Martin</span><span style=""><o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" style=""> </span><span style=""><o:p></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span><o:p> </o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span><o:p> </o:p></span></div><div><div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">--<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Prof. Dr. Martin Schulz, Chair of Computer Architecture and Parallel Systems<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Department of Informatics, TU-Munich, Boltzmannstraße 3, D-85748 Garching<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Member of the Board of Directors at the Leibniz Supercomputing Centre (LRZ)<o:p></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Email:<span class="Apple-converted-space"> </span><a href="mailto:schulzm@in.tum.de" style="color: blue; text-decoration: underline;">schulzm@in.tum.de</a><o:p></o:p></div></div></div></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span><o:p> </o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span><o:p> </o:p></span></div><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(181, 196, 223); padding: 3pt 0cm 0cm;"><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span style="font-size: 12pt;">From:<span class="Apple-converted-space"> </span></span></b><span style="font-size: 12pt;">mpiwg-sessions <<a href="mailto:mpiwg-sessions-bounces@lists.mpi-forum.org" style="color: blue; text-decoration: underline;">mpiwg-sessions-bounces@lists.mpi-forum.org</a>> on behalf of Ralph Castain via mpiwg-sessions <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" style="color: blue; text-decoration: underline;">mpiwg-sessions@lists.mpi-forum.org</a>><br><b>Reply to:<span class="Apple-converted-space"> </span></b>MPI Sessions working group <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" style="color: blue; text-decoration: underline;">mpiwg-sessions@lists.mpi-forum.org</a>><br><b>Date:<span class="Apple-converted-space"> </span></b>Thursday, 5. January 2023 at 07:09<br><b>To:<span class="Apple-converted-space"> </span></b>MPI Sessions working group <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" style="color: blue; text-decoration: underline;">mpiwg-sessions@lists.mpi-forum.org</a>><br><b>Cc:<span class="Apple-converted-space"> </span></b>Ralph Castain <<a href="mailto:rhc@pmix.org" style="color: blue; text-decoration: underline;">rhc@pmix.org</a>><br><b>Subject:<span class="Apple-converted-space"> </span></b>Re: [mpiwg-sessions] [EXTERNAL] RE: MPI_Session_init semantics question/poll<o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Just FWIW: PMIx provides an event you can register to receive whenever a new process set has been defined. We could, I imagine, add one for process set deletion if that is useful.<span class="Apple-converted-space"> </span><o:p></o:p></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">And you can always query for the complete list of current process set names (as well as memberships, if you want it).<o:p></o:p></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Ralph<o:p></o:p></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><br><br><o:p></o:p></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt;" type="cite"><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">On Jan 5, 2023, at 4:50 AM, Holmes, Daniel John via mpiwg-sessions <mpiwg-sessions@lists.mpi-forum.org> wrote:<o:p></o:p></div></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p> </o:p></div><div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Hi Martin,<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">This seems like the best reason so far for differentiating “immediate” as the term that refers to the stronger semantic.<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">immediate = prohibited to delay its return – neither until (remote) progress nor until (remote) specific semantically-related MPI procedure call<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">local = permitted to delay its return until (remote) progress but not until (remote) specific semantically-related MPI procedure call<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">nonlocal = permitted to delay its return (remote) progress and/or until (remote) specific semantically-related MPI procedure call<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div style="border-style: none none solid; border-bottom-width: 1pt; border-bottom-color: windowtext; padding: 0cm 0cm 1pt;"><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Permitting progress in an API that was intended to be “lightweight” is bad. Progress could mean that MPI_Session_init delays its return while it pushes a few GB of buffered-mode send message data into a network – correctness is not affected, but performance expectations are. The canonical example is whether MPI_Comm_rank can delay its return while it does (remote) progress – currently “yes because local, but why would anyone implement it that way? It should be immediate.” The recent debate about whether MPI_Put is nonlocal is relevant here – correctness allows us to say “it doesn’t matter if MPI_Put is nonlocal because the user cannot create a deadlock” but the performance expectations of separating synchronisation from data movement instructions suggest that MPI_Put should be immediate, not even local, but definitely not nonlocal.<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">I would be much happier with some kind of explicit “update the list of process set names” API that has whatever semantic is needed, rather than allowing (remote) progress in any of the existing immediate APIs: “initialise a session”, “get the number of process set names”, “get the nth process set name”, and “make a group from this process set name”.<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">The trouble is: what is the appropriate semantic for the “update the list of process set names” API? We already have MPI_Session_init that will give an updated list of process set names if it is already different to previous lists given by previous sessions. We only need a “delay until something changes” semantic. What needs to change to satisfy this semantic? Is the first difference sufficient? Can/should the user specify what they are looking for? Is any difference actually necessary (should it have a timeout, in case nothing changes)? Is it collective (over which group)? Those questions are currently answered by the mechanism(s) that could possibly cause change to the list of process sets – spawn is an MPI operation, connect/accept are MPI operations, and so on. Initialising another session after one of these operations has completed will already get the up-to-date list of process sets (whether it changed because of the dynamic process model operation or not). Any future proposal for an API that extends/modifies/prunes the list of process set names needs to have a semantic that permits the appropriate consensus in its implementation. The consensus is needed by operations like “give me more resources”, “take these resources back”, etc – not by query functions.<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Best wishes,<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Dan.<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0cm 0cm;"><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span lang="EN-US">From:</span></b><span class="apple-converted-space"><span lang="EN-US"> </span></span><span lang="EN-US">Martin Schulz <schulzm@in.tum.de><span class="apple-converted-space"> </span><br><b>Sent:</b><span class="apple-converted-space"> </span>04 January 2023 22:43<br><b>To:</b><span class="apple-converted-space"> </span>Holmes, Daniel John <daniel.john.holmes@intel.com>; MPI Sessions working group <mpiwg-sessions@lists.mpi-forum.org><br><b>Cc:</b><span class="apple-converted-space"> </span>Pritchard Jr., Howard <howardp@lanl.gov><br><b>Subject:</b><span class="apple-converted-space"> </span>Re: [mpiwg-sessions] [EXTERNAL] RE: MPI_Session_init semantics question/poll</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 12pt;"> <o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="DE">Hi Dan,</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="DE"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">I fully agree with you on the difference between MPI and PVM and the need to keep things constant from the user’s perspective. They should not be impacted by this and hence also the agreement that none of this can be non-local (as this will impact the user).</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">However, at some point the underlying runtime will have to come to a consensus and that may come after a user would expect it relative to a local procedure call. This is not likely or probably not even possible with the current static scheme and implementation, but it will likely happen when we add more dynamic behavior.</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">An example could be newly spawned processes and when they are ready and have established all their individual process set memberships. This could be, of course, pushed off to the user who would need to ensure the right timing, but it may also be better to give the runtime some leeway – again, not in the sense of non-local, but in the sense of weak progress.</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">This was actually my understanding of weak progress anyway – that any MPI routine could delay return for weak progress, but he hardened that only recently to operations only. For the current set of MPI routines that actually does not make a difference in execution, but here we could – by accident – add a limiter for future implementations and this is what I would like to avoid by opening up the chance for these routines to participate in progress.</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Martin</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div><div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">--<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Prof. Dr. Martin Schulz, Chair of Computer Architecture and Parallel Systems<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Department of Informatics, TU-Munich, Boltzmannstraße 3, D-85748 Garching<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Member of the Board of Directors at the Leibniz Supercomputing Centre (LRZ)<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Email:<span class="apple-converted-space"> </span><a href="mailto:schulzm@in.tum.de" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">schulzm@in.tum.de</span></a><span style="font-size: 12pt;"><o:p></o:p></span></div></div></div></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(181, 196, 223); padding: 3pt 0cm 0cm;"><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span style="font-size: 12pt;">From:<span class="apple-converted-space"> </span></span></b><span style="font-size: 12pt;">"Holmes, Daniel John" <<a href="mailto:daniel.john.holmes@intel.com" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">daniel.john.holmes@intel.com</span></a>><br><b>Date:<span class="apple-converted-space"> </span></b>Wednesday, 4. January 2023 at 10:17<br><b>To:<span class="apple-converted-space"> </span></b>Martin Schulz <<a href="mailto:schulzm@in.tum.de" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">schulzm@in.tum.de</span></a>>, MPI Sessions working group <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">mpiwg-sessions@lists.mpi-forum.org</span></a>><br><b>Cc:<span class="apple-converted-space"> </span></b>"Pritchard Jr., Howard" <<a href="mailto:howardp@lanl.gov" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">howardp@lanl.gov</span></a>><br><b>Subject:<span class="apple-converted-space"> </span></b>RE: [mpiwg-sessions] [EXTERNAL] RE: MPI_Session_init semantics question/poll<o:p></o:p></span></div></div></div><div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Hi Martin,<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">MPI is not PVM. We do not wait to see which/how many processes start and join the group/process set before deciding on the membership of the group/process set. The names and the membership of all (built-in/predefined) process sets are known a priori without coordination during the initialisation procedure call(s). Deviation from that membership (e.g. a process fails to start or fails to join up with the other processes) is a fault, which will cause a failure (e.g. a collective operation cannot complete), which will manifest as an error. The process set still exists and a group can still be formed from it; the communicator creation procedure that uses that group will raise an error.<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">For scenarios/implementations where additional process sets “appear” during the execution, those new process sets might not appear until all involved processes will see the same new set name (depending on what the implementation can support); that might mean every involved process will have to have done some progress after the process set was created internally before any process will expose it to the user via MPI calls. That delay must never happen for the built-in/predefined process sets, so we have no conflict or difficulty.<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Best wishes,<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Dan.<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0cm 0cm;"><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span lang="EN-US">From:</span></b><span class="apple-converted-space"><span lang="EN-US"> </span></span><span lang="EN-US">Martin Schulz <<a href="mailto:schulzm@in.tum.de" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">schulzm@in.tum.de</span></a>><span class="apple-converted-space"> </span><br><b>Sent:</b><span class="apple-converted-space"> </span>04 January 2023 19:43<br><b>To:</b><span class="apple-converted-space"> </span>MPI Sessions working group <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">mpiwg-sessions@lists.mpi-forum.org</span></a>>; Holmes, Daniel John <<a href="mailto:daniel.john.holmes@intel.com" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">daniel.john.holmes@intel.com</span></a>><br><b>Cc:</b><span class="apple-converted-space"> </span>Pritchard Jr., Howard <<a href="mailto:howardp@lanl.gov" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">howardp@lanl.gov</span></a>><br><b>Subject:</b><span class="apple-converted-space"> </span>Re: [mpiwg-sessions] [EXTERNAL] RE: MPI_Session_init semantics question/poll</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 12pt;"> <o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Hi all,</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">I agree with this interpretation – I always thought that was the original intent; non-local work should be able to be push off to the first communicator creation.</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">The question about it being an operation and/or a local call is interesting, though – I tend to also see it the same as Dan, but is there a scenario in implementations that may require some kind of progress in other MPI processes (e.g., to internally synchronize on process sets)? If so, would we have to classify at least some calls (perhaps only the query of the process sets) as (local) operations so we can mandate progress? Or maybe “have to” is to harsh, but it would implementations to be more efficient?</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Martin</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div><div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">--<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Prof. Dr. Martin Schulz, Chair of Computer Architecture and Parallel Systems<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Department of Informatics, TU-Munich, Boltzmannstraße 3, D-85748 Garching<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Member of the Board of Directors at the Leibniz Supercomputing Centre (LRZ)<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Email:<span class="apple-converted-space"> </span><a href="mailto:schulzm@in.tum.de" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">schulzm@in.tum.de</span></a><span style="font-size: 12pt;"><o:p></o:p></span></div></div></div></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(181, 196, 223); padding: 3pt 0cm 0cm;"><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span lang="EN-US" style="font-size: 12pt;">From:<span class="apple-converted-space"> </span></span></b><span lang="EN-US" style="font-size: 12pt;">mpiwg-sessions <<a href="mailto:mpiwg-sessions-bounces@lists.mpi-forum.org" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">mpiwg-sessions-bounces@lists.mpi-forum.org</span></a>> on behalf of "Pritchard Jr., Howard via mpiwg-sessions" <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">mpiwg-sessions@lists.mpi-forum.org</span></a>><br><b>Reply to:<span class="apple-converted-space"> </span></b>MPI Sessions working group <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">mpiwg-sessions@lists.mpi-forum.org</span></a>><br><b>Date:<span class="apple-converted-space"> </span></b>Wednesday, 4. January 2023 at 09:30<br><b>To:<span class="apple-converted-space"> </span></b>"Holmes, Daniel John" <<a href="mailto:daniel.john.holmes@intel.com" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">daniel.john.holmes@intel.com</span></a>>, MPI Sessions working group <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">mpiwg-sessions@lists.mpi-forum.org</span></a>><br><b>Cc:<span class="apple-converted-space"> </span></b>"Pritchard Jr., Howard" <<a href="mailto:howardp@lanl.gov" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">howardp@lanl.gov</span></a>><br><b>Subject:<span class="apple-converted-space"> </span></b>Re: [mpiwg-sessions] [EXTERNAL] RE: MPI_Session_init semantics question/poll</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div></div><div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">HI Dan,</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Yes that was my interpretation as well.</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">We can discuss at our next meeting 1/9/23 if there’s time.</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Howard</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(181, 196, 223); padding: 3pt 0cm 0cm;"><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span style="font-size: 12pt;">From:<span class="apple-converted-space"> </span></span></b><span style="font-size: 12pt;">"Holmes, Daniel John" <<a href="mailto:daniel.john.holmes@intel.com" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">daniel.john.holmes@intel.com</span></a>><br><b>Date:<span class="apple-converted-space"> </span></b>Wednesday, January 4, 2023 at 12:05 PM<br><b>To:<span class="apple-converted-space"> </span></b>MPI Sessions working group <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">mpiwg-sessions@lists.mpi-forum.org</span></a>><br><b>Cc:<span class="apple-converted-space"> </span></b>"Pritchard Jr., Howard" <<a href="mailto:howardp@lanl.gov" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">howardp@lanl.gov</span></a>><br><b>Subject:<span class="apple-converted-space"> </span></b>[EXTERNAL] RE: MPI_Session_init semantics question/poll<o:p></o:p></span></div></div></div><div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Hi Howard,<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">It was always intended that MPI_Session_init was a local procedure. In fact, “initialise a session” is not even an MPI operation, so it doesn’t make sense for it to be expressed via a nonlocal procedure.<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Further, it was intended that the nonlocal portion of the work done by MPI_Init that is eventually needed in the pure sessions pattern would be done during the first nonlocal procedure call in that pattern, as follows:<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">MPI_Session_init // local – PMIx fence prohibited<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">MPI_Group_from_pset // local – PMIx fence prohibited<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">MPI_Comm_create_from_group // nonlocal – PMIx fence permitted, if needed<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">The nonlocal work should be unnecessary until the first nonlocal procedure call, so this should all work out fine (modulo some refactoring/debugging).<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Best wishes,<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;">Dan.<span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"> <span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0cm 0cm;"><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span lang="EN-US">From:</span></b><span class="apple-converted-space"><span lang="EN-US"> </span></span><span lang="EN-US">mpiwg-sessions <<a href="mailto:mpiwg-sessions-bounces@lists.mpi-forum.org" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">mpiwg-sessions-bounces@lists.mpi-forum.org</span></a>><span class="apple-converted-space"> </span><b>On Behalf Of<span class="apple-converted-space"> </span></b>Pritchard Jr., Howard via mpiwg-sessions<br><b>Sent:</b><span class="apple-converted-space"> </span>04 January 2023 18:32<br><b>To:</b><span class="apple-converted-space"> </span>MPI Sessions working group <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">mpiwg-sessions@lists.mpi-forum.org</span></a>><br><b>Cc:</b><span class="apple-converted-space"> </span>Pritchard Jr., Howard <<a href="mailto:howardp@lanl.gov" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">howardp@lanl.gov</span></a>><br><b>Subject:</b><span class="apple-converted-space"> </span>[mpiwg-sessions] MPI_Session_init semantics question/poll</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 12pt;"> <o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Hi All,</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">First, Happy New Year!</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">I’ve got a question about the semantics of MPI_Session_init. In particular, I’d be interested in knowing people’s opinion on whether this function is nonlocal or local.</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">We don’t have any text in the current version of the standard that states whether or not MPI_Session_init is a nonlocal operation.</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">I’m considering options for handling this issue: <span class="apple-converted-space"> </span><a href="https://urldefense.com/v3/__https:/github.com/open-mpi/ompi/issues/11166__;!!Bt8fGhp8LhKGRg!CKPfJnVxgJ8KyXfu93oiW-q0IPGmpAtrBZo2vO6bAElAdqtSv6Xv6G48O6Hk2sxr3csENDhZPwUW0mA8_fi98l7TQUw$" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">https://github.com/open-mpi/ompi/issues/11166</span></a><span class="apple-converted-space"> </span>. It turns out that the way to properly resolve this issue depends on whether or not MPI_Session_init has local or nonlocal semantics.</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">I had been working under the assumption that we had intended session initialization to be a local function, but considering how to resolve issue 11166 made me begin to question this assumption.</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Thanks for any ideas,</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US">Howard</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(11, 26, 141);"><br>—</span><span style="font-size: 12pt;"><o:p></o:p></span></div></div></div><div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(11, 26, 141);"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="375" style="width: 281.25pt; border-collapse: collapse;"><tbody><tr style="height: 104.45pt;"><td width="78" valign="top" style="width: 58.4pt; padding: 0cm 5.4pt; height: 104.45pt;"><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 9pt; font-family: Arial, sans-serif;"><image001.png></span><span style="font-size: 12pt;"><o:p></o:p></span></div></div></td><td width="297" valign="top" style="width: 223.1pt; padding: 0cm 5.4pt; height: 104.45pt;"><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span style="font-size: 12pt; font-family: Arial, sans-serif; color: rgb(11, 26, 140);">Howard Pritchard</span></b><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span style="font-size: 10pt; font-family: Arial, sans-serif; color: rgb(84, 89, 97);">Research Scientist</span></b><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span style="font-size: 10pt; font-family: Arial, sans-serif; color: rgb(84, 89, 97);">HPC-ENV</span></b><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 9pt; font-family: Arial, sans-serif;"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span style="font-size: 9pt; font-family: Arial, sans-serif; color: rgb(11, 26, 140);">Los Alamos National Laboratory</span></b><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span style="font-size: 9pt; font-family: Arial, sans-serif; color: rgb(11, 26, 140);"><a href="mailto:howardp@lanl.gov" style="color: blue; text-decoration: underline;"><span style="color: rgb(5, 99, 193);">howardp@lanl.gov</span></a></span></b><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><b><span style="font-size: 9pt; font-family: Arial, sans-serif; color: rgb(11, 26, 140);"> </span></b><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 12pt;"><a href="https://urldefense.com/v3/__https:/www.instagram.com/losalamosnatlab/__;!!Bt8fGhp8LhKGRg!CKPfJnVxgJ8KyXfu93oiW-q0IPGmpAtrBZo2vO6bAElAdqtSv6Xv6G48O6Hk2sxr3csENDhZPwUW0mA8_fi9Rgwox5A$" style="color: blue; text-decoration: underline;"><b><span style="font-size: 9pt; font-family: Arial, sans-serif; color: rgb(11, 26, 140); text-decoration: none;"><image002.png></span></b></a><a href="https://urldefense.com/v3/__https:/twitter.com/LosAlamosNatLab__;!!Bt8fGhp8LhKGRg!CKPfJnVxgJ8KyXfu93oiW-q0IPGmpAtrBZo2vO6bAElAdqtSv6Xv6G48O6Hk2sxr3csENDhZPwUW0mA8_fi9vR2-KGc$" style="color: blue; text-decoration: underline;"><b><span style="font-size: 9pt; font-family: Arial, sans-serif; color: rgb(11, 26, 140); text-decoration: none;"><image003.png></span></b></a><a href="https://urldefense.com/v3/__https:/www.linkedin.com/company/los-alamos-national-laboratory/__;!!Bt8fGhp8LhKGRg!CKPfJnVxgJ8KyXfu93oiW-q0IPGmpAtrBZo2vO6bAElAdqtSv6Xv6G48O6Hk2sxr3csENDhZPwUW0mA8_fi9_F2cjUc$" style="color: blue; text-decoration: underline;"><b><span style="font-size: 9pt; font-family: Arial, sans-serif; color: rgb(11, 26, 140); text-decoration: none;"><image004.png></span></b></a><a href="https://urldefense.com/v3/__https:/www.facebook.com/LosAlamosNationalLab/__;!!Bt8fGhp8LhKGRg!CKPfJnVxgJ8KyXfu93oiW-q0IPGmpAtrBZo2vO6bAElAdqtSv6Xv6G48O6Hk2sxr3csENDhZPwUW0mA8_fi95RavtTU$" style="color: blue; text-decoration: underline;"><b><span style="font-size: 9pt; font-family: Arial, sans-serif; color: rgb(11, 26, 140); text-decoration: none;"><image005.png></span></b></a><o:p></o:p></span></div></div></td></tr></tbody></table><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" style="font-family: Arial, sans-serif;"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div></div></div><div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span lang="EN-US" style="font-size: 12pt;"> </span><span style="font-size: 12pt;"><o:p></o:p></span></div></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="font-size: 9pt; font-family: Helvetica;">_______________________________________________<br>mpiwg-sessions mailing list<br>mpiwg-sessions@lists.mpi-forum.org<br>https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions</span></div></div></blockquote></div></div></div></div></blockquote></div><br></body></html>