<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Hi Tony,
<div class=""><br class="">
</div>
<div class="">I think the result of your weaker semantics is PVM or some other type of formalised chaos :)<br class="">
<div class=""><br class="webkit-block-placeholder">
</div>
<div class="">Handling group-members failing to take part could be useful for the process-fail-stop type of fault tolerance but how does a partial group reach consensus that the other members are never going to join in, rather than just assuming they are being
a bit slow? There are ways - but let us first define what keep-alives/timeouts/RAS looks like in MPI and then look at the implications of that on the whole interface. The nearest MPI semantic is soft spawn, I think. We could look at what might be permitted
if the user supplied an info key “soft = true” to any communicator/window/file creation routine. This is, perhaps, a topic for the FT WG.</div>
<div class=""><br class="">
</div>
<div class="">
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
How many non-members are permitted to join in ad-hoc with no warning? When is the communicator creation finished, if ever? Can new members apply to join even after some (sub)group has reached a consensus and begun using the communicator for communication? The
nearest MPI semantic is connect/accept, I think, but that permits exactly one connection between (the root processes of) two arbitrarily sized pre-existing groups where all members of each group have already reached consensus with all other members of their
group. By induction, that relies on pairwise connections - fixed size of 2 (i.e. exactly one non-member), FCFS, block indefinitely if unmatched.</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<br class="">
</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
There is an event/exception mechanism in the PMIx functionality that we are currently using for the prototype implementation - if PMIx discovers actual process-failure then it can callback into all other processes to tell them about it and they can all react
in one of several appropriate ways (fail the operation, form a smaller group, invite spare processes to act as replacements, request more resources to act as replacements). Some of those reactions might lead to resilience rather than fault tolerance but it
is all hidden behind (or forbidden by) the stronger semantics of MPI, which assumes reliability and a priori coordination (information supplied at X is always consistent with information supplied at Y). Again, this is probably a topic for the FT WG.</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<br class="">
</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
A question that has always bothered me about intercomms - why only two groups? Why not permit a topology of leaders, each of which coordinates a local group, possibly with its own local topology? Does anyone have a compelling use-case for such generality?</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<br class="">
</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Summary: these are interesting research questions but not Sessions WG topics.</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<br class="">
</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Cheers,</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Dan.</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
—<br class="">
Dr Daniel Holmes PhD<br class="">
Applications Consultant in HPC Research<br class="">
<a href="mailto:d.holmes@epcc.ed.ac.uk" class="">d.holmes@epcc.ed.ac.uk</a><br class="">
Phone: +44 (0) 131 651 3465<br class="">
Mobile: +44 (0) 7940 524 088<br class="">
Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
—</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
—</div>
</div>
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On 9 Jan 2019, at 18:14, Pritchard Jr.,<span class="Apple-tab-span" style="white-space:pre">
</span>Howard via mpiwg-sessions <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" class="">mpiwg-sessions@lists.mpi-forum.org</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="font-family: Calibri, sans-serif; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<div class="">
<div class="">
<div class="">
<div class=""><br class="Apple-interchange-newline">
-- </div>
<div class="">Howard Pritchard</div>
</div>
<div class="">B Schedule</div>
<div class="">HPC-ENV</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt;" class="">Office 9, 2<sup class="">nd</sup> floor Research Park<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt;" class="">TA-03, Building 4200, Room 203</div>
</div>
<div class="">Los Alamos National Laboratory</div>
<div class=""><br class="">
</div>
</div>
</div>
</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br class="">
</div>
<span id="OLK_SRC_BODY_SECTION" style="font-family: Calibri, sans-serif; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<div style="font-family: Calibri; font-size: 11pt; text-align: left; border-width: 1pt medium medium; border-style: solid none none; padding: 3pt 0in 0in; border-top-color: rgb(181, 196, 223);" class="">
<span style="font-weight: bold;" class="">From:<span class="Apple-converted-space"> </span></span>"Tony, Skelljum" <<a href="mailto:Tony-Skjellum@utc.edu" class="">Tony-Skjellum@utc.edu</a>><br class="">
<span style="font-weight: bold;" class="">Date:<span class="Apple-converted-space"> </span></span>Wednesday, January 9, 2019 at 10:47 AM<br class="">
<span style="font-weight: bold;" class="">To:<span class="Apple-converted-space"> </span></span>Howard Pritchard <<a href="mailto:howardp@lanl.gov" class="">howardp@lanl.gov</a>><br class="">
<span style="font-weight: bold;" class="">Cc:<span class="Apple-converted-space"> </span></span>MPI Sessions working group <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" class="">mpiwg-sessions@lists.mpi-forum.org</a>><br class="">
<span style="font-weight: bold;" class="">Subject:<span class="Apple-converted-space"> </span></span>Re: MPI_Intercomm_from_group folow-up<br class="">
</div>
<div class=""><br class="">
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="border-left-color: rgb(181, 196, 223); border-left-width: 5px; border-left-style: solid; padding: 0px 0px 0px 5px; margin: 0px 0px 0px 5px;" class="" type="cite">
<div class="">
<div dir="ltr" class="">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; font-family: Calibri, Helvetica, sans-serif;" class="">
<div style="margin-top: 0px; margin-bottom: 0px;" class="">Howard, OK</div>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><br class="">
</div>
<div style="margin-top: 0px; margin-bottom: 0px;" class="">1) What happens when non-group members call the intercomm function</div>
<div style="margin-top: 0px; margin-bottom: 0px;" class="">2) What happens when not-all group members call the intercomm function</div>
<div style="margin-top: 0px; margin-bottom: 0px;" class="">--> Dan convinced me that this is disallowed by the semantics as written</div>
<div style="margin-top: 0px; margin-bottom: 0px;" class="">--> I want to explore weakening those semantics to see where that leads.</div>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><br class="">
</div>
<div style="margin-top: 0px; margin-bottom: 0px;" class="">My further thought is we should explore</div>
<p style="margin-top: 0px; margin-bottom: 0px;" class=""></p>
<ul style="margin-bottom: 0px; margin-top: 0px;" class="">
<li class=""><span style="font-size: 12pt;" class=""></span>Arms-length versions of these functions where it is intentionally allowed for non-members to call functions and not all members to call functions--for FT reasons, maybe in cases where we just need
pt2pt communication</li><li class="">When we introduce topology to groups vs. comms, intercomms with topology of groups on each side also pose an interesting new graph-to-graph connectivity idea that we might exploit for scalability and maybe for FT</li></ul>
<br class="">
<p style="margin-top: 0px; margin-bottom: 0px;" class=""></p>
<div style="margin-top: 0px; margin-bottom: 0px;" class="">Thanks,<br class="">
Tony</div>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><br class="">
</div>
<div id="Signature" class="">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; font-family: Calibri, Helvetica, sans-serif, Helvetica, EmojiFont, 'Apple Color Emoji', 'Segoe UI Emoji', NotoColorEmoji, 'Segoe UI Symbol', 'Android Emoji', EmojiSymbols;" class="">
<div style="margin-top: 0px; margin-bottom: 0px;" class="">Anthony Skjellum, PhD</div>
<div style="margin-top: 0px; margin-bottom: 0px;" class="">Professor of Computer Science and Chair of Excellence</div>
<div style="margin-top: 0px; margin-bottom: 0px;" class="">Director, SimCenter</div>
<div style="margin-top: 0px; margin-bottom: 0px;" class="">University of Tennessee at Chattanooga (UTC)</div>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><a href="mailto:tony-skjellum@utc.edu" class="">tony-skjellum@utc.edu</a><span class="Apple-converted-space"> </span> [or<span class="Apple-converted-space"> </span><a href="mailto:skjellum@gmail.com" class="">skjellum@gmail.com</a>]</div>
<div style="margin-top: 0px; margin-bottom: 0px;" class="">cell: 205-807-4968</div>
<div style="margin-top: 0px; margin-bottom: 0px;" class=""><br class="">
</div>
</div>
</div>
</div>
<hr tabindex="-1" style="display: inline-block; width: 634.046875px;" class="">
<div id="divRplyFwdMsg" dir="ltr" class=""><font face="Calibri,sans-serif" style="font-size: 11pt;" class=""><b class="">From:</b><span class="Apple-converted-space"> </span>Pritchard Jr., Howard <<a href="mailto:howardp@lanl.gov" class="">howardp@lanl.gov</a>><br class="">
<b class="">Sent:</b><span class="Apple-converted-space"> </span>Wednesday, January 9, 2019 12:21:03 PM<br class="">
<b class="">To:</b><span class="Apple-converted-space"> </span>Skjellum, Anthony<br class="">
<b class="">Cc:</b><span class="Apple-converted-space"> </span>MPI Sessions working group<br class="">
<b class="">Subject:</b><span class="Apple-converted-space"> </span>MPI_Intercomm_from_group folow-up</font>
<div class=""> </div>
</div>
<div style="word-wrap: break-word; font-size: 14px; font-family: Calibri, sans-serif;" class="">
<div class="">
<div class="">
<div class="">HI Tony,</div>
<div class=""><br class="">
</div>
<div class="">The audio was really bad a few minutes ago and I couldn’t understand what you were saying.</div>
<div class="">Could you summarize on the WG list what you were suggesting?</div>
<div class=""><br class="">
</div>
<div class="">Thanks,</div>
<div class=""><br class="">
</div>
<div class="">Howard</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">
<div class="">-- </div>
<div class="">Howard Pritchard</div>
</div>
<div class="">B Schedule</div>
<div class="">HPC-ENV</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt;" class="">Office 9, 2<sup class="">nd</sup> floor Research Park</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt;" class="">TA-03, Building 4200, Room 203</div>
</div>
<div class="">Los Alamos National Laboratory</div>
<div class=""><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</span><span style="font-family: Calibri, sans-serif; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""></span><span style="font-family: Calibri, sans-serif; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Calibri, sans-serif; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Calibri, sans-serif; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">mpiwg-sessions
mailing list</span><br style="font-family: Calibri, sans-serif; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" style="font-family: Calibri, sans-serif; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">mpiwg-sessions@lists.mpi-forum.org</a><br style="font-family: Calibri, sans-serif; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<a href="https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions" style="font-family: Calibri, sans-serif; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions</a><br style="font-family: Calibri, sans-serif; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>