<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hello folks<div class=""><br class=""></div><div class="">I had a chance over the holidays to catch up on your docs regarding dynamic sessions - quite interesting. I believe there is support in PMIx for pretty much everything I saw being discussed. We have APIs by which an application can directly request allocation changes from the RM, and events by which the RM can notify (and subsequently negotiate) an application regarding changes to its allocation. So each side has the ability to initiate the process, and then both sides negotiate to a common conclusion. We also have an API by which an application can "register" its willingness to accept RM-initiated "preemption" requests so the RM can incorporate that willingness in its pricing and planning procedures.</div><div class=""><br class=""></div><div class="">Unfortunately, while we have that infrastructure defined in the PMIx Standard and implemented in OpenPMIx, we have not yet seen the required backend support implemented in an RM. I have started working with some folks on integrating support into Slurm, but I do not know the timetable for public release of that work. SchedMD has been ambivalent towards accepting pull requests that extend its PMIx support, so this may well have to be released as a side-project.</div><div class=""><br class=""></div><div class="">I have previously approached Altair about adding support to PBS - nothing has happened yet. I suspect they are waiting for customer demand. I have no knowledge of any other RMs looking into it. As a gap-filling measure, I am adding simulated support in PRRTE so that anyone wanting to develop dynamic resource code can at least have a place where they can develop it and do a little testing. PRRTE doesn't include a scheduler, but I can simulate it by retaining some of the RM-allocated resources as part of a PRRTE-managed "pool".</div><div class=""><br class=""></div><div class="">Meantime, I have started a little personal project to add PMIx support to Kubernetes, hopefully giving it more capability to support HPC applications. The Kubeflow community has a degree of PMIx support, but I want to directly integrate it to Kubernetes itself, including the dynamic resource elements described above. I have no timetable for completing that work - as many of you may know, I am retired and so this is something to do in my spare time. If anyone is interested on tracking progress on this, please let me know.</div><div class=""><br class=""></div><div class="">Thus, I would encourage you to start prodding your favorite RM vendors as this may prove the critical timeline in making dynamic sessions a reality!</div><div class=""><br class=""></div><div class="">Also, if you identify any "gaps" in the PMIx support, please do let me know - I'd be happy to work with you to fill them. The current definitions were developed primarily to support workflow operations and the needs of the dynamic programming model communities (e.g., TensorFlow and Data Analytics). I think those are very similar to what you are identifying, but may perhaps need some tweaking.</div><div class=""><br class=""></div><div class="">Ralph</div><div class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Jan 3, 2022, at 9:28 AM, Pritchard Jr., Howard via mpiwg-sessions <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" class="">mpiwg-sessions@lists.mpi-forum.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta charset="UTF-8" class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">Hello All,<o:p class=""></o:p></span></div><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">Happy New Year!<o:p class=""></o:p></span></div><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">Let’s try to meet today. Items on the agenda:<o:p class=""></o:p></span></div><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><ul type="disc" style="margin-bottom: 0in; margin-top: 0in;" class=""><li class="MsoListParagraph" style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;"><span style="font-size: 13.5pt; font-family: -webkit-standard;" class="">PR #629 (issue #511) -<span class="Apple-converted-space"> </span><a href="https://github.com/mpi-forum/mpi-issues/issues/511" style="color: rgb(5, 99, 193); text-decoration: underline;" class="">https://github.com/mpi-forum/mpi-issues/issues/511</a></span><span style="font-size: 11pt;" class=""><o:p class=""></o:p></span></li></ul><div style="margin: 0in 0in 0in 0.25in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><ul type="disc" style="margin-bottom: 0in; margin-top: 0in;" class=""><li class="MsoListParagraph" style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;"><span style="font-size: 13.5pt; font-family: -webkit-standard;" class="">Pick up where we were on discussion of dynamic sessions requirements, see:</span><span style="font-size: 11pt;" class=""><o:p class=""></o:p></span><ul type="circle" style="margin-bottom: 0in; margin-top: 0in;" class=""><li class="MsoListParagraph" style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;"><span style="font-size: 11pt;" class=""><a href="https://miro.com/app/board/o9J_l_Rxe9Q=/" style="color: rgb(5, 99, 193); text-decoration: underline;" class="">https://miro.com/app/board/o9J_l_Rxe9Q=/</a><o:p class=""></o:p></span></li><li class="MsoListParagraph" style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;"><span style="font-size: 11pt;" class=""><a href="https://docs.google.com/document/d/1l7LQ8eeVOUW69TDVG9LjKJUuerfE3S3teaMFG5DOudM/edit#heading=h.voobxhw94rt3" style="color: rgb(5, 99, 193); text-decoration: underline;" class="">https://docs.google.com/document/d/1l7LQ8eeVOUW69TDVG9LjKJUuerfE3S3teaMFG5DOudM/edit#heading=h.voobxhw94rt3</a><o:p class=""></o:p></span></li><li class="MsoListParagraph" style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;"><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></li></ul></li><li class="MsoListParagraph" style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;"><span style="font-size: 11pt;" class="">If my calendar calculation is right, we will be meeting with the FT WG today<o:p class=""></o:p></span></li></ul><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">Thanks,<o:p class=""></o:p></span></div><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt;" class="">Howard<o:p class=""></o:p></span></div><div class=""><div class=""><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(11, 26, 141);" class=""><br class="">—</span><span style="font-size: 11pt;" class=""><o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(11, 26, 141);" class=""><o:p class=""> </o:p></span></div><table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="375" style="width: 281.25pt; border-collapse: collapse;"><tbody class=""><tr style="height: 104.45pt;" class=""><td width="78" valign="top" style="width: 58.4pt; padding: 0in 5.4pt; height: 104.45pt;" class=""><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 9pt; font-family: Arial, sans-serif;" class=""><span id="cid:image001.png@01D8008C.AD1D0010"><image001.png></span><o:p class=""></o:p></span></div></td><td width="297" valign="top" style="width: 223.1pt; padding: 0in 5.4pt; height: 104.45pt;" class=""><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><b class=""><span style="font-family: Arial, sans-serif; color: rgb(11, 26, 140);" class="">Howard Pritchard<o:p class=""></o:p></span></b></div><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><b class=""><span style="font-size: 10pt; font-family: Arial, sans-serif; color: rgb(84, 89, 97);" class="">Research Scientist<o:p class=""></o:p></span></b></div><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><b class=""><span style="font-size: 10pt; font-family: Arial, sans-serif; color: rgb(84, 89, 97);" class="">HPC-ENV<o:p class=""></o:p></span></b></div><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 9pt; font-family: Arial, sans-serif;" class=""><o:p class=""> </o:p></span></div><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><b class=""><span style="font-size: 9pt; font-family: Arial, sans-serif; color: rgb(11, 26, 140);" class="">Los Alamos National Laboratory<o:p class=""></o:p></span></b></div><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><b class=""><span style="font-size: 9pt; font-family: Arial, sans-serif; color: rgb(11, 26, 140);" class=""><a href="mailto:howardp@lanl.gov" style="color: rgb(5, 99, 193); text-decoration: underline;" class="">howardp@lanl.gov</a><o:p class=""></o:p></span></b></div><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><b class=""><span style="font-size: 9pt; font-family: Arial, sans-serif; color: rgb(11, 26, 140);" class=""><o:p class=""> </o:p></span></b></div><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><a href="https://www.instagram.com/losalamosnatlab/" style="color: rgb(5, 99, 193); text-decoration: underline;" class=""><b class=""><span style="font-size: 9pt; font-family: Arial, sans-serif; color: rgb(11, 26, 140); text-decoration: none;" class=""><span id="cid:image002.png@01D8008C.AD1D0010"><image002.png></span></span></b></a><a href="https://twitter.com/LosAlamosNatLab" style="color: rgb(5, 99, 193); text-decoration: underline;" class=""><b class=""><span style="font-size: 9pt; font-family: Arial, sans-serif; color: rgb(11, 26, 140); text-decoration: none;" class=""><span id="cid:image003.png@01D8008C.AD1D0010"><image003.png></span></span></b></a><a href="https://www.linkedin.com/company/los-alamos-national-laboratory/" style="color: rgb(5, 99, 193); text-decoration: underline;" class=""><b class=""><span style="font-size: 9pt; font-family: Arial, sans-serif; color: rgb(11, 26, 140); text-decoration: none;" class=""><span id="cid:image004.png@01D8008C.AD1D0010"><image004.png></span></span></b></a><a href="https://www.facebook.com/LosAlamosNationalLab/" style="color: rgb(5, 99, 193); text-decoration: underline;" class=""><b class=""><span style="font-size: 9pt; font-family: Arial, sans-serif; color: rgb(11, 26, 140); text-decoration: none;" class=""><span id="cid:image005.png@01D8008C.AD1D0010"><image005.png></span></span></b></a><b class=""><span style="font-size: 9pt; font-family: Arial, sans-serif; color: rgb(11, 26, 140);" class=""><o:p class=""></o:p></span></b></div></td></tr></tbody></table><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><span style="font-size: 11pt; font-family: Arial, sans-serif;" class=""><o:p class=""> </o:p></span></div></div></div><div style="margin: 0in; font-size: 12pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">_______________________________________________</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">mpiwg-sessions mailing list</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><a href="mailto:mpiwg-sessions@lists.mpi-forum.org" style="color: rgb(5, 99, 193); text-decoration: underline; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">mpiwg-sessions@lists.mpi-forum.org</a><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><a href="https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions" style="color: rgb(5, 99, 193); text-decoration: underline; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions</a></div></blockquote></div><br class=""></div></body></html>