<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Hi Kathryn,</p>
<p>The PDF file containing the notes for the 2019-03-21 meeting seems to 0 length, or perhaps the link on the Wiki page is bad. Is it me or is there a problem on the Wiki?<br>
</p>
<p>Cheers, John D.<br>
</p>
<br>
<div class="moz-cite-prefix">On 03/24/19 13:36, Mohror, Kathryn via mpiwg-tools wrote:<br>
</div>
<blockquote cite="mid:MWHPR09MB153629FA0251C1369C61964F9C5D0@MWHPR09MB1536.namprd09.prod.outlook.com" type="cite">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:222496039;
mso-list-template-ids:-1630996188;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1
{mso-list-id:759913447;
mso-list-template-ids:-144645562;}
@list l1:level1
{mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level2
{mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level3
{mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level4
{mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level5
{mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level6
{mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level7
{mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level8
{mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level9
{mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2
{mso-list-id:840701831;
mso-list-template-ids:-1486297582;}
@list l2:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level2
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l2:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3
{mso-list-id:1347516090;
mso-list-template-ids:962083176;}
@list l3:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level2
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal">Hi all,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I added our notes from the last two meetings on debugger interfaces:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><a class="moz-txt-link-freetext" href="https://github.com/mpiwg-tools/tools-issues/wiki/Notes-2019-02-28">https://github.com/mpiwg-tools/tools-issues/wiki/Notes-2019-02-28</a><o:p></o:p></p>
<p class="MsoNormal"><a moz-do-not-send="true" href="https://github.com/mpiwg-tools/tools-issues/wiki/Notes-2019-03-24">https://github.com/mpiwg-tools/tools-issues/wiki/Notes-2019-03-24</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Kathryn<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> mpiwg-tools <a class="moz-txt-link-rfc2396E" href="mailto:mpiwg-tools-bounces@lists.mpi-forum.org">
<mpiwg-tools-bounces@lists.mpi-forum.org></a> <b>On Behalf Of </b>Ahn, Dong H. via mpiwg-tools<br>
<b>Sent:</b> Thursday, March 21, 2019 10:17 AM<br>
<b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:mpiwg-tools@lists.mpi-forum.org">
mpiwg-tools@lists.mpi-forum.org</a><br>
<b>Cc:</b> Ahn, Dong H. <a class="moz-txt-link-rfc2396E" href="mailto:ahn1@llnl.gov">
<ahn1@llnl.gov></a><br>
<b>Subject:</b> Re: [mpiwg-tools] Meeting March 21, 2019<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hi Ralph and all,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Definitely useful use case! <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I have not been able to call into this concall series, but I have a general input as I’m pretty familiar with these debug interfaces (as well as resource manager mechanisms).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Generally speaking, I like any standard interface to be lean and mean with good separations of concerns so that many implementors can implement the interface. In that context, it seems it would be good to discuss what functionalities should
be implemented at the tool’s side and what should be provided by the new MPIR2 standard.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">In this particular use case, a remote tool can easily ssh into the remote node and co-locate an agent to create a session like you are describing using whatever requisite security mechanism for the site. If the standard interface tries
to solve this problem within itself, I fear it may lose good separation of concerns and the interface can quickly become unwieldy.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">But I do agree it would be good to capture this use case somewhere and factor that into when MPIR2 interface starts to emerge.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Just my $0.02.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Dong<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">mpiwg-tools <<a moz-do-not-send="true" href="mailto:mpiwg-tools-bounces@lists.mpi-forum.org">mpiwg-tools-bounces@lists.mpi-forum.org</a>>
on behalf of Ralph H Castain via mpiwg-tools <<a moz-do-not-send="true" href="mailto:mpiwg-tools@lists.mpi-forum.org">mpiwg-tools@lists.mpi-forum.org</a>><br>
<b>Reply-To: </b>"<a moz-do-not-send="true" href="mailto:mpiwg-tools@lists.mpi-forum.org">mpiwg-tools@lists.mpi-forum.org</a>" <<a moz-do-not-send="true" href="mailto:mpiwg-tools@lists.mpi-forum.org">mpiwg-tools@lists.mpi-forum.org</a>><br>
<b>Date: </b>Thursday, March 21, 2019 at 9:37 AM<br>
<b>To: </b>"<a moz-do-not-send="true" href="mailto:mpiwg-tools@lists.mpi-forum.org">mpiwg-tools@lists.mpi-forum.org</a>" <<a moz-do-not-send="true" href="mailto:mpiwg-tools@lists.mpi-forum.org">mpiwg-tools@lists.mpi-forum.org</a>><br>
<b>Cc: </b>Ralph H Castain <<a moz-do-not-send="true" href="mailto:rhc@open-mpi.org">rhc@open-mpi.org</a>><br>
<b>Subject: </b>Re: [mpiwg-tools] Meeting March 21, 2019<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">Quick follow-up to today’s call. I’d like to suggest adding a bullet to the list of requirements for supporting remote tool connections - i.e., where the tool is executing on a node different from the process to which it is attached (RM
or launcher). This is a really useful capability. For example, I am currently running a GUI-based tool on my desktop computer that is attached and controlling an application launcher on a cluster 200 miles away (I confess it is using PMIx). Yes, I could login
to that machine, but then I either am limited to a text screen or must utilize secondary tools like VNC or X11. Much cleaner to just directly connect/control.
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">This is where I was going with the ability to detach and remain running - however, I think some explicit language about remote operations might be useful.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">HTH<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Ralph<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Mar 21, 2019, at 6:46 AM, Mohror, Kathryn via mpiwg-tools <<a moz-do-not-send="true" href="mailto:mpiwg-tools@lists.mpi-forum.org">mpiwg-tools@lists.mpi-forum.org</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Hi all,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks Dick and John for sending your use cases and straw men!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Here are a couple more docs I found from our discussions in the past:<o:p></o:p></p>
</div>
<ol style="margin-top:0in" start="1" type="1">
<li class="MsoListParagraph" style="margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l1
level1 lfo1">
Straw man from Anh Vo (mpi_debugging_requirements-AV-05-10-2016)<o:p></o:p></li><li class="MsoListParagraph" style="margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l1
level1 lfo1">
Needs from Allinea debugger as described by Chris January (mpi_debugging_requirements (002))<o:p></o:p></li><li class="MsoListParagraph" style="margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l1
level1 lfo1">
Notes from Forum meeting 2/27/2017:<span class="apple-converted-space"> </span><a moz-do-not-send="true" href="https://github.com/mpiwg-tools/tools-issues/wiki/Notes-2017-02-27"><span style="color:purple">https://github.com/mpiwg-tools/tools-issues/wiki/Notes-2017-02-27</span></a><o:p></o:p></li></ol>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Talk to you all soon!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Kathryn<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0in 0in 0in">
<div>
<p class="MsoNormal"><b>From:</b><span class="apple-converted-space"> </span>mpiwg-tools <<a moz-do-not-send="true" href="mailto:mpiwg-tools-bounces@lists.mpi-forum.org"><span style="color:purple">mpiwg-tools-bounces@lists.mpi-forum.org</span></a>><span class="apple-converted-space"> </span><b>On
Behalf Of<span class="apple-converted-space"> </span></b>Dirk Schubert via mpiwg-tools<br>
<b>Sent:</b><span class="apple-converted-space"> </span>Thursday, March 21, 2019 3:43 AM<br>
<b>To:</b><span class="apple-converted-space"> </span><a moz-do-not-send="true" href="mailto:mpiwg-tools@lists.mpi-forum.org"><span style="color:purple">mpiwg-tools@lists.mpi-forum.org</span></a><br>
<b>Cc:</b><span class="apple-converted-space"> </span>Dirk Schubert <<a moz-do-not-send="true" href="mailto:Dirk.Schubert@arm.com"><span style="color:purple">Dirk.Schubert@arm.com</span></a>><br>
<b>Subject:</b><span class="apple-converted-space"> </span>Re: [mpiwg-tools] Meeting March 21, 2019<o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div id="divtagdefaultwrapper">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif">Hi Kathryn,</span><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif"> </span><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:12.0pt">The following use cases are important for Arm Forge (debugger DDT and profiler MAP).<span class="apple-converted-space"> </span><br>
<br>
<b>Use case 1: Launch an MPI job under the control of a tool.<span class="apple-converted-space"> </span></b><br>
<br>
<i>Input:</i><span class="apple-converted-space"> </span>The user's regular command line to start the MPI job, for example "mpirun -n 256 ./wave_c". Why? To make it as friction-less as possible to use a tool.<span class="apple-converted-space"> </span><br>
<br>
NB: The tool does not want to parse or interpret the MPI starter arguments, as the tool has no knowledge of all possible MPI starter arguments.<span class="apple-converted-space"> </span><br>
<br>
<i>Requirements:<span class="apple-converted-space"> </span></i></span><o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l2 level1
lfo2"><b><span style="font-size:12.0pt">Hold MPI processes</span></b><span class="apple-converted-space"><span style="font-size:12.0pt"> </span></span><span style="font-size:12.0pt">for
the tool to attach to them and to release them when attached.<br>
Q: Where are processes held? In MPI_Init or _start?<span class="apple-converted-space"> </span></span><o:p></o:p></li><li class="MsoNormal" style="mso-list:l2 level1
lfo2"><b><span style="font-size:12.0pt">Spawn tool daemon</span></b><span class="apple-converted-space"><span style="font-size:12.0pt"> </span></span><span style="font-size:12.0pt">on
nodes where job is running.<br>
Q: Co-spawn or separate?<br>
Q: One per node or one per process?<br>
Q: Environment? Same general environment as MPI processes instead of restricted environment (e.g. $HOME not set or chroot).</span><o:p></o:p></li><li class="MsoNormal" style="mso-list:l2 level1
lfo2"><b><span style="font-size:12.0pt">Acquire the process table</span></b><span class="apple-converted-space"><span style="font-size:12.0pt"> </span></span><span style="font-size:12.0pt">(rank,
hostname, pid and executable path) of spawned MPI processes.<br>
Q: Only global proctable or additionally local (per node) proctable?<span class="apple-converted-space"> </span></span><o:p></o:p></li><li class="MsoNormal" style="mso-list:l2 level1
lfo2"><span style="font-size:12.0pt;color:#757B80">[Optional]</span><span class="apple-converted-space"><span style="font-size:12.0pt"> </span></span><b><span style="font-size:12.0pt">Modify
the environment</span></b><span class="apple-converted-space"><span style="font-size:12.0pt"> </span></span><span style="font-size:12.0pt">of the MPI processes before launching, such as prepending to LD_PRELOAD or LD_LIBRARY_PATH to inject preloads into the
MPI processes (but not other processes, such as MPI daemons or tool daemons).<br>
Q: When? It would be great if modifying the environment could be done just before the MPI processes are forked/exec’ed. Why? For our profiler MAP we need to preload an MPI specific PMPI library, but for some “MPI” starter processes such as srun we don’t know
the real MPI implementation that is used. Acquiring a partial process table (no PIDs) and inspecting the binaries pointed to by the executable paths, could allow us to detect the MPI implementation.</span><o:p></o:p></li><li class="MsoNormal" style="mso-list:l2 level1
lfo2"><span style="font-size:12.0pt;color:#757B80">[Optional]</span><span class="apple-converted-space"><span style="font-size:12.0pt"> </span></span><b><span style="font-size:12.0pt">Raise
queue allocation request and granted events</span></b><span class="apple-converted-space"><span style="font-size:12.0pt"> </span></span><span style="font-size:12.0pt">to allow the tool to disable startup timeouts temporarily while a queue allocation is in
progress, as this can take a long time (applicable to srun).</span><o:p></o:p></li><li class="MsoNormal" style="mso-list:l2 level1
lfo2"><span style="font-size:12.0pt;color:#757B80">[Optional]</span><span class="apple-converted-space"><span style="font-size:12.0pt"> </span></span><b><span style="font-size:12.0pt">Scalable
startup of tool daemons and shipping of preloads</span></b><span class="apple-converted-space"><span style="font-size:12.0pt"> </span></span><span style="font-size:12.0pt">without touching the parallel file system on compute nodes. For example, pushing of
daemon executable file (and dependencies plus preloads) to ram disk on compute nodes or Spindle?<br>
Q: Where files are pushed? A job specific temporary directory such as /tmp/mpi.job.1234/... or configurable?<br>
Q: How can the tool query the location of pushed files to reference them?<br>
Q: Who cleans up the files afterwards? The MPI job/starter itself or the tool?<br>
N: Files should be pushed with original file’s permissions.<span class="apple-converted-space"> </span></span><o:p></o:p></li></ul>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><br>
NB: Optional requirement = A tool could ultimately work without it, but I very much would like to see it supported.<span class="apple-converted-space"> </span><br>
<br>
In PMIx this is the “indirect” use case and Ralph Castain and I have been working together since a while to make sure this use case will be supported by PMIx (excluding some optional requirements for now).</span><o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:12.0pt"><br>
<i>Additional questions/notes:</i></span><o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l0 level1
lfo3"><span style="font-size:12.0pt">How will the handshake work, such that the MPI starter knows it's running under a tool and when it's possible for a tool to interact with the MPI starter
process?</span><o:p></o:p></li><li class="MsoNormal" style="mso-list:l0 level1
lfo3"><span style="font-size:12.0pt">The handshake must support cases where the MPI starter filename that is provided by the user is a wrapper around the real MPI starter executable, for
example XALT. </span><o:p></o:p></li><li class="MsoNormal" style="mso-list:l0 level1
lfo3"><span style="font-size:12.0pt">In PMIx both problems are solved with PMIX_LAUNCHER_PAUSE_FOR_TOOL=1 and PMIX_LAUNCHER_RENDEZVOUS_FILE=<filename>.</span><o:p></o:p></li><li class="MsoNormal" style="mso-list:l0 level1
lfo3"><span style="font-size:12.0pt">Will only the tool or also the tool daemons interact with the MPI job?</span><o:p></o:p></li><li class="MsoNormal" style="mso-list:l0 level1
lfo3"><span style="font-size:12.0pt">Security, for example USER B shall not be able to attach to MPI job of USER A.</span><o:p></o:p></li><li class="MsoNormal" style="mso-list:l0 level1
lfo3"><span style="font-size:12.0pt">Anything to consider for MPMD or heterogeneous systems?</span><o:p></o:p></li><li class="MsoNormal" style="mso-list:l0 level1
lfo3"><span style="font-size:12.0pt">Must be scalable.</span><o:p></o:p></li><li class="MsoNormal" style="mso-list:l0 level1
lfo3"><span style="font-size:12.0pt">Must not require a debugger.<span class="apple-converted-space"> </span></span><o:p></o:p></li></ul>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:12.0pt"><br>
<b>Use case 2: Attach to a running MPI job with a tool.<span class="apple-converted-space"> </span></b><br>
<br>
This use case is a subset of use case 1 with the following requirements:<span class="apple-converted-space"> </span></span><o:p></o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="mso-list:l3 level1
lfo4"><span style="font-size:12.0pt">Acquire the process table of spawned MPI processes of MPI job. </span><o:p></o:p></li><li class="MsoNormal" style="mso-list:l3 level1
lfo4"><span style="font-size:12.0pt">Spawn tool daemon on nodes where MPI job is running.</span><o:p></o:p></li><li class="MsoNormal" style="mso-list:l3 level1
lfo4"><span style="font-size:12.0pt;color:#757B80">[Optional]</span><span class="apple-converted-space"><span style="font-size:12.0pt"> </span></span><span style="font-size:12.0pt">Scalable
startup of tool daemons without touching the file system.</span><o:p></o:p></li></ul>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif">Thanks,</span><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif">Dirk</span><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif"> </span><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><o:p></o:p></span></p>
<div id="Signature">
<div id="divtagdefaultwrapper">
<div>
<div>
<p class="MsoNormal"><b><span style="font-size:10.0pt">Dirk Schubert | Arm |</span></b><span style="font-size:10.0pt"> Staff Software Engineer </span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt"><a moz-do-not-send="true" href="mailto:dirk.schubert@arm.com"><span style="color:purple">dirk.schubert@arm.com</span></a></span><o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:10.0pt;font-family:"Helvetica",sans-serif;color:#3A5979">allinea</span></b><span class="apple-converted-space"><b><span style="font-size:10.0pt;font-family:"Helvetica",sans-serif"> </span></b></span><span style="font-size:10.0pt;font-family:"Helvetica",sans-serif;color:gray">is
now part of<span class="apple-converted-space"><b> </b></span></span><b><span style="font-size:10.0pt;font-family:"Helvetica",sans-serif;color:#138DAC">Arm</span></b><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><o:p></o:p></span></p>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
<div>
<div class="MsoNormal" style="text-align:center" align="center"><span style="font-size:12.0pt">
<hr align="center" size="1" width="100%">
</span></div>
<div id="divRplyFwdMsg">
<div>
<p class="MsoNormal"><b>From:</b><span class="apple-converted-space"> </span>mpiwg-tools <<a moz-do-not-send="true" href="mailto:mpiwg-tools-bounces@lists.mpi-forum.org"><span style="color:purple">mpiwg-tools-bounces@lists.mpi-forum.org</span></a>> on behalf
of Mohror, Kathryn via mpiwg-tools <<a moz-do-not-send="true" href="mailto:mpiwg-tools@lists.mpi-forum.org"><span style="color:purple">mpiwg-tools@lists.mpi-forum.org</span></a>><br>
<b>Sent:</b><span class="apple-converted-space"> </span>17 March 2019 14:46<br>
<b>To:</b><span class="apple-converted-space"> </span><a moz-do-not-send="true" href="mailto:mpiwg-tools@lists.mpi-forum.org"><span style="color:purple">mpiwg-tools@lists.mpi-forum.org</span></a><br>
<b>Cc:</b><span class="apple-converted-space"> </span>Mohror, Kathryn<br>
<b>Subject:</b><span class="apple-converted-space"> </span>[mpiwg-tools] Meeting March 21, 2019<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">Hey everybody,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">For our call this Thursday (3/21) we’ll come back to debugger topics again. The call is at the usual time in the US (8 am Pacific/ 11 am Eastern / 4 pm CET) but note that the US has moved to DST but I don’t
think the EU has done so yet.<span class="apple-converted-space"> </span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">Webex Info:</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"> <span class="apple-converted-space"> </span><a moz-do-not-send="true" href="https://apps.fz-juelich.de/scalasca/mpiwg-tools-webex/index.php"><span style="color:purple">https://apps.fz-juelich.de/scalasca/mpiwg-tools-webex/index.php</span></a></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">Agenda:</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">In the last debugger call, I said that the plan for this meeting would be to get an overview of OMPD and start talking about analogous interfaces for MPI (e.g. revamp MQD). However, in the meantime, I was
convinced that we should focus on process acquisition first. So, the plan will be to talk about process acquisition in this meeting. I’ll go through our notes from 2017(ish) to hopefully find the straw men we drafted back then. Please bring your use cases
and any ideas you have on this front.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">_________________________________________________________________</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">Kathryn Mohror,<span class="apple-converted-space"> </span><a moz-do-not-send="true" href="mailto:kathryn@llnl.gov"><span style="color:purple">kathryn@llnl.gov</span></a>,<span class="apple-converted-space"> </span><a moz-do-not-send="true" href="https://people.llnl.gov/kathryn"><span style="color:purple">https://people.llnl.gov/kathryn</span></a></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><a moz-do-not-send="true" href="https://computation.llnl.gov/casc/data-analysis-group"><span style="color:purple">Data Analysis Group</span></a><span class="apple-converted-space"> </span>@ Lawrence Livermore
National Laboratory, Livermore, CA, USA</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
<div>
<p class="MsoNormal">IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person,
use it for any purpose, or store or copy the information in any medium. Thank you.<o:p></o:p></p>
</div>
<p class="MsoNormal"><mpi_debugging_requirements-AV-05-10-2016.pdf><mpi_debugging_requirements (002).pdf><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">_______________________________________________<br>
mpiwg-tools mailing list<br>
</span><a moz-do-not-send="true" href="mailto:mpiwg-tools@lists.mpi-forum.org"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif;color:purple">mpiwg-tools@lists.mpi-forum.org</span></a><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><br>
</span><a moz-do-not-send="true" href="https://lists.mpi-forum.org/mailman/listinfo/mpiwg-tools"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif;color:purple">https://lists.mpi-forum.org/mailman/listinfo/mpiwg-tools</span></a><o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset> <br>
<pre wrap="">_______________________________________________
mpiwg-tools mailing list
<a class="moz-txt-link-abbreviated" href="mailto:mpiwg-tools@lists.mpi-forum.org">mpiwg-tools@lists.mpi-forum.org</a>
<a class="moz-txt-link-freetext" href="https://lists.mpi-forum.org/mailman/listinfo/mpiwg-tools">https://lists.mpi-forum.org/mailman/listinfo/mpiwg-tools</a>
</pre>
</blockquote>
<br>
</body>
</html>