<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Dan/Howard: can you provide me with a simple reproducer of the problem for OMPI? I may have a little time to track it down - have an idea of the source of the trouble from Dan’s offlist note.<div class=""><br class=""></div><div class="">Ralph</div><div class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Sep 17, 2018, at 8:42 AM, HOLMES Daniel via mpiwg-sessions <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" class="">mpiwg-sessions@lists.mpi-forum.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class="">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Hi Howard, et al,
<div class=""><br class="">
</div>
<div class="">Howard: thanks for checking this on MPICH. Looks like we need a cross-mpirun example code - we could work on that this week.<br class="">
<div class=""><br class="webkit-block-placeholder">
</div>
<div class="">
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
All: *reminder* no telecon today - we’ll meet up during the MPI Forum in a couple of days.</div>
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<br class="Apple-interchange-newline">
Cheers,</div>
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Dan.</div>
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
—<br class="">
Dr Daniel Holmes PhD<br class="">
Applications Consultant in HPC Research<br class="">
<a href="mailto:d.holmes@epcc.ed.ac.uk" class="">d.holmes@epcc.ed.ac.uk</a><br class="">
Phone: +44 (0) 131 651 3465<br class="">
Mobile: +44 (0) 7940 524 088<br class="">
Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT</div>
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
—</div>
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.</div>
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
—</div>
</div>
<br class="">
<div class="">
<blockquote type="cite" class="">
<div class="">On 17 Sep 2018, at 05:39, Pritchard Jr.,<span class="Apple-tab-span" style="white-space:pre">
</span>Howard via mpiwg-sessions <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" class="">mpiwg-sessions@lists.mpi-forum.org</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; font-size: 14px; font-family: Calibri, sans-serif;" class="">
<div class="">
<div class="">Hi Folks,</div>
<div class=""><br class="">
</div>
<div class="">I made some minor corrections to the two problem test cases and they now work</div>
<div class="">with mpich. I opened a PR and assigned Dan as the reviewer.</div>
<div class=""><br class="">
</div>
<div class="">Howard</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">
<div class="">-- </div>
<div class="">Howard Pritchard</div>
</div>
<div class="">B Schedule</div>
<div class="">HPC-ENV</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt;" class="">Office 9, 2<sup class="">nd</sup> floor Research Park<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt;" class="">TA-03, Building 4200, Room 203</div>
</div>
<div class="">Los Alamos National Laboratory</div>
<div class=""><br class="">
</div>
</div>
</div>
<div class=""><br class="">
</div>
<span id="OLK_SRC_BODY_SECTION" class="">
<div style="font-family: Calibri; font-size: 11pt; text-align: left; border-width: 1pt medium medium; border-style: solid none none; padding: 3pt 0in 0in; border-top-color: rgb(181, 196, 223);" class="">
<span style="font-weight:bold" class="">From: </span>mpiwg-sessions <<a href="mailto:mpiwg-sessions-bounces@lists.mpi-forum.org" class="">mpiwg-sessions-bounces@lists.mpi-forum.org</a>> on behalf of MPI Sessions working group <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" class="">mpiwg-sessions@lists.mpi-forum.org</a>><br class="">
<span style="font-weight:bold" class="">Reply-To: </span>MPI Sessions working group <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" class="">mpiwg-sessions@lists.mpi-forum.org</a>><br class="">
<span style="font-weight:bold" class="">Date: </span>Sunday, September 16, 2018 at 10:10 PM<br class="">
<span style="font-weight:bold" class="">To: </span>MPI Sessions working group <<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" class="">mpiwg-sessions@lists.mpi-forum.org</a>><br class="">
<span style="font-weight:bold" class="">Cc: </span>Howard Pritchard <<a href="mailto:howardp@lanl.gov" class="">howardp@lanl.gov</a>><br class="">
<span style="font-weight:bold" class="">Subject: </span>[mpiwg-sessions] MPICH/hydra happier with Dan's test cases (kind of)<br class="">
</div>
<div class=""><br class="">
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;" class="" type="cite">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div class="">
<div class="">
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class="">HI Folks,</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><br class="">
</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class="">MPICH/hydra is happy with <span style="color: rgb(51, 196, 38); font-family: Menlo; font-size: 13px;" class="">libCFG_noMCW,
</span><span style="font-family: Menlo; font-size: 13px;" class="">even with all the sleeps</span></div>
<div class=""><font face="Calibri" class=""><font class="">removed. I ran up to 36 ranks with hydra/mpich and didn</font>’<font class="">t see a problem.</font></font></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><span style="font-family: Menlo; font-size: 13px;" class=""><br class="">
</span></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><font face="Menlo" class=""><span style="font-size: 13px;" class="">Its not so happy with the other tests, I think they are buggy.</span></font></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><font face="Menlo" class=""><span style="font-size: 13px;" class=""><br class="">
</span></font></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><font face="Menlo" class=""><span style="font-size: 13px;" class="">Here’s what I get for </span></font><span style="font-family: Menlo; font-size: 13px;" class="">libCFG_noMCW_multiport</span></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><span style="font-family: Menlo; font-size: 13px;" class=""><br class="">
</span></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class="">
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">hpp@sn-fey1:/usr/projects/hpctools/hpp/mpi_sessions_code_sandbox>mpiexec -n 8 ./libCFG_noMCW_multiport</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">process 1 (MPI_COMM_WORLD) now calling barrier on MPI_COMM_WORLD</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">process 3 (MPI_COMM_WORLD) now calling barrier on MPI_COMM_WORLD</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">process 4 (MPI_COMM_WORLD) calling from group thingy</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">rank 4 non-trivial use-case: target group size 4, localGroup size 1</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">rank 4 opening port</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">process 5 (MPI_COMM_WORLD) now calling barrier on MPI_COMM_WORLD</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">process 6 (MPI_COMM_WORLD) calling from group thingy</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">rank 6 non-trivial use-case: target group size 4, localGroup size 1</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">rank 6 opening port</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">process 7 (MPI_COMM_WORLD) now calling barrier on MPI_COMM_WORLD</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">process 0 (MPI_COMM_WORLD) calling from group thingy</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">rank 0 non-trivial use-case: target group size 4, localGroup size 1</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">process 2 (MPI_COMM_WORLD) calling from group thingy</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">rank 2 non-trivial use-case: target group size 4, localGroup size 1</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">rank 2 opening port</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">rank 2 opened port tag#0$description#<a href="http://sn-fey1.lanl.gov/" class="">sn-fey1.lanl.gov</a>$port#55566$ifname#128.165.227.181$</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">rank 2 publishing port tag#0$description#<a href="http://sn-fey1.lanl.gov/" class="">sn-fey1.lanl.gov</a>$port#55566$ifname#128.165.227.181$ using name foobar10 round 1</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">rank 4 opened port tag#0$description#<a href="http://sn-fey1.lanl.gov/" class="">sn-fey1.lanl.gov</a>$port#45508$ifname#128.165.227.181$</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">rank 4 publishing port tag#0$description#<a href="http://sn-fey1.lanl.gov/" class="">sn-fey1.lanl.gov</a>$port#45508$ifname#128.165.227.181$ using name foobar10 round 2</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">rank 6 opened port tag#0$description#<a href="http://sn-fey1.lanl.gov/" class="">sn-fey1.lanl.gov</a>$port#52388$ifname#128.165.227.181$</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">rank 6 publishing port tag#0$description#<a href="http://sn-fey1.lanl.gov/" class="">sn-fey1.lanl.gov</a>$port#52388$ifname#128.165.227.181$ using name foobar10 round 3</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">rank 2 published port tag#0$description#<a href="http://sn-fey1.lanl.gov/" class="">sn-fey1.lanl.gov</a>$port#55566$ifname#128.165.227.181$ using name foobar10 round 1</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">rank 2 accepting on port tag#0$description#<a href="http://sn-fey1.lanl.gov/" class="">sn-fey1.lanl.gov</a>$port#55566$ifname#128.165.227.181$ (localSize 1)</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">Fatal error in PMPI_Publish_name: Invalid service name (see MPI_Publish_name), error stack:</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">PMPI_Publish_name(134): MPI_Publish_name(service="foobar10 round 2", MPI_INFO_NULL, port="tag#0$description#<a href="http://sn-fey1.lanl.gov/" class="">sn-fey1.lanl.gov</a>$port#45508$ifname#128.165.227.181$")
failed</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">MPID_NS_Publish(67)...: Lookup failed for service name foobar10 round 2</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">Fatal error in PMPI_Publish_name: Invalid service name (see MPI_Publish_name), error stack:</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">PMPI_Publish_name(134): MPI_Publish_name(service="foobar10 round 3", MPI_INFO_NULL, port="tag#0$description#<a href="http://sn-fey1.lanl.gov/" class="">sn-fey1.lanl.gov</a>$port#52388$ifname#128.165.227.181$")
failed</span></div>
<div style="margin: 0px; font-size: 13px; line-height: normal; font-family: Menlo;" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">MPID_NS_Publish(67)...: Lookup failed for service name foobar10 round 3</span></div>
</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class="">
</span></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class="">
</span></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">If I have a chance I”ll play with this test and see if I can get it to work. hmm.. maybe hydra doesn’t like the whitespace
publish names.</span></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class="">
</span></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Howard</span></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class="">
</span></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class="">
</span></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class="">
<div class="">
<div class="">-- </div>
<div class="">Howard Pritchard</div>
</div>
<div class="">B Schedule</div>
<div class="">HPC-ENV</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt;" class="">Office 9, 2<sup class="">nd</sup> floor Research Park<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt;" class="">TA-03, Building 4200, Room 203</div>
</div>
<div class="">Los Alamos National Laboratory</div>
<div class=""><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</span></div>
_______________________________________________<br class="">
mpiwg-sessions mailing list<br class="">
<a href="mailto:mpiwg-sessions@lists.mpi-forum.org" class="">mpiwg-sessions@lists.mpi-forum.org</a><br class="">
<a href="https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions" class="">https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions</a><br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
The University of Edinburgh is a charitable body, registered in<br class="">Scotland, with registration number SC005336.<br class="">_______________________________________________<br class="">mpiwg-sessions mailing list<br class=""><a href="mailto:mpiwg-sessions@lists.mpi-forum.org" class="">mpiwg-sessions@lists.mpi-forum.org</a><br class="">https://lists.mpi-forum.org/mailman/listinfo/mpiwg-sessions<br class=""></div></blockquote></div><br class=""></div></body></html>