<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
tt
{mso-style-priority:99;
font-family:"Courier New";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=WordSection1>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>If we do as Darius suggests and leave this to be
implementation-dependent, then implementations will be free to do the
reasonable thing. Specifically, if the user provides a range of process counts
that would be acceptable, the implementation would be allowed to let the
communicator include only the ranks that were allocated. Any subsequent
failures would result in ranks being non-operational. However, it is not
possible to specify this behavior in the spec because the MPI spec doesn’t have
a concept of the number of processes that the application asked for.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Greg Bronevetsky<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Lawrence Livermore National Lab<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";
color:#1F497D'>(925) 424-5756<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";
color:#1F497D'>bronevetsky@llnl.gov<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";
color:#1F497D'>http://greg.bronevetsky.com <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>
mpi3-ft-bounces@lists.mpi-forum.org
[mailto:mpi3-ft-bounces@lists.mpi-forum.org] <b>On Behalf Of </b>Richard
Treumann<br>
<b>Sent:</b> Friday, September 17, 2010 7:37 AM<br>
<b>To:</b> MPI 3.0 Fault Tolerance and Dynamic Process Control working Group<br>
<b>Subject:</b> Re: [Mpi3-ft] Stabilization Updated & MPI_Comm_size
question<o:p></o:p></span></p>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal style='margin-bottom:12.0pt'><br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>If I used some
mechanism that allows the MPI launch to give me fewer processes than I asked
for and it does give me less than I asked for, what does this mean? </span>
<br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>"</span><tt><span
style='font-size:10.0pt'>number of ranks in the communicator regardless of
whether they're operational</span></tt><span style='font-size:10.0pt;
font-family:"Arial","sans-serif"'>"</span> <br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>It is sort of
weird to say that the processes I was never given are "in the
communicator". I can see the logic in counting a processes that have died
as "</span><tt><span style='font-size:10.0pt'>ranks in the communicator</span></tt><span
style='font-size:10.0pt;font-family:"Arial","sans-serif"'>"</span> <br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>I would not
count chickens that never hatched.</span> <br>
<br>
<br>
<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Dick Treumann
- MPI Team <br>
IBM Systems & Technology Group<br>
Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601<br>
Tele (845) 433-7846 Fax (845) 433-8363<br>
</span><br>
<br>
<o:p></o:p></p>
<table class=MsoNormalTable border=0 cellpadding=0 width="100%"
style='width:100.0%'>
<tr>
<td valign=top style='padding:.75pt .75pt .75pt .75pt'>
<p class=MsoNormal><span style='font-size:7.5pt;font-family:"Arial","sans-serif";
color:#5F5F5F'>From:</span> <o:p></o:p></p>
</td>
<td valign=top style='padding:.75pt .75pt .75pt .75pt'>
<p class=MsoNormal><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>"Bronevetsky,
Greg" <bronevetsky1@llnl.gov></span> <o:p></o:p></p>
</td>
</tr>
<tr>
<td valign=top style='padding:.75pt .75pt .75pt .75pt'>
<p class=MsoNormal><span style='font-size:7.5pt;font-family:"Arial","sans-serif";
color:#5F5F5F'>To:</span> <o:p></o:p></p>
</td>
<td valign=top style='padding:.75pt .75pt .75pt .75pt'>
<p class=MsoNormal><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>"Bronis
R. de Supinski" <bronis@llnl.gov>, "MPI 3.0 Fault Tolerance
and Dynamic Process Control working Group"
<mpi3-ft@lists.mpi-forum.org></span> <o:p></o:p></p>
</td>
</tr>
<tr>
<td valign=top style='padding:.75pt .75pt .75pt .75pt'>
<p class=MsoNormal><span style='font-size:7.5pt;font-family:"Arial","sans-serif";
color:#5F5F5F'>Date:</span> <o:p></o:p></p>
</td>
<td valign=top style='padding:.75pt .75pt .75pt .75pt'>
<p class=MsoNormal><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>09/17/2010
10:03 AM</span> <o:p></o:p></p>
</td>
</tr>
<tr>
<td valign=top style='padding:.75pt .75pt .75pt .75pt'>
<p class=MsoNormal><span style='font-size:7.5pt;font-family:"Arial","sans-serif";
color:#5F5F5F'>Subject:</span> <o:p></o:p></p>
</td>
<td valign=top style='padding:.75pt .75pt .75pt .75pt'>
<p class=MsoNormal><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>Re:
[Mpi3-ft] Stabilization Updated & MPI_Comm_size question</span> <o:p></o:p></p>
</td>
</tr>
<tr>
<td valign=top style='padding:.75pt .75pt .75pt .75pt'>
<p class=MsoNormal><span style='font-size:7.5pt;font-family:"Arial","sans-serif";
color:#5F5F5F'>Sent by:</span> <o:p></o:p></p>
</td>
<td valign=top style='padding:.75pt .75pt .75pt .75pt'>
<p class=MsoNormal><span style='font-size:7.5pt;font-family:"Arial","sans-serif"'>mpi3-ft-bounces@lists.mpi-forum.org</span><o:p></o:p></p>
</td>
</tr>
</table>
<p class=MsoNormal><o:p> </o:p></p>
<div class=MsoNormal align=center style='text-align:center'>
<hr size=2 width="100%" noshade style='color:#ACA899' align=center>
</div>
<p class=MsoNormal style='margin-bottom:12.0pt'><br>
<br>
<br>
<tt><span style='font-size:10.0pt'>I agree that MPI_Comm_size should return the
number of ranks in the communicator regardless of whether they're operational
or not. However, this just pushes the question further back: if the user asked
for N processes but only M have started up, how many ranks should
MPI_COMM_WORLD have? Either choice is going to be self-consistent from MPI's
point of view. If its N, then some ranks will be dead. If it is M then the
application may not have enough processes to work with. The former (N) case has
property that the user doesn't need to add code to check for this condition
since their existing error checking code will catch this situation. The latter
case (M) is nice because it cheaper to check whether we got fewer processes
than expected than to explicitly try to communicate with them. </span></tt><span
style='font-size:10.0pt;font-family:"Courier New"'><br>
<br>
<tt>As such, I don't see a strong motivation for choosing either. However, we
should just pick one and stick with it to avoid unnecessary API divergence.</tt><br>
<br>
<tt>Greg Bronevetsky</tt><br>
<tt>Lawrence Livermore National Lab</tt><br>
<tt>(925) 424-5756</tt><br>
<tt>bronevetsky@llnl.gov</tt><br>
</span><a href="http://BLOCKEDgreg.bronevetsky.com/"><tt><span
style='font-size:10.0pt'>http://BLOCKEDgreg.bronevetsky.com</span></tt></a><tt><span
style='font-size:10.0pt'> </span></tt><span style='font-size:10.0pt;font-family:
"Courier New"'><br>
<br>
<tt>> -----Original Message-----</tt><br>
<tt>> From: mpi3-ft-bounces@lists.mpi-forum.org [</tt></span><a
href="mailto:mpi3-ft-"><tt><span style='font-size:10.0pt'>mailto:mpi3-ft-</span></tt></a><span
style='font-size:10.0pt;font-family:"Courier New"'><br>
<tt>> bounces@lists.mpi-forum.org] On Behalf Of Bronis R. de Supinski</tt><br>
<tt>> Sent: Friday, September 17, 2010 5:23 AM</tt><br>
<tt>> To: MPI 3.0 Fault Tolerance and Dynamic Process Control working Group</tt><br>
<tt>> Subject: Re: [Mpi3-ft] Stabilization Updated & MPI_Comm_size
question</tt><br>
<tt>> </tt><br>
<tt>> </tt><br>
<tt>> I agree with Rich and Darius.</tt><br>
<tt>> </tt><br>
<tt>> On Fri, 17 Sep 2010, Darius Buntinas wrote:</tt><br>
<tt>> </tt><br>
<tt>> ></tt><br>
<tt>> > I don't think we should change the standard in this case.
For</tt><br>
<tt>> MPI_Comm_size to have any useful meaning, it needs to return the size
of</tt><br>
<tt>> the communicator: i.e., if comm_size returns N, you should be able to
do a</tt><br>
<tt>> send to processes 0 through N-1. Of course if some of those
processes have</tt><br>
<tt>> failed, you'll get an error associated with the process failure, but
never</tt><br>
<tt>> an error for an invalid rank.</tt><br>
<tt>> ></tt><br>
<tt>> > As discussed in the section about mpiexec, an implementation may
decide</tt><br>
<tt>> to provide a soft process count argument. So "mpiexec -n 10
-soft 5 ./cpi"</tt><br>
<tt>> can start any number between 5 and 10 processes. But that does
not affect</tt><br>
<tt>> the meaning of the size of MPI_COMM_WORLD: regardless of the number of</tt><br>
<tt>> processes the implementation decides to start, MPI_Comm_size will
return</tt><br>
<tt>> the _actual_ number of processes started.</tt><br>
<tt>> ></tt><br>
<tt>> > -d</tt><br>
<tt>> ></tt><br>
<tt>> > On Sep 17, 2010, at 11:22 AM, Graham, Richard L. wrote:</tt><br>
<tt>> ></tt><br>
<tt>> >> We need to clearly define what N or M is and not leave it to
the</tt><br>
<tt>> implementation. 100% of the codes that seen over the past 15
years that</tt><br>
<tt>> check this value use it to indicate how many process have started.
Any</tt><br>
<tt>> thing else is really useless, aside from letting the user find out how
many</tt><br>
<tt>> processes actually started up, and then know how many did not start
up.</tt><br>
<tt>> >></tt><br>
<tt>> >> Rich</tt><br>
<tt>> >></tt><br>
<tt>> >></tt><br>
<tt>> >> On 9/17/10 4:27 AM, "Josh Hursey"
<jjhursey@open-mpi.org> wrote:</tt><br>
<tt>> >></tt><br>
<tt>> >> So the Run-Through Stabilization proposal has been updated per
our</tt><br>
<tt>> discussion in the working group meeting at the MPI Forum. The changes
are</tt><br>
<tt>> summarized below:</tt><br>
<tt>> >> - Add a Legacy Library Support example</tt><br>
<tt>> >> - Clarify new error classes</tt><br>
<tt>> >> - Update the MPI_Init and MPI_Finalize wording to be simpler
and more</tt><br>
<tt>> direct.</tt><br>
<tt>> >> - Fix wording of group creation calls versus communicator
creation</tt><br>
<tt>> calls.</tt><br>
<tt>> >></tt><br>
<tt>> >> </tt></span><a
href="https://BLOCKEDblockedblockedsvn.mpi-forum.org/trac/mpi-forum-"><tt><span
style='font-size:10.0pt'>https://BLOCKEDBLOCKEDBLOCKEDsvn.mpi-forum.org/trac/mpi-forum-</span></tt></a><span
style='font-size:10.0pt;font-family:"Courier New"'><br>
<tt>> web/wiki/ft/run_through_stabilization</tt><br>
<tt>> >></tt><br>
<tt>> >></tt><br>
<tt>> >> One question that we discussed quite a bit during the meeting
was the</tt><br>
<tt>> issue of the return value of MPI_Comm_size() when processes fail
during</tt><br>
<tt>> launch. I attempted to capture the discussion in the room in the Open</tt><br>
<tt>> Question attached to the discussion of MPI_Init:</tt><br>
<tt>> >> </tt></span><a
href="https://BLOCKEDblockedblockedsvn.mpi-forum.org/trac/mpi-forum-"><tt><span
style='font-size:10.0pt'>https://BLOCKEDBLOCKEDBLOCKEDsvn.mpi-forum.org/trac/mpi-forum-</span></tt></a><span
style='font-size:10.0pt;font-family:"Courier New"'><br>
<tt>> web/wiki/ft/run_through_stabilization#MPI_INIT</tt><br>
<tt>> >></tt><br>
<tt>> >> Open question:</tt><br>
<tt>> >> If the user asks to start N processes on the command line,
and only M</tt><br>
<tt>> processes were successfully launched (where M < N), then what
should be</tt><br>
<tt>> returned from MPI_COMM_SIZE?</tt><br>
<tt>> >></tt><br>
<tt>> >> The return value must be consistent across all alive members
of the</tt><br>
<tt>> group. The issue is if it should return N or M.</tt><br>
<tt>> >></tt><br>
<tt>> >> The feeling in the room was that since the MPI standard does
not define</tt><br>
<tt>> the ability for the user to ask for a specific number of processes
before</tt><br>
<tt>> initthen it is hard to define that this is the number it should be.</tt><br>
<tt>> >></tt><br>
<tt>> >> So it is left to the implementation whether it is M or N. If
it is M,</tt><br>
<tt>> then the user has other techniques to find out what it originally
asked for</tt><br>
<tt>> (e.g., by passing it as a command line argument to the application
itself).</tt><br>
<tt>> >></tt><br>
<tt>> >></tt><br>
<tt>> >> What do people think about the MPI_Comm_size issue?</tt><br>
<tt>> >></tt><br>
<tt>> >> -- Josh</tt><br>
<tt>> >></tt><br>
<tt>> >> ------------------------------------</tt><br>
<tt>> >> Joshua Hursey</tt><br>
<tt>> >> Postdoctoral Research Associate</tt><br>
<tt>> >> Oak Ridge National Laboratory</tt><br>
<tt>> >> </tt></span><a
href="http://BLOCKEDblockedblockedwww.BLOCKEDblockedblockedcs.indiana.edu/~jjhursey"><tt><span
style='font-size:10.0pt'>http://BLOCKEDBLOCKEDBLOCKEDwww.BLOCKEDBLOCKEDBLOCKEDcs.indiana.edu/~jjhursey</span></tt></a><span
style='font-size:10.0pt;font-family:"Courier New"'><br>
<tt>> >></tt><br>
<tt>> >></tt><br>
<tt>> >> _______________________________________________</tt><br>
<tt>> >> mpi3-ft mailing list</tt><br>
<tt>> >> mpi3-ft@lists.mpi-forum.org</tt><br>
<tt>> >> </tt></span><a
href="http://BLOCKEDblockedblockedlists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft"><tt><span
style='font-size:10.0pt'>http://BLOCKEDBLOCKEDBLOCKEDlists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft</span></tt></a><span
style='font-size:10.0pt;font-family:"Courier New"'><br>
<tt>> >></tt><br>
<tt>> >></tt><br>
<tt>> >> _______________________________________________</tt><br>
<tt>> >> mpi3-ft mailing list</tt><br>
<tt>> >> mpi3-ft@lists.mpi-forum.org</tt><br>
<tt>> >> </tt></span><a
href="http://BLOCKEDblockedblockedlists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft"><tt><span
style='font-size:10.0pt'>http://BLOCKEDBLOCKEDBLOCKEDlists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft</span></tt></a><span
style='font-size:10.0pt;font-family:"Courier New"'><br>
<tt>> ></tt><br>
<tt>> ></tt><br>
<tt>> > _______________________________________________</tt><br>
<tt>> > mpi3-ft mailing list</tt><br>
<tt>> > mpi3-ft@lists.mpi-forum.org</tt><br>
<tt>> > </tt></span><a
href="http://BLOCKEDblockedblockedlists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft"><tt><span
style='font-size:10.0pt'>http://BLOCKEDBLOCKEDBLOCKEDlists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft</span></tt></a><span
style='font-size:10.0pt;font-family:"Courier New"'><br>
<tt>> ></tt><br>
<tt>> ></tt><br>
<tt>> _______________________________________________</tt><br>
<tt>> mpi3-ft mailing list</tt><br>
<tt>> mpi3-ft@lists.mpi-forum.org</tt><br>
<tt>> </tt></span><a
href="http://BLOCKEDblockedlists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft"><tt><span
style='font-size:10.0pt'>http://BLOCKEDBLOCKEDlists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft</span></tt></a><span
style='font-size:10.0pt;font-family:"Courier New"'><br>
<br>
<br>
<tt>_______________________________________________</tt><br>
<tt>mpi3-ft mailing list</tt><br>
<tt>mpi3-ft@lists.mpi-forum.org</tt><br>
</span><a href="http://BLOCKEDlists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft"><tt><span
style='font-size:10.0pt'>http://BLOCKEDlists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft</span></tt></a><span
style='font-size:10.0pt;font-family:"Courier New"'><br>
<br>
</span><o:p></o:p></p>
</div>
</div>
</body>
</html>