<html>
<body>
There is one caveat here that we should be aware of. There is not
efficient way to implement this if we want the sender of a message to be
informed that the receiver has been reset because then every send becomes
a send-receive, which will significantly reduce performance. However, if
we're willing to wait until the process receives data from the reset
process either directly or via some dependence through other processes,
then all this can be implemented efficiently. <br><br>
Also, we should keep in mind that for some protocols we need both
piggybacking and non-blocking collectives. The latter is to avoid race
conditions where a process has begun a blocking collective call but needs
to be informed of something having to do with the communication.<br><br>
Greg Bronevetsky<br>
Post-Doctoral Researcher<br>
1028 Building 451<br>
Lawrence Livermore National Lab<br>
(925) 424-5756<br>
bronevetsky1@llnl.gov<br><br>
<blockquote type=cite class=cite cite="">If, as part of ft mpi, some
piggy-back support is provided to the application,<br>
then i don't think this behavior would need to be implemented in the<br>
mpi library.<br><br>
Howard<br><br>
Richard Graham wrote: <br>
<blockquote type=cite class=cite cite=""><font face="Calibri">Can someone
think of a reason to have the library do this over the app ? I can
see that letting the library do this will avoid potential race conditions
that could arise if we let the app do this - basically out of band with
respect to the expected communications traffic.<br>
<br>
Rich<br><br>
<br>
On 10/21/08 11:52 PM, "Thomas Herault"
<<a href="herault.thomas@gmail.htm">herault.thomas@gmail.com</a>>
wrote:<br><br>
</font>
<dl><br><br>
<dd>Le 21 oct. 08 à 22:06, Howard Pritchard a écrit :<br><br>
<dd>> Hello Rich,<br>
<dd>><br>
<dd>> I thought it was also agreed that if process A communicates with
<br>
<dd>> failed process B<br>
<dd>> which had been restarted by another process C, and this was the
<br>
<dd>> first communication<br>
<dd>> from A to B since the restart of B, A would receive the
equivalent <br>
<dd>> of a ECONNRESET error.<br>
<dd>> This was in the context of a case where option 5 below is not
being <br>
<dd>> used by the app.<br>
<dd>><br>
<dd>> Howard<br>
<dd>><br><br>
<dd>Hello Howard,<br><br>
<dd>there was still some discussions about this at the end of the
session.<br><br>
<dd>The argument is that the application could do as well as the library
<br>
<dd>to enforce this detection if this is needed: when a process is <br>
<dd>launched to replace another one, it could define a new
revision/epoch/<br>
<dd>restart number and tag each communication with this number to <br>
<dd>implement the check. If this can be done as efficiently by the <br>
<dd>application as it would be done by the library, asking the
application <br>
<dd>to do it itself would help the library to avoid the additional cost
<br>
<dd>(i.e. piggybacking an integer to each message) when the application
<br>
<dd>does not need that functionality.<br><br>
<dd>It was suggested that the library could provide a generic mean to
<br>
<dd>piggyback this kind of information to each message, in a way similar
<br>
<dd>as what is discussed about piggyback/message logging-based fault
<br>
<dd>tolerance.<br><br>
<dd>Thomas<br><br>
<dd>> Richard Graham wrote:<br>
<dd>>><br>
<dd>>> Here is a summary of what I think that we agreed to
today. Please <br>
<dd>>> correct any errors, and add what I am missing.<br>
<dd>>><br>
<dd>>> • We need to be able to
restore MPI_COMM_WORLD (and it’s <br>
<dd>>> derivatives) to a usable state when a process fails.<br>
<dd>>> • Restoration may involve
having MPI_PROC_NULL replace the lost <br>
<dd>>> process, or may replaced the lost processes with a new
process <br>
<dd>>> (have not specified how this would happen)<br>
<dd>>> • Processes communicating
directly with the failed processes will <br>
<dd>>> be notified via a returned error code about the
failure.<br>
<dd>>> • When a process is notified
of the failure, comm_repair() must be <br>
<dd>>> called. Comm_repair() is not a collective call, and is
what will <br>
<dd>>> initiate the communicator repair associated with the failed
process.<br>
<dd>>> • If a process wants to be
notified of process failure even if it <br>
<dd>>> is not communicating directly with this process, it must
register <br>
<dd>>> for this notification.<br>
<dd>>> • We don’t have enough
information to know how to continue with <br>
<dd>>> support for checkpoint/restart.<br>
<dd>>> • We need to discuss what
needs to do with respect to failure of <br>
<dd>>> collective communications.<br>
<dd>>><br>
<dd>>> There are several issues that came up with respect to these,
which <br>
<dd>>> will be detailed later on.<br>
<dd>>><br>
<dd>>> Rich<br>
<dd>>><br>
<dd>>> _______________________________________________<br>
<dd>>> mpi3-ft mailing list<br>
<dd>>>
<a href="mpi3-ft@lists.mpi-forum.htm">mpi3-ft@lists.mpi-forum.org</a><br>
<dd>>>
<a href="http:///" eudora="autourl">http://</a>
<a href="http:///" eudora="autourl">
lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft</a><br>
<dd>>><br>
<dd>><br>
<dd>><br>
<dd>> --<br>
<dd>><br>
<dd>> Howard Pritchard<br>
<dd>> Cray Inc.<br>
<dd>><br>
<dd>> _______________________________________________<br>
<dd>> mpi3-ft mailing list<br>
<dd>>
<a href="mpi3-ft@lists.mpi-forum.htm">mpi3-ft@lists.mpi-forum.org</a><br>
<dd>>
<a href="http:///" eudora="autourl">http://</a>
<a href="http:///" eudora="autourl">
lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft</a><br><br>
<br>
<dd>_______________________________________________<br>
<dd>mpi3-ft mailing list<br>
<dd><a href="mpi3-ft@lists.mpi-forum.htm">mpi3-ft@lists.mpi-forum.org</a>
<br>
<dd><a href="http:///" eudora="autourl">http://</a>
<a href="http:///" eudora="autourl">
lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft</a><br><br>
</dl><br>
<pre>
_______________________________________________
mpi3-ft mailing list
<a href="mailto:mpi3-ft@lists.mpi-forum.org">
mpi3-ft@lists.mpi-forum.org</a>
<a href="http:///" eudora="autourl">http://</a>
<a href="http:///" eudora="autourl">
lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft</a>
</pre><font face="Courier New, Courier"></font></blockquote><br><br>
<br>
<pre>--
Howard Pritchard
Cray Inc.
</pre><font face="Courier New, Courier"></font>
_______________________________________________<br>
mpi3-ft mailing list<br>
mpi3-ft@lists.mpi-forum.org<br>
<a href="http:///" eudora="autourl">http://</a>
lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-ft</blockquote></body>
</html>