[Mpi3-ft] simplified FT proposal

[William Gropp]

One concern that I have with fault tolerant proposals has to do with races in the specification.  This is an area where users often "just want it to work" but getting it right is tricky.  In the example here, the "alive_group" is really only that at some moment shortly before "MPI_Comm_check" returns (and possibly not even that).  After that, it is really the "group_of_processes_that_was_alive_at_some_point_in_the_past".  Since there are sometimes correlations in failures, this could happen even if the initial failure is rare.  An alternate form might be to have a routine, collective over a communicator, that returns a new communicator meeting some definition of "members were alive at some point during construction".  It wouldn't guarantee you could use it, but it would have cleaner semantics.

Bill

On Jan 13, 2012, at 3:41 PM, Sur, Sayantan wrote:

> I would like to argue for a simplified version of the proposal that covers a large percentage of use-cases and resists adding new “features” for the full-range of ABFT techniques. It is good if we have a more pragmatic view and not sacrifice the entire FT proposal for the 1% fringe cases. Most apps just want to do something like this:
>  
> for(… really long time …) {
>    MPI_Comm_check(work_comm, &is_ok, &alive_group);
>    if(!is_ok) {
>        MPI_Comm_create_group(alive_group, …, &new_comm);
>       // re-balance workload and use new_comm in rest of computation
>        MPI_Comm_free(work_comm); // get rid of old comm
>        work_comm = new_comm;
>    } else {
>      // continue computation using work_comm
>      // if some proc failed in this iteration, roll back work done in this iteration, go back to loop
>    }
> }
>  

William Gropp
Director, Parallel Computing Institute
Deputy Director for Research
Institute for Advanced Computing Applications and Technologies
Paul and Cynthia Saylor Professor of Computer Science
University of Illinois Urbana-Champaign



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-ft/attachments/20120115/13f57c23/attachment-0001.html>