[Mpi3-ft] MPI_Comm_validate parameters

[Darius Buntinas]

On Mar 1, 2011, at 8:30 AM, Joshua Hursey wrote:

>> Hmm.  We could combine validate and get_num_state:
>>   MPI_Comm_validate_global(comm, count_type, &count)
>> This would let the user decide what count to return.
> 
> I think if we let the user define the 'type' here then they may get confused with the interface. Since they could read it as agree upon all processes only in the specified state, instead of agree upon the state of all processes in any state (and returning a flag if there are new failures).

Yeah, I thought about that later.

> I also just noticed that we are missing individual rank accessors (useful when watching one or more critical 'root' ranks, for example):
> ----------------------------
> MPI_Comm_get_state_rank_local(comm, rank, rank_info)
> - Local operation
> - Returns the state of the rank specified within the MPI_Rank_info object as known in "L_i"
> - error if rank is invalid
> 
> MPI_Comm_get_state_rank_global(comm, rank, rank_info)
> - same as MPI_Comm_get_stae_rank_local, but over "G"
> ----------------------------

Yes, these would be useful.

> One side note that we should make explicit is that "L_i" can be updated in two ways. First, when an application calls MPI_comm_validate_local() to update "L_i" with all locally known failures. Secondly, if a failure is detected during any MPI operation (except the validate accessor functions) it will also update "L_i". So if we do a MPI_Send(rank=2), and rank 2 fails during the send then we want to make sure that if the user asks for the state of rank 2 it is identified as failed and not active. The application is implicitly noticing the update to "L_i" from the return code of the MPI_Send operation.
> 
> Do folks see a problem with this additional semantic?

This could present a problem with get_num_state and get_state in a multithreaded environment.  Of course if one has only one thread per communicator, then it's OK, but is that realistic?  The idea of L_i and G being constant between validate calls was to avoid  races like this.

The user should understand that the way to get the state of a process is 
    validate_local();get_state_rank()
or 
    validate_local();get_num_state();malloc();get_state()
And if the process has multiple threads using the same communicator, it can synchronize access to validate_local as appropriate.

I think the _local functions can be considered a convenience for the user, since the user could keep track of L_i herself using the _global values and noticing failed sends/receives.  So if we look at it that way, the fact that get_state doesn't report rank 2 (in your example above) as having failed immediately after the send, might be OK.

-d