[mpiwg-rma] FENCE local completion requirements

[Rajeev Thakur]

> Let me rephrase —
> 
> Is the sentence in the standard to be read as: if there are preceding PUT/GET operations, it *will* be a barrier?
> 
> I don’t think the sentence says that.  I agree that an MPI implementation has to do that, but the standard is not explicit about it.
> 
>  — Pavan

Yes, that's what the standard says. And it says that because the implementation has no other choice if it has to maintain fence semantics.

The standard says

"A fence call usually entails a barrier synchronization: a process completes a call to MPI_WIN_FENCE only after all other processes in the group entered their matching call."

Then it qualifies the "usually" by giving the case where it doesn't act as a barrier:

"However, a call to MPI_WIN_FENCE that is known not to end any epoch (in particular, a call with assert equal to MPI_MODE_NOPRECEDE) does not necessarily act as a barrier."

Which means in all other cases, it acts as a barrier. What are the other cases? Those that are not covered by the above sentence, i.e., those where the fence ends an access or exposure epoch, i.e., where there are RMA operations.

Rajeev



> 
> On Sep 29, 2014, at 10:37 PM, Rajeev Thakur <thakur at mcs.anl.gov> wrote:
> 
>>>>> All RMA operations on win originating at a given process and started before the fence call will complete at that process before the fence call returns.  They will be completed at their target before the fence call returns at the target.
>>>> 
>>>> The intent of this is to avoid an ack at the end of the fence from target to origin. However, since a fence that completes RMA operations is effectively a barrier, when the fence returns on the origin, the origin can be sure that the target has at least called its fence (but may not have exited the fence).
>>>> 
>>>> pg 441, last para: "A fence call usually entails a barrier synchronization: a process completes a call to MPI_WIN_FENCE only after all other processes in the group entered their matching call.  However, a call to MPI_WIN_FENCE that is known not to end any epoch (in particular, a call with assert equal to MPI_MODE_NOPRECEDE) does not necessarily act as a barrier.”
>>> 
>>> Is the above sentence to be read as: if you don’t have MPI_MODE_NOPRECEDE, it *will* be a barrier?  That’s not how I interpret it.  If it does not *guarantee* a barrier, then we still need to do an MPI_BARRIER in the application in the example I gave.
>> 
>> The above sentence reflects what an implementation would need to do in order to support fence semantics, namely, if there are RMA operations preceding a fence, then a process cannot get out of the fence unless all RMA operations targeted to it are completed. And it doesn't know how many RMA operations are targeted to it, and from whom, until every other process has reached its fence. Therefore, every process has to wait for every other process to reach its fence -- effectively a barrier.
>> 
>> The cases where this situation does not occur are where there are guaranteed to be no preceding RMA operations. One such case is the very first fence in the program. Another case is where the user says don't worry there were no RMA operations, by passing an assert. In these cases the implementation of the fence may not act like a barrier.
>> 
>> Whatever an implementation does, it has to support the fence semantics, which are: 
>> 
>> 1. The memory of a target process cannot be accessed until the target has called fence.
>> 2. A process cannot exit from a fence until all RMA operations targeted to it from other processes have completed, and all RMA operations from it to others have locally completed.
>> 
>> Rajeev
>> 
>> _______________________________________________
>> mpiwg-rma mailing list
>> mpiwg-rma at lists.mpi-forum.org
>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma
> 
> --
> Pavan Balaji  ✉️
> http://www.mcs.anl.gov/~balaji
> 
> _______________________________________________
> mpiwg-rma mailing list
> mpiwg-rma at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma