[mpiwg-rma] FENCE local completion requirements

[Rajeev Thakur]

> Now to my next point — barrier synchronization only means that other processes have entered WIN_FENCE, not returned from WIN_FENCE.  This means that they might not have completed all operations at the target.
> 
> If we change my initial example to the following:
> 
> MPI_WIN_FENCE(win1)
> MPI_PUT(win1)
> MPI_WIN_FENCE(win1)
> 
> MPI_WIN_LOCK(win2)
> MPI_GET(win2)
> MPI_WIN_UNLOCK(win2)
> 
> .. the GET might not see the data written by PUT, since the WIN_FENCE only guarantees local completion and barrier semantics of the second FENCE only ensures that the target has “called” FENCE, not “returned" from FENCE.

Yes, that would need explicit synchronization before the lock, and it is covered in the standard: pg 448, ln 11-20.

Rajeev


On Sep 29, 2014, at 11:09 PM, "Balaji, Pavan" <balaji at anl.gov> wrote:

> 
> On Sep 29, 2014, at 11:03 PM, Rajeev Thakur <thakur at mcs.anl.gov> wrote:
> 
>>> Let me rephrase —
>>> 
>>> Is the sentence in the standard to be read as: if there are preceding PUT/GET operations, it *will* be a barrier?
>>> 
>>> I don’t think the sentence says that.  I agree that an MPI implementation has to do that, but the standard is not explicit about it.
>>> 
>>> — Pavan
>> 
>> Yes, that's what the standard says. And it says that because the implementation has no other choice if it has to maintain fence semantics.
>> 
>> The standard says
>> 
>> "A fence call usually entails a barrier synchronization: a process completes a call to MPI_WIN_FENCE only after all other processes in the group entered their matching call."
>> 
>> Then it qualifies the "usually" by giving the case where it doesn't act as a barrier:
>> 
>> "However, a call to MPI_WIN_FENCE that is known not to end any epoch (in particular, a call with assert equal to MPI_MODE_NOPRECEDE) does not necessarily act as a barrier."
>> 
>> Which means in all other cases, it acts as a barrier.
> 
> The right way to phrase that would have been:
> 
> A call to MPI_WIN_FENCE that ends an epoch entails a barrier synchronization: a process completes a call to MPI_WIN_FENCE only after all other processes in the group entered their matching call.  A call to MPI_WIN_FENCE that is known not to end any epoch (in particular, a call with assert equal to MPI_MODE_NOPRECEDE) does not necessarily act as a barrier.
> 
> If this was the intention, it should be fixed in the standard.
> 
> Now to my next point — barrier synchronization only means that other processes have entered WIN_FENCE, not returned from WIN_FENCE.  This means that they might not have completed all operations at the target.
> 
> If we change my initial example to the following:
> 
> MPI_WIN_FENCE(win1)
> MPI_PUT(win1)
> MPI_WIN_FENCE(win1)
> 
> MPI_WIN_LOCK(win2)
> MPI_GET(win2)
> MPI_WIN_UNLOCK(win2)
> 
> .. the GET might not see the data written by PUT, since the WIN_FENCE only guarantees local completion and barrier semantics of the second FENCE only ensures that the target has “called” FENCE, not “returned" from FENCE.
> 
>  — Pavan
> 
> --
> Pavan Balaji  ✉️
> http://www.mcs.anl.gov/~balaji
> 
> _______________________________________________
> mpiwg-rma mailing list
> mpiwg-rma at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma