[Mpi3-rma] Memory barriers in MPI_WIN_LOCK_ALL mode

Torsten Hoefler htor at illinois.edu
Tue Oct 30 08:54:51 CDT 2012

On Tue, Oct 30, 2012 at 09:09:15AM -0500, Pavan Balaji wrote:
> On 10/30/2012 08:26 AM, Torsten Hoefler wrote:
>>> Actually, the flush only needs to block till its visible to another MPI
>>> RMA operation, not to the CPU (as in, not to load/store).
>> In the unified memory model, it has to guarantee visibility to
>> load/store operations as well.
> How will the origin process guarantee this without a WIN_SYNC?
Well, this depends on the architecture. On x86, for example, the origin
could issue an mfence which would guarantee global ordering (sequential
consistency for this statement). The "hand waiving" (barrier or whatnot)
would then be ordered after that and cannot pass the store at the

>>> I don't think  the target process can guarantee that the PUT is
>>> visible to a load/store  without an additional memory barrier.
>> The flush of the source process has to ensure that.
> How?
See above, mfence.

>>> Since in the MPI standard, we  don't specify that the user needs to
>>> call a WIN_SYNC in this case, I'm  asking if the MPI implementation
>>> needs to do a memory barrier internally.
>> Yes, win_synch is only needed in the separate model, which would then do
>> a memory barrier. The MPI library has to guarantee that there is no
>> inconsistency between public and private copy in the unified model (or
>> it cannot claim that it supports unified).
> I'm not sure any architecture can do this without internally adding  
> memory barriers in a lot of MPI operations other than the usual  
> FLUSH/SYNC/UNLOCK type of operations.
I think this would be possible, given the loose consistency we specify.

> Note that the MPI standard only says that in the UNIFIED model, the data  
> will "eventually" appear (pg. 454, bullet item 6).  No guarantees that  
> the data actually is visible for load operations immediately after a 
> flush.
Yes, I realized that while trying to formalize the memory model. This
contradicts other statements like "public and private copies are
identical" (p. 436) in conjunction with "the operation is completed at
the target [by a call to win_flush]" (p. 454). I think we should
reconsider those "eventual" clauses because, if we assume that there is
no happens-before consistency guarantee, and instead just a weak "in
some time" statement, then we could never use the unified model for
anything correct (because "eventually" may as well be in 10^16 years).
While, if we remove this eventual statements, the semantics make sense
and seem implementable.

I have this on a longer list of issues to bring up at the next forum


### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ -----
Torsten Hoefler           | Assistant Professor
Dept. of Computer Science | ETH Zürich
Universitätsstrasse 6     | Zurich-8092, Switzerland
CAB E 64.1                | Phone: +41 76 309 79 29

More information about the mpiwg-rma mailing list