[Mpi3-rma] [EXTERNAL] Re: Memory barriers in MPI_WIN_LOCK_ALL mode

[Torsten Hoefler]

On Tue, Oct 30, 2012 at 10:44:06AM -0500, Pavan Balaji wrote:
>
> On 10/30/2012 10:13 AM, Jeff Hammond wrote:
>>>> Even in shared memory, the mfence is useless for guaranteeing visibility
>>>> to any other thread/process. For visibility on architectures that reorder
>>>> loads, the P1 must issue a read memory barrier after seeing the hand wave
>>>> and before reading from the window.
>>>
>>> Right, so it's possible that on platforms that require a read barrier
>>> because they reorder loads, the MPI implementation will not be able to
>>> support the unified model.  Or they'll have to have a read barrier before
>>> every get in the unified model or some such thing.
>>
>> Seems perfectly reasonable to have a read barrier before a Get on a
>> processor with a weaker memory model.
>
> That can be expensive to do this for every local GET operation.
Well, some architectures (x86) just enable this by default and take the
cost of the missed opportunities (mostly speculation afaik) for each
memory operation. This may now turn into an endless architecture debate
:-).

> But I think we are converging --
>
> for shared memory, we do agree that you need some sort of memory barrier  
> within the GET operation for all local GETs.  I agree that on x86-like  
> architectures, this might not be needed.
Yes

> for network operations, this is harder unless the network guarantees  
> that the data is placed in the remote memory.  This, for example, is  
> guaranteed by Mellanox adapters, but not by InfiniBand itself.
Yes, I always assumed IB guarantees that but I couldn't cite the page
of the standard. Isn't it that for an RDMA, the CQ entry will only
appear at the source if the message is committed to remote memory?

Best,
  Torsten

-- 
### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ -----
Torsten Hoefler           | Assistant Professor
Dept. of Computer Science | ETH Zürich
Universitätsstrasse 6     | Zurich-8092, Switzerland
CAB E 64.1                | Phone: +41 76 309 79 29