[Mpi3-rma] Updated Proposal 1
manoj at pnl.gov
Wed Dec 8 12:31:11 CST 2010
Jim/Jeff are correct.
It is location consistent and we do fence (in our ORDER Macro) to preserve
location consistency. Fence doesnot happen everytime - only if consequent
data transfers happen to the same location. With get/put, you only need it
when a process does a put and followed by get. In practice, this
doesnot happen often as the source process has already the data in place
(i.e. get is not required).
Manojkumar Krishnan | High Performance Computing | (509) 372-4206
On Tue, 7 Dec 2010, Jeff Hammond wrote:
>>>> Maybe someone familiar with ARMCI on the list can tell us how they
>>>> achieve this on any network? Is there an internal flush before/after
>>>> each GET operation?
>>> Put followed by a get is location consistent: the get will give you
>>> the back value you just put (although I'm not sure how much GA relies
>>> on this behavior).
>> Do you know how this is implemented? Does a GET internally do a FENCE to
>> ensure remote completion of the previous PUTs?
> Yes, the "ORDER" macro (armcip.h) inserts the "FENCE_NODE" macro
> (copy.h) for most cases, which corresponds to fencing the node. I am
> looking at the trunk right now so the rest of you may not be able to
> see the same code, but, other than the directory structure, not much
> in ARMCI has changed recently.
> On BG/P, I have to be quite explicit about synchronization since DCMF
> (wisely) provides the lowest level of consistency (dynamic routing,
> non-blocking, no ordering) by default. This means that if I want good
> bandwidth, I flush before starting every operation, whereas if I want
> good latency, I flush after every operation.
>>> For a get followed by a put, the get operation is blocking and
>>> doesn't return until data is present in the local buffer. So,
>>> corruption by a subsequent put/acc operation is not really possible.
>> I see. Thanks.
> Yes, but Put followed by Get is ordered, so one has to round-trip
> complete the Put for the Get to make sense.
More information about the mpiwg-rma