[Mpi3-rma] Updated Proposal 1

[Jeff Hammond]

Pavan,

>>> Maybe someone familiar with ARMCI on the list can tell us how they
>>> achieve this on any network? Is there an internal flush before/after
>>> each GET operation?
>>
>> Put followed by a get is location consistent: the get will give you
>> the back value you just put (although I'm not sure how much GA relies
>> on this behavior).

> Do you know how this is implemented? Does a GET internally do a FENCE to
> ensure remote completion of the previous PUTs?

Yes, the "ORDER" macro (armcip.h) inserts the "FENCE_NODE" macro
(copy.h) for most cases, which corresponds to fencing the node.  I am
looking at the trunk right now so the rest of you may not be able to
see the same code, but, other than the directory structure, not much
in ARMCI has changed recently.

On BG/P, I have to be quite explicit about synchronization since DCMF
(wisely) provides the lowest level of consistency (dynamic routing,
non-blocking, no ordering) by default.  This means that if I want good
bandwidth, I flush before starting every operation, whereas if I want
good latency, I flush after every operation.

>> For a get followed by a put, the get operation is blocking and
>> doesn't return until data is present in the local buffer. So,
>> corruption by a subsequent put/acc operation is not really possible.
>
> I see. Thanks.

Yes, but Put followed by Get is ordered, so one has to round-trip
complete the Put for the Get to make sense.

Jeff


-- 
Jeff Hammond
Argonne Leadership Computing Facility
jhammond at alcf.anl.gov / (630) 252-5381
http://www.linkedin.com/in/jeffhammond