[mpiwg-rma] [EXTERNAL] Re: Synchronization on shared memory windows

[Jeff Hammond]

"For the purposes of synchronizing the private and public window,
MPI_WIN_SYNC has the effect of ending and reopening an access and
exposure epoch on the window (note that it does not actually end an
epoch or complete any pending MPI RMA operations)."

I think this is interpreted to mean that this call is only valid
inside of an existing epoch and thus if you want to call it, you need
to use it inside of a passive-target epoch.  Thus, it is not merely a
portable abstraction for a memory barrier.

I think we should fix MPICH and/or MPI-Next to allow the more general
use such that your code is standard-compliant and executes correctly.

I await violent disagreement from others :-)

Jeff

On Tue, Feb 4, 2014 at 12:34 PM, Rolf Rabenseifner <rabenseifner at hlrs.de> wrote:
> Brian, Pavan, and Jeff,
>
> you convinced me. I did it, see attached file, and my mpich based Cray lib tells
>
> Rank 0 [Tue Feb  4 19:31:28 2014] [c9-1c2s7n0] Fatal error in MPI_Win_sync: Wrong synchronization of RMA calls , error stack:
> MPI_Win_sync(113)...: MPI_Win_sync(win=0xa0000001) failed
> MPIDI_Win_sync(2495): Wrong synchronization of RMA calls
>
> (only once in each process).
>
> I expect, that this is now an implementation bug that should be
> fixed by mpich and cray?
>
> Best regards
> Rolf
>
> ----- Original Message -----
>> From: "Brian W Barrett" <bwbarre at sandia.gov>
>> To: "MPI WG Remote Memory Access working group" <mpiwg-rma at lists.mpi-forum.org>
>> Cc: "Stefan Andersson" <stefan at cray.com>, "Bill Long" <longb at cray.com>
>> Sent: Tuesday, February 4, 2014 7:09:02 PM
>> Subject: Re: [mpiwg-rma] [EXTERNAL] Re: Synchronization on shared memory windows
>>
>> On 2/4/14 11:01 AM, "Rolf Rabenseifner" <rabenseifner at hlrs.de> wrote:
>>
>> >The MPI_WIN_SYNC (not the Fortran MPI_F_SYNC_REG)
>> >has no meaning in the unified memory model if all accesses
>> >are done without RMA routines.
>> >It has only a meaning if different public and privat copy is
>> >there (MPI-3.0 p450:46-p451:2).
>> >MPI-3.0 p456:3 - p457:7 define the rules for the unified memory
>> >model
>> >but there is no need to use MPI_WIN_SYNC.
>>
>> Right, there's no need from an MPI point of view, but that doesn't
>> mean
>> that the language/compiler/processor doesn't have a need for extra
>> synchronization.
>>
>> >The combination of X=13 and MPI_F_SYNC_REG(X)
>> >before MPI_Barrier should guarantee that all bytes of X are
>> >stored in memory. The same should be valid in C,
>> >because the C compiler has no chance to see whether
>> >MPI_Barrier will access the bytes of X or not.
>> >And if it is guaranteed to be in the unified memory,
>> >then the other process (B) should be able to correctly
>> >read the data after the return from its barrier.
>> >
>> >What is wrong with my thinking?
>> >Which detail do I miss?
>>
>> According to my reading of the spec, MPI_F_SYNC_REG only prevents the
>> language/compiler from moving the store, but does not say anything
>> about
>> processor ordering.  So the WIN_SYNC in my last e-mail will add the
>> processor memory barrier, which will give you all the semantics you
>> need.
>>
>> Shared memory programming is a disaster in most languages today, so
>> we
>> decided to pass that disaster on to the user.  We really can't help,
>> without adding lots of overhead (ie, using put/get/rma
>> synchronization).
>> So if a user already knows how to do shared memory programming, this
>> will
>> feel natural.  If they don't, it's going to hurt badly :/.
>>
>>
>> Brian
>>
>> --
>>   Brian W. Barrett
>>   Scalable System Software Group
>>   Sandia National Laboratories
>>
>>
>>
>>
>> _______________________________________________
>> mpiwg-rma mailing list
>> mpiwg-rma at lists.mpi-forum.org
>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma
>>
>
> --
> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de
> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530
> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832
> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner
> Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
>
> _______________________________________________
> mpiwg-rma mailing list
> mpiwg-rma at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma



-- 
Jeff Hammond
jeff.science at gmail.com