[Mpi3-rma] [EXTERNAL] Re: request-based ops

Mon Jun 17 10:38:12 CDT 2013

Hi Brian,

Re: iunlock, ifence --

It might make sense to look at defining MPI_Win_test for these
synchronization modes.  Some of the semantics might already be defined for
us.

 ~Jim.

On Mon, Jun 17, 2013 at 9:44 AM, Barrett, Brian W <bwbarre at sandia.gov>wrote:

> First, I apologize for reply on top; outlook webmail sucks.
>
> I think I agree with Jeff, option #2 is icky.
>
> I think I also agree with Jim that my performance concerns could be
> overstated, as one could make iflush be a flush and return a completed
> request (to be test/waited on later).  There's no locking concerns, as you
> have to have the lock in order to call flush (in my implementation,
> anyway).  But it would limit other implementations which queued rma ops,
> and that would be unfortunate.
>
> Since Pavan's on a make everything non-blocking kick, perhaps it's
> instructive to think about the ifence or iunlock (which I don't like, but I
> think is useful in this conversation).  In both, I think it's a relatively
> clean semantic to not allow communication operations between the operation
> and the test/wait, similar to how we wouldn't allow communication from
> another thread during fence / unlock, as the access epoch closes when the
> synchronization call starts.  In that case, why would we put different
> semantics in for flush?
>
> Or, another way of thinking about it is that for all non-blocking
> communication operations, we don't allow the user to modify state between
> the non-blocking operation and the test/wait.  In most cases, the state is
> the user buffer, but in this case, the state is the operation state.  So
> again, I think there's good precedent for the easy to define, easy to
> understand rule that you can't start new communication operations between
> the iflush and the test/wait on that window, just like you couldn't modify
> the user buffer for isend/ibcast/etc.
>
> Brian
>
> --
>   Brian W. Barrett
>   Scalable System Software Group
>   Sandia National Laboratories
> ________________________________________
> From: mpi3-rma-bounces at lists.mpi-forum.org [
> mpi3-rma-bounces at lists.mpi-forum.org] on behalf of Jeff Hammond [
> jhammond at alcf.anl.gov]
> Sent: Monday, June 17, 2013 8:14 AM
> To: MPI 3.0 Remote Memory Access working group
> Subject: Re: [Mpi3-rma] [EXTERNAL] Re: request-based ops
>
> I do not want to consider interpretation 2 because it is problematic
> for some implementations.  There are cases  where it requires the
> flush to happen when the request is waited upon, in which case there
> is absolutely no benefit over doing a blocking flush.
>
> Actually, now that I think about it, every implementation has to issue
> or re-issue a flush at the time the wait is called if there have been
> an RMA operations issued (to the relevant targets) since the time the
> first fence may have been issued.  Hence, an implementation that
> issues the iflush eagerly may do twice the work whereas one that does
> not is providing zero benefit over the blocking flush.
>
> The only case where 2 is going to be effective and efficient is the
> case where it reduces to 1 because no new RMA operations are issued
> between the iflush and the wait.
>
> Jeff
>
> On Mon, Jun 17, 2013 at 8:52 AM, Jim Dinan <james.dinan at gmail.com> wrote:
> > It seems like there are two possible semantics for which operations are
> > complete by a call to MPI_Win_iflush:
> >
> > (1) Completes all operations issued by the origin process on the given
> > window before MPI_Win_iflush was called.
> > (2) Completes all operations issued by the origin process on the given
> > window before MPI_Win_iflush completed.
> >
> > So far, we've just been looking at #1, but I think that #2 is worth
> > considering.  Option #2 allows an implementation that just checks if the
> > counters are equal.  This avoids the issue where #1 can't be implemented
> in
> > terms of #2, because issuing an unbounded number of operations while
> testing
> > on the iflush request can cause the iflush to not complete indefinitely.
> >
> > Option #2 does not directly provide the functionality that Jeff was
> looking
> > for, but this could be implemented using two windows.  Issue a bunch of
> > operations on win1 and iflush on win2, when win2 has been flushed,
> switch to
> > issuing operations on win2 and iflushing win1.
> >
> >  ~Jim.
> >
> > On Mon, Jun 17, 2013 at 7:38 AM, Jim Dinan <james.dinan at gmail.com>
> wrote:
> >>
> >> Sorry, I should have been more specific.  An implementation of iflush
> that
> >> waits for the completion of all messages should be valid.  Such an
> >> implementation would compare counters and return true if they are the
> same.
> >> This implementation could have the issue I mentioned in the previous
> >> message, where the user continuously issuing operations can prevent
> iflush
> >> from completing.
> >>
> >> Jim.
> >>
> >> On Jun 16, 2013 10:13 AM, "Pavan Balaji" <balaji at mcs.anl.gov> wrote:
> >>>
> >>>
> >>> On 06/16/2013 10:02 AM, Jim Dinan wrote:
> >>>>
> >>>> If the channel is unordered, a message after the iflush can increment
> >>>> the counter, while one before the iflush has not yet completed.  So,
> the
> >>>> counter is not enough to mark a particular point in time.
> >>>
> >>>
> >>> Ah, good point.
> >>>
> >>>> An implementation of iflush as flush should still be valid, right?
>  Just
> >>>
> >>>
> >>> No.  You cannot do this if the user only uses TEST.
> >>>
> >>> MPI_WIN_IFLUSH(&req);
> >>> while (MPI_TEST(req) is not done);
> >>>
> >>>  -- Pavan
> >>>
> >>> --
> >>> Pavan Balaji
> >>> http://www.mcs.anl.gov/~balaji
> >
> >
> >
> > _______________________________________________
> > mpi3-rma mailing list
> > mpi3-rma at lists.mpi-forum.org
> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>
>
>
> --
> Jeff Hammond
> Argonne Leadership Computing Facility
> University of Chicago Computation Institute
> jhammond at alcf.anl.gov / (630) 252-5381
> http://www.linkedin.com/in/jeffhammond
> https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond
> ALCF docs: http://www.alcf.anl.gov/user-guides
> _______________________________________________
> mpi3-rma mailing list
> mpi3-rma at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>
>
> _______________________________________________
> mpi3-rma mailing list
> mpi3-rma at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-rma/attachments/20130617/0fbf05cf/attachment-0001.html>