[mpi3-coll] Non-blocking Collectives Proposal Draft

Richard Graham rlgraham at ornl.gov
Fri Oct 17 08:41:58 CDT 2008


Alexander,
  I am not sure what the really trying to get after.  The example I gave is
exactly what I intended for it to be, not a complete working MPI example,
but the core as it relates to progress.  The example that Torsten gave
should progress and complete, whether one uses asynchronous progress
mechanisms or the synchronous approach most of us take today.  If the
question is about the semantics of MPI_Ibarrier(), I think the only
specification is that it starts when the ibarrier is posted (what ever that
means, which I am sure is implementation specific) and ends with the wait ­
I don¹t think that there is any other specification here.
  Have I completely missed what your question is ?

Rich


On 10/17/08 8:51 AM, "Supalov, Alexander" <alexander.supalov at intel.com>
wrote:

> Dear Rich,
> 
> Thanks. You probably meant Recv(0) in Proc 1. I also miss a Wait for
> req_r. So, a complete example would probably look like:
> 
> Proc 0                  Proc 1
> 
> Irecv(1,req_r)          Irecv(0,req_r)
> Isend(1,req_s)          Isend(0,req_s)
> Waitall(req_s, req_r)   Recv(0)
> Send(1)                 Waitall(req_s,req_r)
> 
> This would work. However, let's do one change, replacing Isend by
> Issend. Will this still work?
> 
> Irecv(1,req_r)          Irecv(0,req_r)
> Issend(1,req_s)         Issend(0,req_s)
> Waitall(req_s, req_r)   Recv(0)
> Send(1)                 Waitall(req_s,req_r)
> 
> Another possible complication: let's swap the Send and Recv, and make
> Send synchronous (or rendezvous):
> 
> Irecv(1,req_r)          Irecv(0,req_r)
> Issend(1,req_s)         Issend(0,req_s)
> Waitall(req_s, req_r)   Ssend(0)
> Recv(1)                 Waitall(req_s,req_r)
> 
> Here I have rather strong doubts that this will work. Do you?
> 
> Best regards.
> 
> Alexander
> 
> -----Original Message-----
> From: mpi3-coll-bounces at lists.mpi-forum.org
> [mailto:mpi3-coll-bounces at lists.mpi-forum.org] On Behalf Of Graham,
> Richard L.
> Sent: Friday, October 17, 2008 2:35 PM
> To: mpi3-coll at lists.mpi-forum.org
> Subject: Re: [mpi3-coll] Non-blocking Collectives Proposal Draft
> 
> Alexander,
>   Do you see this as different in principal from
> 
> Proc 0.            Proc 1
> irecv(1,req_r)     Irecv(0,req_r)
> Isend(1,req_s).   Isend(0,req_s)
> Wait(req_s).      Send(0)
> Send(1).           Wait(req_s)
> 
> I see these two cases as basically the same from a progress perspective.
> 
> Rich
> 
> ----- Original Message -----
> From: mpi3-coll-bounces at lists.mpi-forum.org
> <mpi3-coll-bounces at lists.mpi-forum.org>
> To: MPI-3 Collective Subgroup Discussions
> <mpi3-coll at lists.mpi-forum.org>
> Sent: Fri Oct 17 07:22:02 2008
> Subject: Re: [mpi3-coll] Non-blocking Collectives Proposal Draft
> 
> Dear Rich,
> 
> Thank you. Note that the first operation is an Ibarrier. According to
> Torsten's explanation, process 0 will block in the Wait, which it may
> well do if it is Wait in process 1 that actually completes its Ibarrier.
> 
> Now, how can process 0 make progress in its Wait when there's no
> matching Wait on the other side yet? Do we assume that any messages that
> may have been sent by process 0's IBarrier will be handled as unexpected
> messages by process 1 that is busy doing its (also unmatched) part of
> the Bcast?
> 
> Likewise, do we assume that all messages that process 1 has to send in
> its Ibarrier can leave process 1 before it enters its Wait? Aren't we
> assuming a bit too much? In other words, aren't we assuming that all
> message sends and receives will be started in Ibarrier, and will be able
> to proceed even though the Wait is some blocking collective ops away?
> 
> Given the way most MPIs implement nonblocking pt2pt ops now, something
> looks fishy here. In any case, I'd like to see a detailed explanation
> how things will work, and moreover, a proof that this does not preclude
> some algorithms - both in the sense of the pt2pt operations deployed,
> and in the sense of collective exchange pattern, for the applicable
> message sizes - from being used.
> 
> Best regards.
> 
> Alexander
> 
> -----Original Message-----
> From: mpi3-coll-bounces at lists.mpi-forum.org
> [mailto:mpi3-coll-bounces at lists.mpi-forum.org] On Behalf Of Graham,
> Richard L.
> Sent: Friday, October 17, 2008 11:57 AM
> To: mpi3-coll at lists.mpi-forum.org
> Subject: Re: [mpi3-coll] Non-blocking Collectives Proposal Draft
> 
> To me this does not look like a deadlock, proc o will make progress in
> the wait, and just like with point-to-point commuications process 1 will
> make general mpi progress while in the bcast.  This includes progress on
> the ibcast so that process 0 can comete it's ibcast and move on to the
> bcast.
> 
> Rich
> 
> ----- Original Message -----
> From: mpi3-coll-bounces at lists.mpi-forum.org
> <mpi3-coll-bounces at lists.mpi-forum.org>
> To: MPI-3 Collective Subgroup Discussions
> <mpi3-coll at lists.mpi-forum.org>
> Sent: Fri Oct 17 05:14:04 2008
> Subject: Re: [mpi3-coll] Non-blocking Collectives Proposal Draft
> 
> Dear Torsten,
> 
> Thanks. I'm still unsure I understand how the example would work if
> processes blocked in the Wait calls as you said below. In this case
> process 0 would have no chance to reach the Bcast, while process 1 would
> be inside it and never reached the Wait.
> 
> Here again is the original diagram:
> 
> Process 0                       Process 1
> 
> MPI_Ibarrier(req)               MPI_Ibarrier(req)
> MPI_Wait(req)*
> MPI Bcast                       MPI_Bcast*
>                                 MPI_Wait(req)
> 
> The stars mark the point where the respective processes would block, as
> far as I understand. Looks like deadlock?
> 
> Apart from this, stating that the nonblocking collectives can block in
> Wait presumes a certain implementation. What if Ibarrier was able to do
> all right away, and Wait were just a no-op?
> 
> I think mixing blocking and nonblocking collectives in the proposed way
> should either be very carefully considered, or simply disallowed.
> 
> Best regards.
> 
> Alexander
> 
> -----Original Message-----
> From: mpi3-coll-bounces at lists.mpi-forum.org
> [mailto:mpi3-coll-bounces at lists.mpi-forum.org] On Behalf Of Torsten
> Hoefler
> Sent: Thursday, October 16, 2008 11:15 PM
> To: MPI-3 Collective Subgroup Discussions
> Subject: Re: [mpi3-coll] Non-blocking Collectives Proposal Draft
> 
> Hello Alexander,
> 
>> > Thank you. The pt2pt progress comment below was taken into account
> when
>> > you removed the nonblocking collective progress description.
> ah, ok - so I think a reference to the chapter should be sufficient.
> 
>> > As for the example, if it is legal, we should probably count the
>> > blocking operations as well when we decide how many simultaneous ops
> are
>> > to be supported.
> yes, maybe. I am not sure but this could be an issue. It's on the agenda
> https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/forum201008
> 
>> > Why "if" above: I'm not sure what a delayed nonblocking barrier would
>> > mean for process 1. Won't one process possibly block somewhere? Can
> you
>> > explain this please?
> the barriers will only block in the respective MPI_Wait() (or a tight
> MPI_Test() loop). But the operation can be initiated as soon as the
> critical region is left (and there mihgt be some other independent
> computation to do).
> 
>> > The format matter is complicated. I think it is worth discussing at
> the
>> > Forum as we're likely to have more and more proposals approaching the
>> > final stage, when editing them in PDF format will become an issue.
> yes, it's on the WG agenda - maybe we should discuss this in the whole
> forum?
> 
> Best,
>   Torsten
> 
> --
>  bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ -----
> Torsten Hoefler       | Postdoctoral Researcher
> Open Systems Lab      | Indiana University
> 150 S. Woodlawn Ave.  | Bloomington, IN, 474045, USA
> Lindley Hall Room 135 | +01 (812) 855-3608
> _______________________________________________
> mpi3-coll mailing list
> mpi3-coll at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll
> ---------------------------------------------------------------------
> Intel GmbH
> Dornacher Strasse 1
> 85622 Feldkirchen/Muenchen Germany
> Sitz der Gesellschaft: Feldkirchen bei Muenchen
> Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
> Registergericht: Muenchen HRB 47456 Ust.-IdNr.
> VAT Registration No.: DE129385895
> Citibank Frankfurt (BLZ 502 109 00) 600119052
> 
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
> 
> 
> _______________________________________________
> mpi3-coll mailing list
> mpi3-coll at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll
> 
> _______________________________________________
> mpi3-coll mailing list
> mpi3-coll at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll
> ---------------------------------------------------------------------
> Intel GmbH
> Dornacher Strasse 1
> 85622 Feldkirchen/Muenchen Germany
> Sitz der Gesellschaft: Feldkirchen bei Muenchen
> Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
> Registergericht: Muenchen HRB 47456 Ust.-IdNr.
> VAT Registration No.: DE129385895
> Citibank Frankfurt (BLZ 502 109 00) 600119052
> 
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
> 
> 
> _______________________________________________
> mpi3-coll mailing list
> mpi3-coll at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll
> 
> _______________________________________________
> mpi3-coll mailing list
> mpi3-coll at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll
> ---------------------------------------------------------------------
> Intel GmbH
> Dornacher Strasse 1
> 85622 Feldkirchen/Muenchen Germany
> Sitz der Gesellschaft: Feldkirchen bei Muenchen
> Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
> Registergericht: Muenchen HRB 47456 Ust.-IdNr.
> VAT Registration No.: DE129385895
> Citibank Frankfurt (BLZ 502 109 00) 600119052
> 
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
> 
> 
> _______________________________________________
> mpi3-coll mailing list
> mpi3-coll at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-coll/attachments/20081017/9440de8b/attachment-0001.html>


More information about the mpiwg-coll mailing list