<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD><TITLE>Re: [mpi3-coll] Non-blocking Collectives Proposal Draft</TITLE>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.2900.3354" name=GENERATOR></HEAD>
<BODY>
<DIV dir=ltr align=left><SPAN class=385321215-17102008><FONT face=Arial
size=2>Dear Rich,</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=385321215-17102008><FONT face=Arial
size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=385321215-17102008><FONT face=Arial
size=2>Thank you. I'm basically after making sure that we don't design something
that does not work. From this discussion, it appears that the existing progress
rules do cover the proposal well enough to proceed.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=385321215-17102008><FONT face=Arial
size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=385321215-17102008><FONT face=Arial
size=2>Best regards.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=385321215-17102008><FONT face=Arial
size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=385321215-17102008><FONT face=Arial
size=2>Alexander</FONT></SPAN></DIV><BR>
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> mpi3-coll-bounces@lists.mpi-forum.org
[mailto:mpi3-coll-bounces@lists.mpi-forum.org] <B>On Behalf Of </B>Richard
Graham<BR><B>Sent:</B> Friday, October 17, 2008 3:42 PM<BR><B>To:</B> MPI-3
Collective Subgroup Discussions<BR><B>Subject:</B> Re: [mpi3-coll] Non-blocking
Collectives Proposal Draft<BR></FONT><BR></DIV>
<DIV></DIV><FONT face="Calibri, Verdana, Helvetica, Arial"><SPAN
style="FONT-SIZE: 11pt">Alexander,<BR> I am not sure what the really
trying to get after. The example I gave is exactly what I intended for it
to be, not a complete working MPI example, but the core as it relates to
progress. The example that Torsten gave should progress and complete,
whether one uses asynchronous progress mechanisms or the synchronous approach
most of us take today. If the question is about the semantics of
MPI_Ibarrier(), I think the only specification is that it starts when the
ibarrier is posted (what ever that means, which I am sure is implementation
specific) and ends with the wait – I don’t think that there is any other
specification here.<BR> Have I completely missed what your question
is ?<BR><BR>Rich<BR><BR><BR>On 10/17/08 8:51 AM, "Supalov, Alexander" <<A
href="alexander.supalov@intel.com">alexander.supalov@intel.com</A>>
wrote:<BR><BR></SPAN></FONT>
<BLOCKQUOTE><FONT face="Calibri, Verdana, Helvetica, Arial"><SPAN
style="FONT-SIZE: 11pt">Dear Rich,<BR><BR>Thanks. You probably meant Recv(0)
in Proc 1. I also miss a Wait for<BR>req_r. So, a complete example would
probably look like:<BR><BR>Proc 0
Proc
1<BR><BR>Irecv(1,req_r)
Irecv(0,req_r)<BR>Isend(1,req_s)
Isend(0,req_s)<BR>Waitall(req_s,
req_r) Recv(0)<BR>Send(1)
Waitall(req_s,req_r)<BR><BR>This
would work. However, let's do one change, replacing Isend by<BR>Issend. Will
this still work?<BR><BR>Irecv(1,req_r)
Irecv(0,req_r)<BR>Issend(1,req_s)
Issend(0,req_s)<BR>Waitall(req_s,
req_r) Recv(0)<BR>Send(1)
Waitall(req_s,req_r)<BR><BR>Another
possible complication: let's swap the Send and Recv, and make<BR>Send
synchronous (or rendezvous):<BR><BR>Irecv(1,req_r)
Irecv(0,req_r)<BR>Issend(1,req_s)
Issend(0,req_s)<BR>Waitall(req_s,
req_r) Ssend(0)<BR>Recv(1)
Waitall(req_s,req_r)<BR><BR>Here
I have rather strong doubts that this will work. Do you?<BR><BR>Best
regards.<BR><BR>Alexander<BR><BR>-----Original Message-----<BR>From: <A
href="mpi3-coll-bounces@lists.mpi-forum.org">mpi3-coll-bounces@lists.mpi-forum.org</A><BR>[<A
href="mailto:mpi3-coll-bounces@lists.mpi-forum.org">mailto:mpi3-coll-bounces@lists.mpi-forum.org</A>]
On Behalf Of Graham,<BR>Richard L.<BR>Sent: Friday, October 17, 2008 2:35
PM<BR>To: <A
href="mpi3-coll@lists.mpi-forum.org">mpi3-coll@lists.mpi-forum.org</A><BR>Subject:
Re: [mpi3-coll] Non-blocking Collectives Proposal
Draft<BR><BR>Alexander,<BR> Do you see this as different in
principal from<BR><BR>Proc 0.
Proc
1<BR>irecv(1,req_r) Irecv(0,req_r)<BR>Isend(1,req_s).
Isend(0,req_s)<BR>Wait(req_s).
Send(0)<BR>Send(1).
Wait(req_s)<BR><BR>I
see these two cases as basically the same from a progress
perspective.<BR><BR>Rich<BR><BR>----- Original Message -----<BR>From: <A
href="mpi3-coll-bounces@lists.mpi-forum.org">mpi3-coll-bounces@lists.mpi-forum.org</A><BR><<A
href="mpi3-coll-bounces@lists.mpi-forum.org">mpi3-coll-bounces@lists.mpi-forum.org</A>><BR>To:
MPI-3 Collective Subgroup Discussions<BR><<A
href="mpi3-coll@lists.mpi-forum.org">mpi3-coll@lists.mpi-forum.org</A>><BR>Sent:
Fri Oct 17 07:22:02 2008<BR>Subject: Re: [mpi3-coll] Non-blocking Collectives
Proposal Draft<BR><BR>Dear Rich,<BR><BR>Thank you. Note that the first
operation is an Ibarrier. According to<BR>Torsten's explanation, process 0
will block in the Wait, which it may<BR>well do if it is Wait in process 1
that actually completes its Ibarrier.<BR><BR>Now, how can process 0 make
progress in its Wait when there's no<BR>matching Wait on the other side yet?
Do we assume that any messages that<BR>may have been sent by process 0's
IBarrier will be handled as unexpected<BR>messages by process 1 that is busy
doing its (also unmatched) part of<BR>the Bcast?<BR><BR>Likewise, do we assume
that all messages that process 1 has to send in<BR>its Ibarrier can leave
process 1 before it enters its Wait? Aren't we<BR>assuming a bit too much? In
other words, aren't we assuming that all<BR>message sends and receives will be
started in Ibarrier, and will be able<BR>to proceed even though the Wait is
some blocking collective ops away?<BR><BR>Given the way most MPIs implement
nonblocking pt2pt ops now, something<BR>looks fishy here. In any case, I'd
like to see a detailed explanation<BR>how things will work, and moreover, a
proof that this does not preclude<BR>some algorithms - both in the sense of
the pt2pt operations deployed,<BR>and in the sense of collective exchange
pattern, for the applicable<BR>message sizes - from being used.<BR><BR>Best
regards.<BR><BR>Alexander<BR><BR>-----Original Message-----<BR>From: <A
href="mpi3-coll-bounces@lists.mpi-forum.org">mpi3-coll-bounces@lists.mpi-forum.org</A><BR>[<A
href="mailto:mpi3-coll-bounces@lists.mpi-forum.org">mailto:mpi3-coll-bounces@lists.mpi-forum.org</A>]
On Behalf Of Graham,<BR>Richard L.<BR>Sent: Friday, October 17, 2008 11:57
AM<BR>To: <A
href="mpi3-coll@lists.mpi-forum.org">mpi3-coll@lists.mpi-forum.org</A><BR>Subject:
Re: [mpi3-coll] Non-blocking Collectives Proposal Draft<BR><BR>To me this does
not look like a deadlock, proc o will make progress in<BR>the wait, and just
like with point-to-point commuications process 1 will<BR>make general mpi
progress while in the bcast. This includes progress on<BR>the ibcast so
that process 0 can comete it's ibcast and move on to
the<BR>bcast.<BR><BR>Rich<BR><BR>----- Original Message -----<BR>From: <A
href="mpi3-coll-bounces@lists.mpi-forum.org">mpi3-coll-bounces@lists.mpi-forum.org</A><BR><<A
href="mpi3-coll-bounces@lists.mpi-forum.org">mpi3-coll-bounces@lists.mpi-forum.org</A>><BR>To:
MPI-3 Collective Subgroup Discussions<BR><<A
href="mpi3-coll@lists.mpi-forum.org">mpi3-coll@lists.mpi-forum.org</A>><BR>Sent:
Fri Oct 17 05:14:04 2008<BR>Subject: Re: [mpi3-coll] Non-blocking Collectives
Proposal Draft<BR><BR>Dear Torsten,<BR><BR>Thanks. I'm still unsure I
understand how the example would work if<BR>processes blocked in the Wait
calls as you said below. In this case<BR>process 0 would have no chance to
reach the Bcast, while process 1 would<BR>be inside it and never reached the
Wait.<BR><BR>Here again is the original diagram:<BR><BR>Process 0
Process
1<BR><BR>MPI_Ibarrier(req)
MPI_Ibarrier(req)<BR>MPI_Wait(req)*<BR>MPI
Bcast
MPI_Bcast*<BR> MPI_Wait(req)<BR><BR>The
stars mark the point where the respective processes would block, as<BR>far as
I understand. Looks like deadlock?<BR><BR>Apart from this, stating that the
nonblocking collectives can block in<BR>Wait presumes a certain
implementation. What if Ibarrier was able to do<BR>all right away, and Wait
were just a no-op?<BR><BR>I think mixing blocking and nonblocking collectives
in the proposed way<BR>should either be very carefully considered, or simply
disallowed.<BR><BR>Best regards.<BR><BR>Alexander<BR><BR>-----Original
Message-----<BR>From: <A
href="mpi3-coll-bounces@lists.mpi-forum.org">mpi3-coll-bounces@lists.mpi-forum.org</A><BR>[<A
href="mailto:mpi3-coll-bounces@lists.mpi-forum.org">mailto:mpi3-coll-bounces@lists.mpi-forum.org</A>]
On Behalf Of Torsten<BR>Hoefler<BR>Sent: Thursday, October 16, 2008 11:15
PM<BR>To: MPI-3 Collective Subgroup Discussions<BR>Subject: Re: [mpi3-coll]
Non-blocking Collectives Proposal Draft<BR><BR>Hello Alexander,<BR><BR>>
Thank you. The pt2pt progress comment below was taken into
account<BR>when<BR>> you removed the nonblocking collective progress
description.<BR>ah, ok - so I think a reference to the chapter should be
sufficient.<BR><BR>> As for the example, if it is legal, we should probably
count the<BR>> blocking operations as well when we decide how many
simultaneous ops<BR>are<BR>> to be supported.<BR>yes, maybe. I am not sure
but this could be an issue. It's on the agenda<BR><A
href="https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/forum201008">https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/forum201008</A><BR><BR>>
Why "if" above: I'm not sure what a delayed nonblocking barrier would<BR>>
mean for process 1. Won't one process possibly block somewhere?
Can<BR>you<BR>> explain this please?<BR>the barriers will only block in the
respective MPI_Wait() (or a tight<BR>MPI_Test() loop). But the operation can
be initiated as soon as the<BR>critical region is left (and there mihgt be
some other independent<BR>computation to do).<BR><BR>> The format matter is
complicated. I think it is worth discussing at<BR>the<BR>> Forum as we're
likely to have more and more proposals approaching the<BR>> final stage,
when editing them in PDF format will become an issue.<BR>yes, it's on the WG
agenda - maybe we should discuss this in the
whole<BR>forum?<BR><BR>Best,<BR> Torsten<BR><BR>--<BR> bash$
:(){ :|:&};: --------------------- <A
href="http://www.unixer.de/">http://www.unixer.de/</A> -----<BR>Torsten
Hoefler | Postdoctoral Researcher<BR>Open
Systems Lab | Indiana University
<BR>150 S. Woodlawn Ave. | Bloomington, IN, 474045,
USA<BR>Lindley Hall Room 135 | +01 (812)
855-3608<BR>_______________________________________________<BR>mpi3-coll
mailing list<BR><A
href="mpi3-coll@lists.mpi-forum.org">mpi3-coll@lists.mpi-forum.org</A><BR><A
href="http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll">http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll</A><BR>---------------------------------------------------------------------<BR>Intel
GmbH<BR>Dornacher Strasse 1<BR>85622 Feldkirchen/Muenchen Germany<BR>Sitz der
Gesellschaft: Feldkirchen bei Muenchen<BR>Geschaeftsfuehrer: Douglas Lusk,
Peter Gleissner, Hannes Schwaderer<BR>Registergericht: Muenchen HRB 47456
Ust.-IdNr.<BR>VAT Registration No.: DE129385895<BR>Citibank Frankfurt (BLZ 502
109 00) 600119052<BR><BR>This e-mail and any attachments may contain
confidential material for<BR>the sole use of the intended recipient(s). Any
review or distribution<BR>by others is strictly prohibited. If you are not the
intended<BR>recipient, please contact the sender and delete all
copies.<BR><BR><BR>_______________________________________________<BR>mpi3-coll
mailing list<BR><A
href="mpi3-coll@lists.mpi-forum.org">mpi3-coll@lists.mpi-forum.org</A><BR><A
href="http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll">http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll</A><BR><BR>_______________________________________________<BR>mpi3-coll
mailing list<BR><A
href="mpi3-coll@lists.mpi-forum.org">mpi3-coll@lists.mpi-forum.org</A><BR><A
href="http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll">http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll</A><BR>---------------------------------------------------------------------<BR>Intel
GmbH<BR>Dornacher Strasse 1<BR>85622 Feldkirchen/Muenchen Germany<BR>Sitz der
Gesellschaft: Feldkirchen bei Muenchen<BR>Geschaeftsfuehrer: Douglas Lusk,
Peter Gleissner, Hannes Schwaderer<BR>Registergericht: Muenchen HRB 47456
Ust.-IdNr.<BR>VAT Registration No.: DE129385895<BR>Citibank Frankfurt (BLZ 502
109 00) 600119052<BR><BR>This e-mail and any attachments may contain
confidential material for<BR>the sole use of the intended recipient(s). Any
review or distribution<BR>by others is strictly prohibited. If you are not the
intended<BR>recipient, please contact the sender and delete all
copies.<BR><BR><BR>_______________________________________________<BR>mpi3-coll
mailing list<BR><A
href="mpi3-coll@lists.mpi-forum.org">mpi3-coll@lists.mpi-forum.org</A><BR><A
href="http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll">http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll</A><BR><BR>_______________________________________________<BR>mpi3-coll
mailing list<BR><A
href="mpi3-coll@lists.mpi-forum.org">mpi3-coll@lists.mpi-forum.org</A><BR><A
href="http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll">http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll</A><BR>---------------------------------------------------------------------<BR>Intel
GmbH<BR>Dornacher Strasse 1<BR>85622 Feldkirchen/Muenchen Germany<BR>Sitz der
Gesellschaft: Feldkirchen bei Muenchen<BR>Geschaeftsfuehrer: Douglas Lusk,
Peter Gleissner, Hannes Schwaderer<BR>Registergericht: Muenchen HRB 47456
Ust.-IdNr.<BR>VAT Registration No.: DE129385895<BR>Citibank Frankfurt (BLZ 502
109 00) 600119052<BR><BR>This e-mail and any attachments may contain
confidential material for<BR>the sole use of the intended recipient(s). Any
review or distribution<BR>by others is strictly prohibited. If you are not the
intended<BR>recipient, please contact the sender and delete all
copies.<BR><BR><BR>_______________________________________________<BR>mpi3-coll
mailing list<BR><A
href="mpi3-coll@lists.mpi-forum.org">mpi3-coll@lists.mpi-forum.org</A><BR><A
href="http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll">http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-coll</A><BR><BR></SPAN></FONT></BLOCKQUOTE><pre>---------------------------------------------------------------------
Intel GmbH
Dornacher Strasse 1
85622 Feldkirchen/Muenchen Germany
Sitz der Gesellschaft: Feldkirchen bei Muenchen
Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
Registergericht: Muenchen HRB 47456 Ust.-IdNr.
VAT Registration No.: DE129385895
Citibank Frankfurt (BLZ 502 109 00) 600119052
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
</pre></BODY></HTML>