<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.2900.3314" name=GENERATOR></HEAD>
<BODY>
<DIV dir=ltr align=left><SPAN class=824550214-20062008><FONT face=Arial
color=#0000ff size=2>Agree, thanks.</FONT></SPAN></DIV><BR>
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B>
mpi3-subsetting-bounces@lists.mpi-forum.org
[mailto:mpi3-subsetting-bounces@lists.mpi-forum.org] <B>On Behalf Of </B>Richard
Treumann<BR><B>Sent:</B> Friday, June 20, 2008 3:57 PM<BR><B>To:</B> MPI 3.0
Sub-setting working group<BR><B>Subject:</B> Re: [Mpi3-subsetting] MPI
subsetting: charting the way forward at atelecon next week?<BR></FONT><BR></DIV>
<DIV></DIV>
<P>Hi Alexander<BR><BR>Comments imbedded below.<BR><BR>I have no objections to
someone providing a rationale for assertions related to MPI-IO and MPI_1sided.
If the rationale is sound I have no objection to putting them in the proposal.
<BR><BR>I feel the proposal should be evaluated by the following
algorithm.<BR><BR>If (this concept is one that seems plausible) {<BR>for each
proposed assertion {<BR>if (rationale not solid) <BR>discard<BR>if (deal breaker
downside) <BR>discard<BR>}<BR>if ((concept makes sense) & (set of worthwhile
assertions is not empty))<BR>make this part of MPI 2.2<BR><BR>I do not see much
reason to get every assertion that eventually gains traction into MPI 2.2. MPI
3.0 is soon enough for any that do not make the MPI 2.2 cut. I do not want to
see the concept fall because some particular assertion is controversial.
<BR><BR>I consider <FONT size=4>MPI_NO_EAGER_THROTTLE </FONT>to be the single
most valuable assertion for MPI 2.2 because it is needed to allow MPI to scale
to the levels we are already seeing.<BR><BR><BR>Dick Treumann - MPI Team/TCEM
<BR>IBM Systems & Technology Group<BR>Dept 0lva / MS P963 -- 2455 South Road
-- Poughkeepsie, NY 12601<BR>Tele (845) 433-7846 Fax (845)
433-8363<BR><BR><BR><TT>mpi3-subsetting-bounces@lists.mpi-forum.org wrote on
06/20/2008 02:58:41 AM:<BR><BR>> Dear Dick,</TT><BR><TT>>
</TT><BR><TT>> A couple of suggestions re your
proposal:</TT><BR><TT>> </TT><BR><TT>> - If ASSERTIONS is put at the
end of the MPI_INIT_ASSERTED argument <BR>> list, in C++ one can declare the
last argument as having a zero <BR>> default value, and skip it if necessary.
This might help with <BR>> deprecation of the earlier MPI_INIT_*
calls.</TT><BR><BR><TT>I have no objection. It seems reasonable to let C++
default the </TT><BR><TT>assertions parameter to "none"</TT><BR><BR><TT>> -
In non-Cray parts of the world, an MPI_INT followed by MPI_FLOAT <BR>> is
likely to be a 4-byte int followed by a 4-byte float. This <BR>> sometimes
depends on the compiler settings in effect, too.</TT><BR><BR><TT>My rationale is
not specific to any particular architecture. </TT><BR><TT>Some MPI datatypes are
made entirely </TT><BR><TT>from the same base type. Some are mixtures of types.
If libmpi knows </TT><BR><TT>at the moment a datatype is committed that the send
side and receive</TT><BR><TT>side will always use the same internal
representions then it does not </TT><BR><TT>need to keep track of the fact that
one instance of {MPI_INT,MPI_FLOAT}</TT><BR><TT>has two distinct parts. The send
side can gather and ship 8 bytes </TT><BR><TT>and the receive side can scatter
the 8 bytes. If one side might use 4</TT><BR><TT>byte integers while the other
side uses 8 byte integers then at </TT><BR><TT>least one side will need to know
there is a conversion to be done for </TT><BR><TT>the MPI_INT part. If an MPI
job does a spawn or join that links to a</TT><BR><TT>different architecture
after the datatype has been committed, and</TT><BR><TT>the MPI_Type_commit has
discarded the details, it is too late to get </TT><BR><TT>them back. On
the other hand, if it is known there will never be a</TT><BR><TT>different
architecture added to the job, the extra information can be</TT><BR><TT>safely
discarded.</TT><BR><BR><TT>> - I don't think MPI_NO_THREAD_CONTENTION is
really necessary. The <BR>> original thread level settings, in particular,
the use of anything <BR>> but MPI_THREAD_MULTIPLE, seem to capture the
semantics that you proposed.</TT><BR><BR><TT>This one is kind of tricky and I
also am not sure what it would mean. If</TT><BR><TT>we find a clear value we can
keep it and if not we can remove it.</TT><BR><BR><TT>> - I can't fully follow
the motivation for MPI_NO_ANY_SOURCE <BR>> deprioritization. AFAIK, a
rendezvous exchange usually starts with a<BR>> ready-to-send packet that
contains the size of the message. In this <BR>> case the receiving side will
normally reply with a ready-to-receive <BR>> regardless of the buffer space
available, and flag MPI_ERR_TRUNCATED<BR>> on message arrival if necessary.
In this case, neither <BR>> MPI_ANY_SOURCE not MPI_NO_ANY_SOURCE seem to get
into way.</TT><BR><BR><TT>My point is that MPI_NO_ANY_SOURCE might allow this
round trip </TT><BR><TT>protocol to be replaced by a 1/2 rendezvous protocol. If
it is known</TT><BR><TT>that MPI_ANY_SOURCE will not be used then the receive
side can send</TT><BR><TT>an "envelop and ready for data" packet to the send
side. As long as </TT><BR><TT>the send side knows it will receive the "envelop
and ready for data" </TT><BR><TT>packet when the receive is posted, it does not
need to do the first 1/2</TT><BR><TT>of the rendezvous. The message matching can
be done at the send side.</TT><BR><BR><TT>A send for which the receive was
preposted has a </TT><BR><TT>good chance of finding the "envelop and ready for
data" sitting in </TT><BR><TT>an early queue and the large send can avoid any
rendezvous delay.</TT><BR><TT>Data begins to flow immediately vs waiting for a
round trip of a </TT><BR><TT>full rendezvous. In many cases we cut the delay in
half and best </TT><BR><TT>case we eliminate rendezvous delay completely. If the
receive side </TT><BR><TT>is late in posting the receive we still save a packet
traversal but</TT><BR><TT>do not save any time.</TT><BR><BR><TT>If there may be
an MPI_ANY_SOURCE then this does not work because the</TT><BR><TT>receive side
that has an MPI_ANY_SOURCE cannot guess which sender to </TT><BR><TT>notify so
the sender cannot count on getting a 1/2 rendezvous </TT><BR><TT>notification
for a message that should match the MPI_ANY_SOURCE
</TT><BR><TT>receive.</TT><BR><BR><TT>The problem that made me lower the
priority is that many MPIs use an</TT><BR><TT>eager protocol for small messages
and a rendezvous protocol for large</TT><BR><TT>messages. If the send side
and receive side have the same size buffer</TT><BR><TT>then both sides can reach
the same conclusion: eager vs 1/2 rendezvous.</TT><BR><TT>If both decide on
eager, the receive side will not send an</TT><BR><TT>"envelop and ready for
data" packet and the send side will not look </TT><BR><TT>for one. If both sides
decide on 1/2 rendezvous then the receive side</TT><BR><TT>will send an "envelop
and ready for data" packet and the send side will</TT><BR><TT>look for and
consume the notice. If the send side is for an 8 byte </TT><BR><TT>message
and the receive uses a "big enough" receive buffer of 64KB </TT><BR><TT>then the
two sides will probably not be able to reach the same </TT><BR><TT>conclusion
about the protocol. The receive side will ship off an</TT><BR><TT>"envelop and
ready for data" packet that the send side will not </TT><BR><TT>know what to do
with.</TT><BR><TT> </TT><BR><BR><TT>> </TT><BR><TT>> Best
regards.</TT><BR><TT>> </TT><BR><TT>> Alexander</TT><BR><TT>>
</TT><BR><TT>> From: Supalov, Alexander <BR>> Sent: Friday, June 20,
2008 8:29 AM<BR>> To: 'MPI 3.0 Sub-setting working group'<BR>> Subject:
RE: [Mpi3-subsetting] MPI subsetting: charting the way <BR>> forward at
atelecon next week?<BR></TT><BR><TT>> Dear Dick,</TT><BR><TT>>
</TT><BR><TT>> Thank you. I remember we exchanged a couple of emails
about the <BR>> possible extensions to the set of assertions, like one-sided
and <BR>> I/O, and in my recollection, almost reached an agreement that this
<BR>> can improve performance and possibly memory footprint, as well as
be<BR>> expressed thru assertions. Do you still feel favorable about
this?</TT><BR><TT>> </TT><BR><TT>> Best regards.</TT><BR><TT>>
</TT><BR><TT>> Alexander</TT><BR><TT>>
<BR><BR></TT></P><pre>---------------------------------------------------------------------
Intel GmbH
Dornacher Strasse 1
85622 Feldkirchen/Muenchen Germany
Sitz der Gesellschaft: Feldkirchen bei Muenchen
Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
Registergericht: Muenchen HRB 47456 Ust.-IdNr.
VAT Registration No.: DE129385895
Citibank Frankfurt (BLZ 502 109 00) 600119052
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
</pre></BODY></HTML>