<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:p="urn:schemas-microsoft-com:office:powerpoint" xmlns:a="urn:schemas-microsoft-com:office:access" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" xmlns:b="urn:schemas-microsoft-com:office:publisher" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:c="urn:schemas-microsoft-com:office:component:spreadsheet" xmlns:odc="urn:schemas-microsoft-com:office:odc" xmlns:oa="urn:schemas-microsoft-com:office:activation" xmlns:html="http://www.w3.org/TR/REC-html40" xmlns:q="http://schemas.xmlsoap.org/soap/envelope/" xmlns:rtc="http://microsoft.com/officenet/conferencing" xmlns:D="DAV:" xmlns:Repl="http://schemas.microsoft.com/repl/" xmlns:mt="http://schemas.microsoft.com/sharepoint/soap/meetings/" xmlns:x2="http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ppda="http://www.passport.com/NameSpace.xsd" xmlns:ois="http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir="http://schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" xmlns:udc="http://schemas.microsoft.com/data/udc" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sub="http://schemas.microsoft.com/sharepoint/soap/2002/1/alerts/" xmlns:ec="http://www.w3.org/2001/04/xmlenc#" xmlns:sp="http://schemas.microsoft.com/sharepoint/" xmlns:sps="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:udcs="http://schemas.microsoft.com/data/udc/soap" xmlns:udcxf="http://schemas.microsoft.com/data/udc/xmlfile" xmlns:udcp2p="http://schemas.microsoft.com/data/udc/parttopart" xmlns:wf="http://schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:dsss="http://schemas.microsoft.com/office/2006/digsig-setup" xmlns:dssi="http://schemas.microsoft.com/office/2006/digsig" xmlns:mdssi="http://schemas.openxmlformats.org/package/2006/digital-signature" xmlns:mver="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns:mrels="http://schemas.openxmlformats.org/package/2006/relationships" xmlns:spwp="http://microsoft.com/sharepoint/webpartpages" xmlns:ex12t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:ex12m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:pptsl="http://schemas.microsoft.com/sharepoint/soap/SlideLibrary/" xmlns:spsl="http://microsoft.com/webservices/SharePointPortalServer/PublishedLinksService" xmlns:Z="urn:schemas-microsoft-com:" xmlns:st="" xmlns="http://www.w3.org/TR/REC-html40">

<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]-->
<style>
<!--
 /* Font Definitions */
 @font-face
        {font-family:Calibri;
        panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
        {font-family:Tahoma;
        panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
        {font-family:Verdana;
        panose-1:2 11 6 4 3 5 4 4 2 4;}
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0in;
        margin-bottom:.0001pt;
        font-size:12.0pt;
        font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
        {mso-style-priority:99;
        color:blue;
        text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
        {mso-style-priority:99;
        color:purple;
        text-decoration:underline;}
p
        {mso-style-priority:99;
        mso-margin-top-alt:auto;
        margin-right:0in;
        mso-margin-bottom-alt:auto;
        margin-left:0in;
        font-size:12.0pt;
        font-family:"Times New Roman","serif";}
p.ecxmsonormal, li.ecxmsonormal, div.ecxmsonormal
        {mso-style-name:ecxmsonormal;
        mso-margin-top-alt:auto;
        margin-right:0in;
        mso-margin-bottom-alt:auto;
        margin-left:0in;
        font-size:12.0pt;
        font-family:"Times New Roman","serif";}
p.ecxmsochpdefault, li.ecxmsochpdefault, div.ecxmsochpdefault
        {mso-style-name:ecxmsochpdefault;
        mso-margin-top-alt:auto;
        margin-right:0in;
        mso-margin-bottom-alt:auto;
        margin-left:0in;
        font-size:12.0pt;
        font-family:"Times New Roman","serif";}
span.ecxmsohyperlink
        {mso-style-name:ecxmsohyperlink;}
span.ecxmsohyperlinkfollowed
        {mso-style-name:ecxmsohyperlinkfollowed;}
span.ecxemailstyle18
        {mso-style-name:ecxemailstyle18;}
p.ecxmsonormal1, li.ecxmsonormal1, div.ecxmsonormal1
        {mso-style-name:ecxmsonormal1;
        mso-margin-top-alt:auto;
        margin-right:0in;
        margin-bottom:0in;
        margin-left:0in;
        margin-bottom:.0001pt;
        font-size:12.0pt;
        font-family:"Times New Roman","serif";}
span.ecxmsohyperlink1
        {mso-style-name:ecxmsohyperlink1;
        color:blue;
        text-decoration:underline;}
span.ecxmsohyperlinkfollowed1
        {mso-style-name:ecxmsohyperlinkfollowed1;
        color:purple;
        text-decoration:underline;}
span.ecxemailstyle181
        {mso-style-name:ecxemailstyle181;
        font-family:"Calibri","sans-serif";
        color:#1F497D;}
p.ecxmsochpdefault1, li.ecxmsochpdefault1, div.ecxmsochpdefault1
        {mso-style-name:ecxmsochpdefault1;
        mso-margin-top-alt:auto;
        margin-right:0in;
        mso-margin-bottom-alt:auto;
        margin-left:0in;
        font-size:10.0pt;
        font-family:"Times New Roman","serif";}
span.EmailStyle28
        {mso-style-type:personal-reply;
        font-family:"Calibri","sans-serif";
        color:#1F497D;}
.MsoChpDefault
        {mso-style-type:export-only;
        font-size:10.0pt;}
@page Section1
        {size:8.5in 11.0in;
        margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
        {page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
 <o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
 <o:shapelayout v:ext="edit">
  <o:idmap v:ext="edit" data="1" />
 </o:shapelayout></xml><![endif]-->
</head>

<body lang=EN-US link=blue vlink=purple>

<div class=Section1>

<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>What are you talking about?  This thread related to whether
there should be an allflushall() call added in addition to a flushall().  I
argued that it is not obvious that there is a real use case where allflushall()
is substantially better than simply flushall()+barrier().  IBM guys said that
the allflushall() could invoke an allreduce() of a relatively large vector to
get better performance.  I challenged that notion and asked for a back of the
envelope calculation.   You seemed to be replying to that thread… I’m not sure
what comparison related to allflushall() you are referring to or how you would
map your example below into an allflushall() call.  If that is not the thread
you meant to refer to, please clarify.<o:p></o:p></span></p>

<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>

<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Keith<o:p></o:p></span></p>

<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>

<div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'>

<div>

<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>

<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>
mpi3-rma-bounces@lists.mpi-forum.org
[mailto:mpi3-rma-bounces@lists.mpi-forum.org] <b>On Behalf Of </b>Vinod
tipparaju<br>
<b>Sent:</b> Wednesday, May 19, 2010 1:50 PM<br>
<b>To:</b> MPI 3.0 Remote Memory Access working group<br>
<b>Subject:</b> Re: [Mpi3-rma] RMA proposal 1 update<o:p></o:p></span></p>

</div>

</div>

<p class=MsoNormal><o:p> </o:p></p>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>

</div>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>Are
you referring to this email?<o:p></o:p></span></p>

</div>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>

</div>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>>you
allreduce(MPI_SUM, 1 million element integer vector == 4MB)? Then you know how
many things you should have received and when you have received all of those
you can enter a barrier?<o:p></o:p></span></p>

</div>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>

</div>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>>>
If each origin (in a fence epoch) keeps track of the count(s) of RMA<o:p></o:p></span></p>

</div>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>>>
operations to each of its targets, then an allreduce of those arrays<o:p></o:p></span></p>

</div>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>>>
will<o:p></o:p></span></p>

</div>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>>>
tell each target how many operations were done to itself and can be<o:p></o:p></span></p>

</div>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>>>
used to<o:p></o:p></span></p>

</div>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>>>
determine completion.<o:p></o:p></span></p>

</div>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>

</div>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>please
help undestand this better, where does allreduce (in MPI terms) came into the
picture here? There is no big vector. I want to do get_1, add_1, get_2, add_2,
get_3, add_3 when there is not support for collective reduce in the network.
Can this be worse than complete_1, complete_2, complete_3<o:p></o:p></span></p>

</div>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p> </o:p></span></p>

</div>

<p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-size:10.0pt;
font-family:"Verdana","sans-serif"'>Vinod.<br>
<br>
<o:p></o:p></span></p>

<div class=MsoNormal align=center style='text-align:center'><span
style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>

<hr size=2 width="100%" align=center id=stopSpelling>

</span></div>

<p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-size:10.0pt;
font-family:"Verdana","sans-serif"'>From: keith.d.underwood@intel.com<br>
To: mpi3-rma@lists.mpi-forum.org<br>
Date: Wed, 19 May 2010 13:38:43 -0600<br>
Subject: Re: [Mpi3-rma] RMA proposal 1 update<o:p></o:p></span></p>

<div>

<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Yes, it can be worse.  See later emails in the thread…</span><span
style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p></o:p></span></p>

<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> </span><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p></o:p></span></p>

<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Keith</span><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p></o:p></span></p>

<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'> </span><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p></o:p></span></p>

<div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'>

<div>

<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>

<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>
mpi3-rma-bounces@lists.mpi-forum.org
[mailto:mpi3-rma-bounces@lists.mpi-forum.org] <b>On Behalf Of </b>Vinod
tipparaju<br>
<b>Sent:</b> Wednesday, May 19, 2010 1:34 PM<br>
<b>To:</b> MPI 3.0 Remote Memory Access working group<br>
<b>Subject:</b> Re: [Mpi3-rma] RMA proposal 1 update</span><span
style='font-size:10.0pt;font-family:"Verdana","sans-serif"'><o:p></o:p></span></p>

</div>

</div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'> <o:p></o:p></span></p>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'> <o:p></o:p></span></p>

</div>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>I
am very much for supporting collective remote completion. Many optimizations
are possible here.<o:p></o:p></span></p>

</div>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'> <o:p></o:p></span></p>

</div>

<p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-size:10.0pt;
font-family:"Verdana","sans-serif"'>>The question then turns to the
"other networks". If you can't figure out remote completion, then the
collective is going to be pretty heavy, right?<o:p></o:p></span></p>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>May
be this will help make the point. When there is a network that does support a
collective remote completion semantic (say, using its collective network reduce
operation), would this functionality not help get better performance on that
network? Would this functionality, for n collective completions,  ever be
worse than completion_1+completion_2+completion_3+...completion_n? If not, why
not have it? <o:p></o:p></span></p>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'> <o:p></o:p></span></p>

<div>

<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Verdana","sans-serif"'>>
From: keith.d.underwood@intel.com<br>
> To: mpi3-rma@lists.mpi-forum.org<br>
> Date: Sun, 16 May 2010 20:32:36 -0600<br>
> Subject: Re: [Mpi3-rma] RMA proposal 1 update<br>
> <br>
> Before doing that, can someone sketch out the platform/API and the
implementation that makes that more efficient? There is no gain for Portals (3
or 4). There is no gain for anything that supports Cray SHMEM reasonably well
(shmem_quiet() is approximately the same semantics as MPI_flush_all). Hrm, you
can probably say the same thing about anything that supports UPC well - a
strict access is basically a MPI_flush_all(); MPI_Put(); MPI_flush_all();...
Also, I thought somebody said that IB gave you a notification of remote
completion...<br>
> <br>
> The question then turns to the "other networks". If you can't
figure out remote completion, then the collective is going to be pretty heavy,
right?<br>
> <br>
> Keith<br>
> <br>
> > -----Original Message-----<br>
> > From: mpi3-rma-bounces@lists.mpi-forum.org [mailto:mpi3-rma-<br>
> > bounces@lists.mpi-forum.org] On Behalf Of Jeff Hammond<br>
> > Sent: Sunday, May 16, 2010 7:27 PM<br>
> > To: MPI 3.0 Remote Memory Access working group<br>
> > Subject: Re: [Mpi3-rma] RMA proposal 1 update<br>
> > <br>
> > Tortsten,<br>
> > <br>
> > There seemed to be decent agreement on adding MPI_Win_all_flush_all<br>
> > (equivalent to MPI_Win_flush_all called from every rank in the<br>
> > communicator associated with the window) since this function can be<br>
> > implemented far more efficiently as a collective than the equivalent<br>
> > point-wise function calls.<br>
> > <br>
> > Is there a problem with adding this to your proposal?<br>
> > <br>
> > Jeff<br>
> > <br>
> > On Sun, May 16, 2010 at 12:48 AM, Torsten Hoefler <htor@illinois.edu><br>
> > wrote:<br>
> > > Hello all,<br>
> > ><br>
> > > After the discussions at the last Forum I updated the group's
first<br>
> > > proposal.<br>
> > ><br>
> > > The proposal (one-side-2.pdf) is attached to the wiki page<br>
> > > https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/RmaWikiPage<br>
> > ><br>
> > > The changes with regards to the last version are:<br>
> > ><br>
> > > 1) added MPI_NOOP to MPI_Get_accumulate and MPI_Accumulate_get<br>
> > ><br>
> > > 2) (re)added MPI_Win_flush and MPI_Win_flush_all to passive
target<br>
> > mode<br>
> > ><br>
> > > Some remarks:<br>
> > ><br>
> > > 1) We didn't straw-vote on MPI_Accumulate_get, so this function
might<br>
> > >   go. The removal would be very clean.<br>
> > ><br>
> > > 2) Should we allow MPI_NOOP in MPI_Accumulate (this does not
make<br>
> > sense<br>
> > >   and is incorrect in my current proposal)<br>
> > ><br>
> > > 3) Should we allow MPI_REPLACE in<br>
> > MPI_Get_accumulate/MPI_Accumulate_get?<br>
> > >   (this would make sense and is allowed in the current
proposal but<br>
> > we<br>
> > >   didn't talk about it in the group)<br>
> > ><br>
> > ><br>
> > > All the Best,<br>
> > >  Torsten<br>
> > ><br>
> > > --<br>
> > >  bash$ :(){ :|:&};: ---------------------
http://www.unixer.de/ -----<br>
> > > Torsten Hoefler         | Research Associate<br>
> > > Blue Waters Directorate | University of Illinois<br>
> > > 1205 W Clark Street     | Urbana, IL, 61801<br>
> > > NCSA Building           | +01 (217)
244-7736<br>
> > > _______________________________________________<br>
> > > mpi3-rma mailing list<br>
> > > mpi3-rma@lists.mpi-forum.org<br>
> > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma<br>
> > ><br>
> > <br>
> > <br>
> > <br>
> > --<br>
> > Jeff Hammond<br>
> > Argonne Leadership Computing Facility<br>
> > jhammond@mcs.anl.gov / (630) 252-5381<br>
> > http://www.linkedin.com/in/jeffhammond<br>
> > <br>
> > _______________________________________________<br>
> > mpi3-rma mailing list<br>
> > mpi3-rma@lists.mpi-forum.org<br>
> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma<br>
> <br>
> _______________________________________________<br>
> mpi3-rma mailing list<br>
> mpi3-rma@lists.mpi-forum.org<br>
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma<o:p></o:p></span></p>

</div>

</div>

</div>

</div>

</div>

</div>

</div>

</body>

</html>