<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:st1="urn:schemas-microsoft-com:office:smarttags" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 11 (filtered medium)">
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]--><o:SmartTagType
namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="country-region"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="PostalCode"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="State"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="City"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="place"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="Street"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="address"/>
<!--[if !mso]>
<style>
st1\:*{behavior:url(#default#ieooui) }
</style>
<![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:blue;
text-decoration:underline;}
p
{mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman";}
tt
{font-family:"Courier New";}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:Arial;
color:navy;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
{page:Section1;}
/* List Definitions */
@list l0
{mso-list-id:2082484324;
mso-list-template-ids:1550978328;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=blue>
<div class=Section1>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Imagine, for example, MPI running on a
Cell SPE. Yes, it sounds crazy, but people are working on it. If
you look at the state in MPI-2 today, and assume it will grow proportionally
with complexity, MPI-3 could be really nasty in that context. So, while I
agree that an arbitrarily large number of permutations is a terrible idea for
everyone involved (implementers wouldn’t leverage all of the options,
ISVs wouldn’t test them, people who write third party libraries would
have a huge headache dealing with the arbitrary combination of features chosen
by the application), it seems like it would be prudent to try to figure out how
to provide some mechanisms in this direction. I don’t know if this
overlaps with your idea about assertions or not, but they do seem to be
related.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Keith<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'>
<div>
<div class=MsoNormal align=center style='text-align:center'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'>
<hr size=2 width="100%" align=center tabindex=-1>
</span></font></div>
<p class=MsoNormal><b><font size=2 face=Tahoma><span style='font-size:10.0pt;
font-family:Tahoma;font-weight:bold'>From:</span></font></b><font size=2
face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma'>
mpi-22-bounces@lists.mpi-forum.org [mailto:mpi-22-bounces@lists.mpi-forum.org] <b><span
style='font-weight:bold'>On Behalf Of </span></b>Richard Treumann<br>
<b><span style='font-weight:bold'>Sent:</span></b> Friday, May 09, 2008 3:12 AM<br>
<b><span style='font-weight:bold'>To:</span></b> MPI 2.2<br>
<b><span style='font-weight:bold'>Subject:</span></b> Re: [Mpi-22] Memory
footprint concern</span></font><o:p></o:p></p>
</div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><o:p> </o:p></span></font></p>
<p style='margin-bottom:12.0pt'><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'>Hi Alexander<br>
<br>
If we would envision defining subsets in such a way that a machine with limited
memory per node and un-fancy OS can clearly document what its MPI offers, maybe
it is not a huge addition to complexity. For example if an MPI implementation
could say it offers subsets:<br>
MPI 1.2<br>
MPI-IO<br>
MPI-Intercomm Collectives<br>
and nothing else and the meaning would be well defined that may not be too bad.
<br>
<br>
If you are expecting such a machine to have all or most of MPI-3 available and
each user would select only the parts he wants at job launch time it seems to
me the complexity for implementation and testing will be prohibitive. In
particular if you hope subsets will give better performance because of what
they leave out and really be compact because they leave out most unneeded bytes
of code the number of permutations seem to me to make that a vain hope.<br>
<br>
I have been ambivalent about whether my assertions proposal really has much
connection to the subsets concept mostly because I think to be useful
assertions must be fairly simple. The only assertion I am 100% sure will pay
off and be used is MPI_NO_EAGER_THROTTLE. There are MPI implementations today
that act as if the assertion MPI_NO_EAGER_THROTTLE were present on all
applications but that is not an option for a vendor MPI because if somebody
contacts service and says "Your MPI violates the standard" a vendor
like IBM must "fix" it. IBM MPI supports MPI_CANCEL on MPI_ISEND but
at least one MPI does not because it is "too expensive". In effect
they act like every application has the assertion MPI_NO_SEND_CANCELS. IBM MPI
could make use of MPI_NO_REQUEST_MIX if it was available.<br>
<br>
Dick <br>
<br>
Dick Treumann - MPI Team/TCEM <br>
IBM Systems & Technology Group<br>
<st1:address w:st="on"><st1:Street w:st="on">Dept</st1:Street> 0lva</st1:address>
/ MS P963 -- <st1:Street w:st="on"><st1:address w:st="on">2455 South Road</st1:address></st1:Street>
-- <st1:place w:st="on"><st1:City w:st="on">Poughkeepsie</st1:City>, <st1:State
w:st="on">NY</st1:State> <st1:PostalCode w:st="on">12601</st1:PostalCode></st1:place><br>
Tele (845) 433-7846 Fax (845) 433-8363<br>
<br>
<img width=16 height=16 id="_x0000_i1025"
src="cid:image001.gif@01C8B1B0.6C7C3920"
alt="Inactive hide details for "Supalov, Alexander" <alexander.supalov@intel.com>"
border=0>"Supalov, Alexander" <alexander.supalov@intel.com><br>
<br>
<o:p></o:p></span></font></p>
<table class=MsoNormalTable border=0 cellspacing=0 cellpadding=0 width="100%"
style='width:100.0%'>
<tr>
<td width="40%" valign=top style='width:40.0%;padding:0in 0in 0in 0in'>
<p class=MsoNormal style='margin-left:2.0in'><b><font size=2
face="Times New Roman"><span style='font-size:10.0pt;font-weight:bold'>"Supalov,
Alexander" <alexander.supalov@intel.com></span></font></b><font
size=2><span style='font-size:10.0pt'> </span></font><br>
<font size=2><span style='font-size:10.0pt'>Sent by:
mpi-22-bounces@lists.mpi-forum.org</span></font> <o:p></o:p></p>
<p style='margin-left:2.0in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>05/08/2008 05:52 PM</span></font> <o:p></o:p></p>
<table class=MsoNormalTable border=1 cellpadding=0 style='margin-left:2.0in'>
<tr>
<td width=168 valign=top bgcolor=white style='width:1.75in;background:white;
padding:.75pt .75pt .75pt .75pt'>
<p class=MsoNormal align=center style='text-align:center'><font size=2
face="Times New Roman"><span style='font-size:10.0pt'>Please respond to<br>
"MPI 2.2" <mpi-22@lists.mpi-forum.org></span></font><o:p></o:p></p>
</td>
</tr>
</table>
<p class=MsoNormal><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'><o:p></o:p></span></font></p>
</td>
<td width="60%" valign=top style='width:60.0%;padding:0in 0in 0in 0in'>
<table class=MsoNormalTable border=0 cellspacing=0 cellpadding=0 width="100%"
style='width:100.0%'>
<tr>
<td width="1%" valign=top style='width:1.0%;padding:0in 0in 0in 0in'>
<p class=MsoNormal><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'><img width=58 height=1 id="_x0000_i1026"
src="cid:image004.gif@01C8B1B0.6C7C3920" border=0><o:p></o:p></span></font></p>
<p class=MsoNormal align=right style='text-align:right'><font size=2
face="Times New Roman"><span style='font-size:10.0pt'>To</span></font><o:p></o:p></p>
</td>
<td width="100%" valign=top style='width:100.0%;padding:0in 0in 0in 0in'>
<p class=MsoNormal><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'><img width=1 height=1 id="_x0000_i1027"
src="cid:image005.gif@01C8B1B0.6C7C3920" border=0><br>
</span></font><font size=2><span style='font-size:10.0pt'>"MPI
2.2" <mpi-22@lists.mpi-forum.org></span></font><o:p></o:p></p>
</td>
</tr>
<tr>
<td width="1%" valign=top style='width:1.0%;padding:0in 0in 0in 0in'>
<p class=MsoNormal><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'><img width=58 height=1 id="_x0000_i1028"
src="cid:image004.gif@01C8B1B0.6C7C3920" border=0><o:p></o:p></span></font></p>
<p class=MsoNormal align=right style='text-align:right'><font size=2
face="Times New Roman"><span style='font-size:10.0pt'>cc</span></font><o:p></o:p></p>
</td>
<td width="100%" valign=top style='width:100.0%;padding:0in 0in 0in 0in'>
<p class=MsoNormal><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'><img width=1 height=1 id="_x0000_i1029"
src="cid:image005.gif@01C8B1B0.6C7C3920" border=0><o:p></o:p></span></font></p>
</td>
</tr>
<tr>
<td width="1%" valign=top style='width:1.0%;padding:0in 0in 0in 0in'>
<p class=MsoNormal><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'><img width=58 height=1 id="_x0000_i1030"
src="cid:image004.gif@01C8B1B0.6C7C3920" border=0><o:p></o:p></span></font></p>
<p class=MsoNormal align=right style='text-align:right'><font size=2
face="Times New Roman"><span style='font-size:10.0pt'>Subject</span></font><o:p></o:p></p>
</td>
<td width="100%" valign=top style='width:100.0%;padding:0in 0in 0in 0in'>
<p class=MsoNormal><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'><img width=1 height=1 id="_x0000_i1031"
src="cid:image005.gif@01C8B1B0.6C7C3920" border=0><br>
</span></font><font size=2><span style='font-size:10.0pt'>Re: [Mpi-22]
Memory footprint concern</span></font><o:p></o:p></p>
</td>
</tr>
</table>
<p class=MsoNormal><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'><o:p> </o:p></span></font></p>
<table class=MsoNormalTable border=0 cellspacing=0 cellpadding=0>
<tr>
<td width=58 valign=top style='width:43.5pt;padding:0in 0in 0in 0in'>
<p class=MsoNormal><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'><img width=1 height=1 id="_x0000_i1032"
src="cid:image005.gif@01C8B1B0.6C7C3920" border=0><o:p></o:p></span></font></p>
</td>
<td width=336 valign=top style='width:3.5in;padding:0in 0in 0in 0in'>
<p class=MsoNormal><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'><img width=1 height=1 id="_x0000_i1033"
src="cid:image005.gif@01C8B1B0.6C7C3920" border=0><o:p></o:p></span></font></p>
</td>
</tr>
</table>
<p class=MsoNormal><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'><o:p></o:p></span></font></p>
</td>
</tr>
</table>
<p style='margin-bottom:12.0pt'><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'><br>
</span></font><font color=blue face=Arial><span style='font-family:Arial;
color:blue'>Dear Dick,</span></font><br>
<br>
<font color=blue face=Arial><span style='font-family:Arial;color:blue'>By the
looks of it, MPI-3 is going to be big. Petascale machines may not have OS we're
accustomed to, dynamic libraries, and some other things. Smaller system
libraries - and smaller MPI - may be needed there. Some of the envisioned MPI-3
features will be needed for some applications, some won't. Same with MPI-2 and
MPI-1. Defining subsets may help to open a way to custom cut MPI libraries
suitable for particular application classes. How subsets will be implemented is
a different matter.</span></font><br>
<br>
<font color=blue face=Arial><span style='font-family:Arial;color:blue'>Best
regards.</span></font><br>
<br>
<font color=blue face=Arial><span style='font-family:Arial;color:blue'>Alexander</span></font><o:p></o:p></p>
<div class=MsoNormal><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'>
<hr size=2 width="100%" align=left>
</span></font></div>
<p class=MsoNormal><b><font size=3 face=Tahoma><span style='font-size:12.0pt;
font-family:Tahoma;font-weight:bold'>From:</span></font></b><font face=Tahoma><span
style='font-family:Tahoma'> mpi-22-bounces@lists.mpi-forum.org [<a
href="mailto:mpi-22-bounces@lists.mpi-forum.org">mailto:mpi-22-bounces@lists.mpi-forum.org</a>]
<b><span style='font-weight:bold'>On Behalf Of </span></b>Richard Treumann<b><span
style='font-weight:bold'><br>
Sent:</span></b> Thursday, May 08, 2008 11:23 PM<b><span style='font-weight:
bold'><br>
To:</span></b> MPI 2.2<b><span style='font-weight:bold'><br>
Subject:</span></b> [Mpi-22] Memory footprint concern</span></font><o:p></o:p></p>
<p><font size=5 face="Times New Roman"><span style='font-size:18.0pt'>Can
somebody help me understand this "smaller memory footprint" issue
that is part f the subsetting goal better. What systems does it affect? What
does "memory footprint" really mean? In the 64 bit address space,
virtual address range is not a problem.<br>
<br>
On systems I am most familiar with (AIX and I have been told Linux too), if you
have a library that contains 1000 subroutines and you run a program than only
calls 6 then only the pages that are touched by code for those 6 functions must
get placed in real memory. The rest of the object code stays on disk. Program
and library text is demand paged. The loading is on page boundries, not
subroutine boundries. <br>
<br>
With a shared library, if I run a program on a node and touch 6 subroutines and
you run a different program that touches those 6 and 10 more then code for all
16 subroutines may be kept in memory but the rest of the library will stay on
disk. You and I will share the object code for the 6 subroutines we are both
calling.</span></font><font size=4><span style='font-size:13.5pt'><br>
</span></font><font size=5><span style='font-size:18.0pt'><br>
Someone who wanted to make a libmpi that has MPI-1sided or MPI-IO well isolated
in the library structure so simple MPI jobs would not force this extra code
into memory could do that today. The user does not need to promise not to call
MPI-IO subroutines for them not to to take real memory. The "subsets"
would need to be devised by the MPI implementor but would be transparent to the
MPI user and not dictated by the standard. The "subsets" the user did
not call would remain paged out.<br>
<br>
Perhaps all static data defined by the library will come into real memory for
each process but is there much reduction from being able to somehow not bring
in the static data MPI-IO would require because somebody had promised not to
use it?<br>
<br>
Dick </span></font><font size=4><span style='font-size:13.5pt'><br>
<br>
Dick Treumann - MPI Team/TCEM <br>
IBM Systems & Technology Group<br>
<st1:address w:st="on"><st1:Street w:st="on">Dept</st1:Street> 0lva</st1:address>
/ MS P963 -- <st1:Street w:st="on"><st1:address w:st="on">2455 South Road</st1:address></st1:Street>
-- <st1:place w:st="on"><st1:City w:st="on">Poughkeepsie</st1:City>, <st1:State
w:st="on">NY</st1:State> <st1:PostalCode w:st="on">12601</st1:PostalCode></st1:place><br>
Tele (845) 433-7846 Fax (845) 433-8363</span></font> <o:p></o:p></p>
<p><tt><font size=4 face="Courier New"><span style='font-size:13.5pt'>---------------------------------------------------------------------</span></font></tt><font
size=4 face="Courier New"><span style='font-size:13.5pt;font-family:"Courier New"'><br>
<tt><font face="Courier New">Intel GmbH</font></tt><br>
<tt><font face="Courier New">Dornacher Strasse 1</font></tt><br>
<tt><font face="Courier New">85622 Feldkirchen/Muenchen <st1:country-region
w:st="on"><st1:place w:st="on">Germany</st1:place></st1:country-region></font></tt><br>
<tt><font face="Courier New">Sitz der Gesellschaft: Feldkirchen bei Muenchen</font></tt><br>
<tt><font face="Courier New">Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner,
Hannes Schwaderer</font></tt><br>
<tt><font face="Courier New">Registergericht: Muenchen HRB 47456 Ust.-IdNr.</font></tt><br>
<tt><font face="Courier New">VAT Registration No.: DE129385895</font></tt><br>
<tt><font face="Courier New">Citibank Frankfurt (BLZ 502 109 00) 600119052</font></tt><br>
<br>
<tt><font face="Courier New">This e-mail and any attachments may contain
confidential material for</font></tt><br>
<tt><font face="Courier New">the sole use of the intended recipient(s). Any
review or distribution</font></tt><br>
<tt><font face="Courier New">by others is strictly prohibited. If you are not
the intended</font></tt><br>
<tt><font face="Courier New">recipient, please contact the sender and delete
all copies.</font></tt><br>
</span></font><tt><font size=2 face="Courier New"><span style='font-size:10.0pt'>_______________________________________________</span></font></tt><font
size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'><br>
<tt><font face="Courier New">mpi-22 mailing list</font></tt><br>
<tt><font face="Courier New">mpi-22@lists.mpi-forum.org</font></tt><br>
<tt><font face="Courier New"><a
href="http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22">http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22</a></font></tt></span></font><o:p></o:p></p>
</div>
</div>
</body>
</html>