<html><body>
<p>Hi Alexander<br>
<br>
If we would envision defining subsets in such a way that a machine with limited memory per node and un-fancy OS can clearly document what its MPI offers, maybe it is not a huge addition to complexity. For example if an MPI implementation could say it offers subsets:<br>
MPI 1.2<br>
MPI-IO<br>
MPI-Intercomm Collectives<br>
and nothing else and the meaning would be well defined that may not be too bad. <br>
<br>
If you are expecting such a machine to have all or most of MPI-3 available and each user would select only the parts he wants at job launch time it seems to me the complexity for implementation and testing will be prohibitive. In particular if you hope subsets will give better performance because of what they leave out and really be compact because they leave out most unneeded bytes of code the number of permutations seem to me to make that a vain hope.<br>
<br>
I have been ambivalent about whether my assertions proposal really has much connection to the subsets concept mostly because I think to be useful assertions must be fairly simple. The only assertion I am 100% sure will pay off and be used is MPI_NO_EAGER_THROTTLE. There are MPI implementations today that act as if the assertion MPI_NO_EAGER_THROTTLE were present on all applications but that is not an option for a vendor MPI because if somebody contacts service and says "Your MPI violates the standard" a vendor like IBM must "fix" it. IBM MPI supports MPI_CANCEL on MPI_ISEND but at least one MPI does not because it is "too expensive". In effect they act like every application has the assertion MPI_NO_SEND_CANCELS. IBM MPI could make use of MPI_NO_REQUEST_MIX if it was available.<br>
<br>
Dick <br>
<br>
Dick Treumann - MPI Team/TCEM <br>
IBM Systems & Technology Group<br>
Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601<br>
Tele (845) 433-7846 Fax (845) 433-8363<br>
<br>
<img width="16" height="16" src="cid:1__=0ABBFED7DF90A7D38f9e8a93df938@us.ibm.com" border="0" alt="Inactive hide details for "Supalov, Alexander" <alexander.supalov@intel.com>">"Supalov, Alexander" <alexander.supalov@intel.com><br>
<br>
<br>
<table width="100%" border="0" cellspacing="0" cellpadding="0">
<tr valign="top"><td style="background-image:url(cid:2__=0ABBFED7DF90A7D38f9e8a93df938@us.ibm.com); background-repeat: no-repeat; " width="40%">
<ul>
<ul>
<ul>
<ul><b><font size="2">"Supalov, Alexander" <alexander.supalov@intel.com></font></b><font size="2"> </font><br>
<font size="2">Sent by: mpi-22-bounces@lists.mpi-forum.org</font>
<p><font size="2">05/08/2008 05:52 PM</font>
<table border="1">
<tr valign="top"><td width="168" bgcolor="#FFFFFF"><div align="center"><font size="2">Please respond to<br>
"MPI 2.2" <mpi-22@lists.mpi-forum.org></font></div></td></tr>
</table>
</ul>
</ul>
</ul>
</ul>
</td><td width="60%">
<table width="100%" border="0" cellspacing="0" cellpadding="0">
<tr valign="top"><td width="1%"><img width="58" height="1" src="cid:3__=0ABBFED7DF90A7D38f9e8a93df938@us.ibm.com" border="0" alt=""><br>
<div align="right"><font size="2">To</font></div></td><td width="100%"><img width="1" height="1" src="cid:3__=0ABBFED7DF90A7D38f9e8a93df938@us.ibm.com" border="0" alt=""><br>
<font size="2">"MPI 2.2" <mpi-22@lists.mpi-forum.org></font></td></tr>
<tr valign="top"><td width="1%"><img width="58" height="1" src="cid:3__=0ABBFED7DF90A7D38f9e8a93df938@us.ibm.com" border="0" alt=""><br>
<div align="right"><font size="2">cc</font></div></td><td width="100%"><img width="1" height="1" src="cid:3__=0ABBFED7DF90A7D38f9e8a93df938@us.ibm.com" border="0" alt=""><br>
</td></tr>
<tr valign="top"><td width="1%"><img width="58" height="1" src="cid:3__=0ABBFED7DF90A7D38f9e8a93df938@us.ibm.com" border="0" alt=""><br>
<div align="right"><font size="2">Subject</font></div></td><td width="100%"><img width="1" height="1" src="cid:3__=0ABBFED7DF90A7D38f9e8a93df938@us.ibm.com" border="0" alt=""><br>
<font size="2">Re: [Mpi-22] Memory footprint concern</font></td></tr>
</table>
<table border="0" cellspacing="0" cellpadding="0">
<tr valign="top"><td width="58"><img width="1" height="1" src="cid:3__=0ABBFED7DF90A7D38f9e8a93df938@us.ibm.com" border="0" alt=""></td><td width="336"><img width="1" height="1" src="cid:3__=0ABBFED7DF90A7D38f9e8a93df938@us.ibm.com" border="0" alt=""></td></tr>
</table>
</td></tr>
</table>
<br>
<font color="#0000FF" face="Arial">Dear Dick,</font><br>
<font size="4"> </font><br>
<font color="#0000FF" face="Arial">By the looks of it, MPI-3 is going to be big. Petascale machines may not have OS we're accustomed to, dynamic libraries, and some other things. Smaller system libraries - and smaller MPI - may be needed there. Some of the envisioned MPI-3 features will be needed for some applications, some won't. Same with MPI-2 and MPI-1. Defining subsets may help to open a way to custom cut MPI libraries suitable for particular application classes. How subsets will be implemented is a different matter.</font><br>
<font size="4"> </font><br>
<font color="#0000FF" face="Arial">Best regards.</font><br>
<font size="4"> </font><br>
<font color="#0000FF" face="Arial">Alexander</font><br>
<br>
<hr width="100%" size="2" align="left"><b><font face="Tahoma">From:</font></b><font face="Tahoma"> mpi-22-bounces@lists.mpi-forum.org [<a href="mailto:mpi-22-bounces@lists.mpi-forum.org">mailto:mpi-22-bounces@lists.mpi-forum.org</a>] </font><b><font face="Tahoma">On Behalf Of </font></b><font face="Tahoma">Richard Treumann</font><b><font face="Tahoma"><br>
Sent:</font></b><font face="Tahoma"> Thursday, May 08, 2008 11:23 PM</font><b><font face="Tahoma"><br>
To:</font></b><font face="Tahoma"> MPI 2.2</font><b><font face="Tahoma"><br>
Subject:</font></b><font face="Tahoma"> [Mpi-22] Memory footprint concern</font><font size="4"><br>
</font>
<p><font size="5">Can somebody help me understand this "smaller memory footprint" issue that is part f the subsetting goal better. What systems does it affect? What does "memory footprint" really mean? In the 64 bit address space, virtual address range is not a problem.<br>
<br>
On systems I am most familiar with (AIX and I have been told Linux too), if you have a library that contains 1000 subroutines and you run a program than only calls 6 then only the pages that are touched by code for those 6 functions must get placed in real memory. The rest of the object code stays on disk. Program and library text is demand paged. The loading is on page boundries, not subroutine boundries. <br>
<br>
With a shared library, if I run a program on a node and touch 6 subroutines and you run a different program that touches those 6 and 10 more then code for all 16 subroutines may be kept in memory but the rest of the library will stay on disk. You and I will share the object code for the 6 subroutines we are both calling.</font><font size="4"><br>
</font><font size="5"><br>
Someone who wanted to make a libmpi that has MPI-1sided or MPI-IO well isolated in the library structure so simple MPI jobs would not force this extra code into memory could do that today. The user does not need to promise not to call MPI-IO subroutines for them not to to take real memory. The "subsets" would need to be devised by the MPI implementor but would be transparent to the MPI user and not dictated by the standard. The "subsets" the user did not call would remain paged out.<br>
<br>
Perhaps all static data defined by the library will come into real memory for each process but is there much reduction from being able to somehow not bring in the static data MPI-IO would require because somebody had promised not to use it?<br>
<br>
Dick </font><font size="4"><br>
<br>
Dick Treumann - MPI Team/TCEM <br>
IBM Systems & Technology Group<br>
Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601<br>
Tele (845) 433-7846 Fax (845) 433-8363</font>
<p><tt><font size="4">---------------------------------------------------------------------<br>
Intel GmbH<br>
Dornacher Strasse 1<br>
85622 Feldkirchen/Muenchen Germany<br>
Sitz der Gesellschaft: Feldkirchen bei Muenchen<br>
Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer<br>
Registergericht: Muenchen HRB 47456 Ust.-IdNr.<br>
VAT Registration No.: DE129385895<br>
Citibank Frankfurt (BLZ 502 109 00) 600119052<br>
<br>
This e-mail and any attachments may contain confidential material for<br>
the sole use of the intended recipient(s). Any review or distribution<br>
by others is strictly prohibited. If you are not the intended<br>
recipient, please contact the sender and delete all copies.<br>
</font></tt><tt>_______________________________________________<br>
mpi-22 mailing list<br>
mpi-22@lists.mpi-forum.org<br>
</tt><tt><a href="http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22">http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22</a></tt><tt><br>
</tt>
<p></body></html>