[Mpi-22] Memory footprint concern

Fri May 9 13:34:33 CDT 2008

Imagine, for example, MPI running on a Cell SPE.  Yes, it sounds crazy,
but people are working on it.  If you look at the state in MPI-2 today,
and assume it will grow proportionally with complexity, MPI-3 could be
really nasty in that context.  So, while I agree that an arbitrarily
large number of permutations is a terrible idea for everyone involved
(implementers wouldn't leverage all of the options, ISVs wouldn't test
them, people who write third party libraries would have a huge headache
dealing with the arbitrary combination of features chosen by the
application), it seems like it would be prudent to try to figure out how
to provide some mechanisms in this direction. I don't know if this
overlaps with your idea about assertions or not, but they do seem to be
related.

Keith

________________________________

From: mpi-22-bounces_at_[hidden]
[mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard
Treumann
Sent: Friday, May 09, 2008 3:12 AM
To: MPI 2.2
Subject: Re: [Mpi-22] Memory footprint concern

Hi Alexander

If we would envision defining subsets in such a way that a machine with
limited memory per node and un-fancy OS can clearly document what its
MPI offers, maybe it is not a huge addition to complexity. For example
if an MPI implementation could say it offers subsets:
MPI 1.2
MPI-IO
MPI-Intercomm Collectives
and nothing else and the meaning would be well defined that may not be
too bad. 

If you are expecting such a machine to have all or most of MPI-3
available and each user would select only the parts he wants at job
launch time it seems to me the complexity for implementation and testing
will be prohibitive. In particular if you hope subsets will give better
performance because of what they leave out and really be compact because
they leave out most unneeded bytes of code the number of permutations
seem to me to make that a vain hope.

I have been ambivalent about whether my assertions proposal really has
much connection to the subsets concept mostly because I think to be
useful assertions must be fairly simple. The only assertion I am 100%
sure will pay off and be used is MPI_NO_EAGER_THROTTLE. There are MPI
implementations today that act as if the assertion MPI_NO_EAGER_THROTTLE
were present on all applications but that is not an option for a vendor
MPI because if somebody contacts service and says "Your MPI violates the
standard" a vendor like IBM must "fix" it. IBM MPI supports MPI_CANCEL
on MPI_ISEND but at least one MPI does not because it is "too
expensive". In effect they act like every application has the assertion
MPI_NO_SEND_CANCELS. IBM MPI could make use of MPI_NO_REQUEST_MIX if it
was available.

Dick 

Dick Treumann - MPI Team/TCEM 
IBM Systems & Technology Group
Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601
Tele (845) 433-7846 Fax (845) 433-8363

 "Supalov, Alexander" <alexander.supalov_at_[hidden]>

"Supalov, Alexander" <alexander.supalov_at_[hidden]> 
Sent by: mpi-22-bounces_at_[hidden] 

05/08/2008 05:52 PM 

Please respond to
"MPI 2.2" <mpi-22_at_[hidden]>

To

"MPI 2.2" <mpi-22_at_[hidden]>

cc

Subject

Re: [Mpi-22] Memory footprint concern

Dear Dick,

By the looks of it, MPI-3 is going to be big. Petascale machines may not
have OS we're accustomed to, dynamic libraries, and some other things.
Smaller system libraries - and smaller MPI - may be needed there. Some
of the envisioned MPI-3 features will be needed for some applications,
some won't. Same with MPI-2 and MPI-1. Defining subsets may help to open
a way to custom cut MPI libraries suitable for particular application
classes. How subsets will be implemented is a different matter.

Best regards.

Alexander

________________________________

From: mpi-22-bounces_at_[hidden] [
mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann
Sent: Thursday, May 08, 2008 11:23 PM
To: MPI 2.2
Subject: [Mpi-22] Memory footprint concern

Can somebody help me understand this "smaller memory footprint" issue
that is part f the subsetting goal better. What systems does it affect?
What does "memory footprint" really mean? In the 64 bit address space,
virtual address range is not a problem.

On systems I am most familiar with (AIX and I have been told Linux too),
if you have a library that contains 1000 subroutines and you run a
program than only calls 6 then only the pages that are touched by code
for those 6 functions must get placed in real memory. The rest of the
object code stays on disk. Program and library text is demand paged. The
loading is on page boundries, not subroutine boundries. 

With a shared library, if I run a program on a node and touch 6
subroutines and you run a different program that touches those 6 and 10
more then code for all 16 subroutines may be kept in memory but the rest
of the library will stay on disk. You and I will share the object code
for the 6 subroutines we are both calling.

Someone who wanted to make a libmpi that has MPI-1sided or MPI-IO well
isolated in the library structure so simple MPI jobs would not force
this extra code into memory could do that today. The user does not need
to promise not to call MPI-IO subroutines for them not to to take real
memory. The "subsets" would need to be devised by the MPI implementor
but would be transparent to the MPI user and not dictated by the
standard. The "subsets" the user did not call would remain paged out.

Perhaps all static data defined by the library will come into real
memory for each process but is there much reduction from being able to
somehow not bring in the static data MPI-IO would require because
somebody had promised not to use it?

Dick 

Dick Treumann - MPI Team/TCEM 
IBM Systems & Technology Group
Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601
Tele (845) 433-7846 Fax (845) 433-8363 

---------------------------------------------------------------------
Intel GmbH
Dornacher Strasse 1
85622 Feldkirchen/Muenchen Germany
Sitz der Gesellschaft: Feldkirchen bei Muenchen
Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
Registergericht: Muenchen HRB 47456 Ust.-IdNr.
VAT Registration No.: DE129385895
Citibank Frankfurt (BLZ 502 109 00) 600119052

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
_______________________________________________
mpi-22 mailing list
mpi-22_at_[hidden]
http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

* 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpi-22/attachments/20080509/a184ccf2/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: application/octet-stream
Size: 156 bytes
Desc: not available
URL: <http://lists.mpi-forum.org/pipermail/mpi-22/attachments/20080509/a184ccf2/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.gif
Type: application/octet-stream
Size: 73 bytes
Desc: not available
URL: <http://lists.mpi-forum.org/pipermail/mpi-22/attachments/20080509/a184ccf2/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.gif
Type: application/octet-stream
Size: 73 bytes
Desc: not available
URL: <http://lists.mpi-forum.org/pipermail/mpi-22/attachments/20080509/a184ccf2/attachment-0002.obj>