[Mpi3-subsetting] MPI3: Proposal to remove PMPI-Requirement

Fri Mar 7 14:48:39 CST 2008

Hello Bronis,
the idea was not to eliminate PMPI per se. But rather allow in the standard a 
well-defined approach, with which function calls (due to the 
PMPI-requirement) can be macroized and additionally allowing the user to pass 
more information to the library. This should enable the compiler to do "a 
better job"...

- The users that do not want it, do not specify the pre-processor flag and may 
happily use PMPI-based tools.
- The users that do care, may specify it -- and depending on the 
implementation/hardware may see performance improvements. If the 
MPI-implementation does not yet process these macros, well so be it.

Therefore I see this approach as the last step before deployment of an 
application.

The proposed approach is just for discussion and by no means was meant to be 
solution for "the subset" -- however it may be a starting point and 
pre-processor based information a valid way, that goes hand-in-hand with 
other (MPI_Info-based?) subset-approaches...

Thanks,
Rainer

On Friday 07 March 2008 17:20, Bronis R. de Supinski wrote:
> > > A properly designed subsetting approach will NOT try to
> > > partition the profiling interface into its own subset.
> > > Instead, the interface must be partitioned into different
> > > subsets, with each PMPI function in the same subset as the
> > > corresponding MPI function.
> >
> > I'm not sure that I understand your rationale here (perhaps we can
> > discuss more in Chicago?).
>
> Sure, we can discuss it in Chicago. However, The rationale is
> simple enough. The user of a subset wants to be able to profile
> the functionality in a given subset.
>
> > In my view of the world, at least one way to effect subsetting could
> > be something that is dynamically loaded (or not) at run-time.  Hence,
> > you could have PMPI or not, depending on options given to mpirun (for
> > example).  I'm obviously waving my hands a bit here, but that's the
> > premise I was assuming (details TBD, of course).
>
> I have no objection to your premise. However, this proposal
> is specifically designed to prevent dynamic expansion of the
> feature set to include the PMPI calls (or at least to doing
> so effectively since there are no MPI calls between which
> the tool could interpose). In fact, the ability to change
> the subset dynamically is beyond what I was requiring (but
> is desirable). All I stated was that the expansion should
> not require recompilation in order to include the profiling
> interface for what is included in the subset.
>
> If you (or anyone else) can design a mechanism that allows
> using the macros that Rainer wants to enable without precluding
> that interposition dynamically then I offer no opposition to
> the idea. However, I would also suggest that such a mechanism
> does not need any change to the standard; it is already compliant.
>
> >  From your later mail:
> > > Also, while I see your point that only a small set of
> > > stuff needs to be placed into mpi.h, that does not
> > > address the real question: What is the actual application
> > > level benefit? Why do I care if I can get MPI_Comm_size
> > > or MPI_Comm_rank inlined? Good programming practice
> > > does not call them frequently enough for any performance
> > > gain to matter. Alternatively, suppose I can squash some
> > > stuff out of the MPI_Send call path. Will it really matter
> > > to real applications? It seems unlikely, particularly
> > > relative to the cost of lost flexibility for understanding
> > > communication behavior and performance.
> >
> > Rainer's proposal came from joint discussions with NEC where (correct
> > me if I'm wrong, Hubert) select functions *are* [optionally?] inlined
> > in mpi.h because the cost of function calls on their platform is so
> > high.
>
> While one can argue whether the standard should avoid precluding
> the exploitation of as yet unbuilt clever hardware mechanisms,
> it seems pretty clear that it is not the job of the standard to
> accommodate poorly designed hardware. If the cost of function
> calls is so excessively high that ONE prevents effective use of
> the hardware then the vendor needs to reconsider their design.
>
> In any event, it appears that you are stating that a solution was
> found: offer a non-compliant alternative implementation. This is
> a far more appropriate solution in this case. Since I don't believe
> Rainer was proposing that the PMPI interface should be optional
> (just somehow not included in a base subset), the implementor
> still must support it. That means no implementation effort is
> saved by saying they could have a compliant implementation that
> does not always support the interface (but always offers the
> option to include that support). The implementor is always free
> to offer non-compliant alternatives; it is up the users whether
> they want to build with it.
>
> > Additionally, I'm always wary of the term "real applications" -- what
> > a "real" application is to a US DOE lab is very different than what a
> > "real application" is to, say, someone in the oil and gas
> > industry.  :-)  The scale difference alone is enough to make the
> > latency gains by inlining select MPI functions important (meaning: non-
> > US-DOE-labs tend to run at [much] smaller scale; fabric latency may be
> > constrained to a single switch, and therefore *can* see benefit from
> > inlining).
> >
> > As I understand Rainer's proposal, the inlining aspects also
> > predicated on apps that enter production and are never changed.
> > Perhaps this is not a pattern used much at DOE labs, but it is a
> > pattern I see with many customers.
>
> I'm not going to get into a pissing match with you over what
> applications do what. You seem to almost be using "US DOE lab"
> pejoratively. What I can say without the slightest hesitation
> is that recompilation is often a significant barrier to tool
> acceptance for application users and those users are NOT
> limited to one site or even one type of site (yes, I know users
> who are not at "DOE labs").
>
> My opposition to changes that would force tools to require
> recompilation is borne from experience.
>
> Bronis
> _______________________________________________
> mpi3-subsetting mailing list
> mpi3-subsetting_at_[hidden]
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting

-- 
----------------------------------------------------------------
Dipl.-Inf. Rainer Keller   http://www.hlrs.de/people/keller
 HLRS                          Tel: ++49 (0)711-685 6 5858
 Nobelstrasse 19                  Fax: ++49 (0)711-685 6 5832
 70550 Stuttgart                    email: keller_at_[hidden]     
 Germany                             AIM/Skype:rusraink