[MPI3 Fortran] Deprecate mpif.h?

Sat Mar 6 00:26:19 CST 2010

Hi everybody,

Following Jeff's constructive criticism, I found a potentially relevant data point in those results of the last year's MPI-3 questionnaire that I currently have access to. This data point appears to support the "gut feeling" numbers in the earlier MPI/Fortran table, and quantify the overwhelming user desire for backward compatibility, also, naturally, in the Fortran area.

Disclaimer: this is my interpretation of data done very early on a Saturday morning, so I welcome your comments.

Let me reproduce that artificial, gut feeling based table first:

		MPI-2.2		MPI-3
F77/mpif.h	85%			5%

F90/mpi	5%			2%

F2008/mpi3	2%			1%

Now, in the MPI-3 questionnaire results, as cited in Jeff's presentation to be found in http://meetings.mpi-forum.org/secretary/2010/01/slides/mpi3-bwcompat.pdf, slide 3, one can see:

"Survey data points (with the usual disclaimers)
 Only 16% think recompiling to MPI-3 is bad: Good!
 Only 11% willing to change source code to MPI-3: Boo!"

I.e., with all the necessary reservations, we may have about 11% of users who would be willing to add the MPI-3 features to their code. Others don't want to touch their codes at all. Well, 11% lie surprisingly close to the total of 7% in the right column of the above table! This sum indicates those who'd be willing to try some MPI-3 features, thus introducing source code changes into their apps.

Further in the same slide we see (again in Jeff's words):

"Do we care:
 Running MPI-3 codes in MPI-1/2 implementations
 	I don’t think so
 Integrating MPI-3 features in existing MPI-1/2 codes
 	I DO think so
 Run MPI-1/2 codes without source code changes in MPI-3
 	YES (per survey data)"

This, again with all the necessary reservations, indicates that users directly ask us not to break the backward compatibility. From this, I derive that the desire to introduce a new, backward incompatible MPI-3 Fortran interface may go against the overwhelming user will for backward compatibility (89% above), as may the desire to deprecate or freeze the conventional mpif.h and the mpi module.

Note that I'm not challenging the desire to do "the right thing" as far as the Fortran/MPI semantic mismatches are concerned. I'm just drawing your attention to the fact that users will need a lot of convincing to accept a backward compatibility break, and may be happier if we give them a chance to try at least some MPI-3 features without such a break.

Best regards.

Alexander

PS. By the way, 84% of those willing to recompile do not mesh well with the 11% who want to introduce code changes. Why would people want to recompile if there's no change? Given now from our commercial partners overwhelming feedback against any code change and recompilation for MPI-3, or, for that matter, introduction of two libraries (MPI-2 and MPI-3) that would effectively double the support overhead, I suspect we may have very few commercial ISVs and their end users on the original survey respondent list.

This observation makes me think that the original questionnaire may have (over)sampled the more cutting edge, educational, open source oriented part of the user spectrum, and thus should be taken with a grain of salt as far as general guidance is concerned. I even think that we may want to consider resampling specifically the traditionally more conservative commercial part, using the same questions, in order to verify the survey results in that area. Would the Forum be interested in such an additional sample?

-----Original Message-----
From: Supalov, Alexander 
Sent: Friday, March 05, 2010 8:50 PM
To: MPI-3 Fortran working group
Subject: RE: [MPI3 Fortran] Deprecate mpif.h?

Thank you. I think we're getting to a more balanced approach here. I hope the Fortran deprecation will never look like the C++ deprecation in MPI-2.2. There MPI had very few users (small but positive number), so this was not a big deal. Here we have the majority of the HPC apps I guess, especially in the classical domains I mentioned in my email to Jeff.

There's still one snag, though: once we deprecate the mpif.h, how can we still extend it? I think these two decision are connected: once we deprecate, we freeze. And once we freeze, we "make" people move to MPI-3. And those being "moved" may resist it indefinitely (cf. MPI-2). What's the point in deprecation then? Negative impact on adoption at the price of correctness. May be this pans out, may be it won't.

I'd rather be for "tempting" and "luring" the inquisitive users into the bright new world by showing some particularly relevant MPI-3 features to the F77 adepts thru the extended mpif.h, letting them smell the wind of the wilderness but making all advanced features available only in the "safe" F2008 environment, as dictated by the language and standard logic.

In the meantime I talked to Nick and understand now that the non-blocking collectives in particular may not be an adequate example of such a "bait": there are simply too many issues with the buffers already in the F77 binding to aggravate them by all the issues that were identified during the discussion of the non-blocking collectives (copy in/out, code movement, etc.).

Lets abstract that out for now. Still, there are some aspects that people may like. FT? Hybrid? Fast RMA?

In the meantime, the feedback I'm getting across a wide commercial ISV range is that a backward incompatible MPI-3 will be a major issue for them. Moreover, those who include mpif.h now won't move until they see MPI-3 offering compelling value. The significance of the mpi3 backward incompatible module is thus but a part of the wider backward compatibility question. I'm looking forward to more data - and to the analysis of the questionnaire results that were collected last year.

-----Original Message-----
From: mpi3-fortran-bounces at lists.mpi-forum.org [mailto:mpi3-fortran-bounces at lists.mpi-forum.org] On Behalf Of Craig Rasmussen
Sent: Friday, March 05, 2010 7:11 PM
To: MPI-3 Fortran working group
Subject: Re: [MPI3 Fortran] Deprecate mpif.h?

On Mar 5, 2010, at 3:44 AM, Supalov, Alexander wrote:

> Thanks. I like very much the balanced approach you outline,  
> understand the technical reasons for the desire to improve the  
> Fortran interface, and agree that the current interface is basically  
> broken (but it works!) and that something must be done.
>
> I just disagree with the proposed staging. Namely, I think that  
> deprecating or freezing the current binding now will send a wrong  
> signal to the masses and increase fear, uncertainty, and doubt in  
> relation to the MPI standard.

I would argue that using mpi3 will reduce fear, uncertainty, and doubt  
because the MPI-3 API will no longer be in conflict with the Fortran  
standard.  So for users this is a good thing and I believe you would  
agree.

> Instead, I argue that we should take a much smoother path into the  
> future, a path that will probably take several decades rather than  
> years, and should not start with a U-turn.
>

A couple of points:

On deprecation.  This is a statement on how the bindings are viewed by  
the standard and not on availability.  As I've been told repeatedly by  
Fortran compiler vendors, just because a feature is deprecated doesn't  
mean it will EVER be removed from the compiler.  So I think  
deprecation is the smooth path to the future that will allow codes to  
slowly migrate over several decades.  It just states that programmers  
should begin the migration path at an appropriate time and not put it  
off forever.  If mpif.h didn't break the Fortran language then I might  
feel differently.  I just don't see how it is a good idea for MPI to  
OFFICIALLY support a standard that is broken.

On freezing mpif.h.  I see this as a separate though related issue.   
New functions could be added to mpif.h although this would be kind of  
weird.  But since it will take time before all Fortran compilers in  
use will support the MPI3 module, it may be appropriate to add new  
functions to mpif.h in the short term.

In any case, I think the MPI standard can be written so that users  
clearly understand that:
1. MPI3 is the future as it will comply with the Fortran standard.
2. mpif.h is deprecated though it will remain around for some time.   
It is deprecated from the standard, not deleted.
3. There is a smooth migration path from mpif.h to using the MPI3  
module.

-craig

_______________________________________________
mpi3-fortran mailing list
mpi3-fortran at lists.mpi-forum.org
http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran
---------------------------------------------------------------------
Intel GmbH
Dornacher Strasse 1
85622 Feldkirchen/Muenchen Germany
Sitz der Gesellschaft: Feldkirchen bei Muenchen
Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
Registergericht: Muenchen HRB 47456 Ust.-IdNr.
VAT Registration No.: DE129385895
Citibank Frankfurt (BLZ 502 109 00) 600119052

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.