[MPI3 Fortran] Deprecate mpif.h?

Fri Mar 5 04:38:06 CST 2010

Thanks. It looks like we're scanning different quadrants of the matrix I introduced earlier. This may be the source of the disagreement and a part of your frustration, so, as you said, we need to get data to continue a productive discussion.

It appears to me that you don't want to do this in the name of the Forum as a whole, so, let me ask our partners what they think about deprecating the mpif.h/mpi module, and introducing a backward incompatible mpi3 module that will make them recheck (possibly rewrite)/rebuild/revalidate/recertify their mission critical applications. I hope this will help us estimate the impact of the proposed actions, but this will take time. I gather, we may try to have our data collected and processed by the meeting in May.

Could you please consider to reciprocating this in "your" quadrant? Sure, if someone else wants to engage on this, you're welcome. We can discuss the details offline.

I don't think that I missed anything in my reply to Bill, but let me rescan his email and reply separately if need be.

For now, let me clarify my take on your 3 points - I did not go into them individually because I hoped that the general approach would cover them all. Apparently, it did not.

> 2. Saying that the mpi3 module is only for Fortran 2008 compilers is incorrect.  IIRC, the Open MPI prototype implementation of "use mpi3" has been usable with the Intel Fortran compiler since v10.something, for example (it's up to v11.1.something these days). 

I'm afraid you miss my point. As soon as I just say "use mpi3" in an application that used "include mpif.h" or "use mpi" before, I have to recheck, possibly rewrite, rebuild, revalidate, and recertify my application on all OS, platforms, compilers even if I don't change anything else, including the compiler. This is a huge hit that seems to be ignored on this thread.

For comparison, it takes several years for a serious, mission critical ISV to just move to another MPI, without changing anything but the include path and the libraries underneath. You are suggesting to change the Fortran binding and dump the old one. Go figure how long and hard they will think about it.

These are not students who take half an hour to try out a different MPI and report success. That is what I'm trying to convey to you.

> 3. Painting a picture that only advanced / bleeding edge developers will be able to use mpi3 without major disruption is incorrect.  One of the reasons that the conversion functions exist is for exactly this purpose: you can extend legacy Fortran MPI applications with new MPI3 functionality.

I most respectfully disagree. If I have to use conversion functions in my old code, I have to basically rescan all of my application - several million lines of code. This means astronomic overhead. Even if I limit the include mpi3 to one part of the application, and continue with the mpif.h in the other, I have to verify they really work together. This is bad enough by itself, but how can I do the latter if the mpif.h is deprecated? I'm building my business on sand then. The backend overhead mentioned above comes on top.

> 4. Also, stating that you need to "use mpi3" globally throughout an application is incorrect.  As Bill pointed out earlier in this thread, in the scenario you raised, there's potentially only a small amount of new code that needs to use the mpi3 module.

I understand this. However, even if I change a tiny part of a mission critical application, full revalidation/recertification may be necessary, which takes months of calendar time and huge resources.

Let me try to make this more tangible. Next time you step on board an airplane, you tread on the results of the process I outlined. Next time you drive, you do the same. Next time you plug into the mains, you do the same. And so on.

I wish you a most pleasant ride to San Jose.

-----Original Message-----
From: mpi3-fortran-bounces at lists.mpi-forum.org [mailto:mpi3-fortran-bounces at lists.mpi-forum.org] On Behalf Of Jeff Squyres
Sent: Friday, March 05, 2010 12:03 AM
To: MPI-3 Fortran working group
Subject: Re: [MPI3 Fortran] Deprecate mpif.h?

I don't understand this conversation.

You continue to imply that there is a huge impact to legacy Fortran MPI applications if mpif.h is deprecated.  But you also continue to ignore the technical points from this thread.  For example, you haven't answered Bill's questions about adding NBS to existing apps.  You also didn't disagree with my points 2, 3, or 4.  Bills'a and my points can be loosely summed up as: *what* huge impact?  You have not provided any concrete examples that were correct.  

Rolf's questions notwithstanding (as I said earlier, he sent some good, concrete technical items that bear some thought), nothing has yet convinced me that the impact of deprecating mpif.h is large.  So why are we having meaningless discussions about whose opinion of Fortran developers is right?

Forgive me if my frustration with this conversation continues to grow.

On Mar 4, 2010, at 4:30 PM, Supalov, Alexander wrote:

> Thanks. I understand your frustration. It appears that we may be talking to different user groups and operate in slightly different environments. The labs and universities, open source on one hand, established shrink-wrapped app vendors, closed source on the other, roughly. Does not have much to do with the current place of work - more with the background I guess.
> 
> I did not pretend that the numbers were correct. It was just an assumption based on my knowledge of the commercial ISVs and the corporate SW development environment (see the other mail). However, these were numbers rather than general speculation about good and evil I've observed so far. If there are better numbers, let us look at them. If you have user feedback, please put it on the table. If there are no hard facts in this house of glass, let's not throw stones. Let's just admit that we're currently operating on gut feelings.
> 
> Now that frustration has been dealt with, I hope, let's take a more constructive tack.
> 
> The data we can get by asking the users what they think (assuming we can explain them what we want, and they understand) - what level of detail would you expect to see? Certainly, company names, apps, and any numbers cannot be revealed.
> 
> Did we have a question like this in last year's questionnaire? If not, and we want to do a good job at harvesting anonymous opinion, we may want to do another round of questioning. I'd certainly take part in the general process.
> 
> -----Original Message-----
> From: mpi3-fortran-bounces at lists.mpi-forum.org [mailto:mpi3-fortran-bounces at lists.mpi-forum.org] On Behalf Of Jeff Squyres
> Sent: Thursday, March 04, 2010 10:02 PM
> To: MPI-3 Fortran working group
> Subject: Re: [MPI3 Fortran] Deprecate mpif.h?
> 
> Alexander:
> 
> 1. You made up those numbers.  We cannot make decisions based on numbers that are totally fabricated.
> 
> 2. Saying that the mpi3 module is only for Fortran 2008 compilers is incorrect.  IIRC, the Open MPI prototype implementation of "use mpi3" has been usable with the Intel Fortran compiler since v10.something, for example (it's up to v11.1.something these days). 
> 
> 3. Painting a picture that only advanced / bleeding edge developers will be able to use mpi3 without major disruption is incorrect.  One of the reasons that the conversion functions exist is for exactly this purpose: you can extend legacy Fortran MPI applications with new MPI3 functionality.
> 
> 4. Also, stating that you need to "use mpi3" globally throughout an application is incorrect.  As Bill pointed out earlier in this thread, in the scenario you raised, there's potentially only a small amount of new code that needs to use the mpi3 module.

> 
> Rolf raised some good technical points that need thought (e.g., datatype handling).  I'd much rather focus on facts and real issues rather than discuss fear, uncertainty, and doubt.  If you have marketing numbers and feedback from real Fortran MPI apps developers that show that deprecating mpif.h is a terrible idea, great -- let's see that data and discuss it.  So far, the feedback I have heard has been exactly opposite of what you're saying (e.g., talk to Torsten, who interfaces with many Fortran MPI app developers across several different fields).  Fabricating numbers based on your assumptions and then using those numbers as proof is not a good basis for discussion.
> 
> (I apologize for my frustration level)
> 
> 
> 
> On Mar 4, 2010, at 3:20 PM, Supalov, Alexander wrote:
> 
> > Thanks. Let us look into the following table:
> >
> >                 MPI-2.2         MPI-3
> > F77/mpif.h      85%                     5%
> >
> > F90/mpi 5%                      2%
> >
> > F2008/mpi3      2%                      1%
> >
> > You appear to say that we have users in all cells: F77 diehards who don't care of the MPI news; ultra-modernists who will use F2008 and mpi3 module, and all hues in between. The numbers are just bogus representing how many people we may have in each "interest group" out of the total Fortran/MPI community. I did not put 99% into the upper left corner only to not reduce other cells to decimal fractions. :)
> >
> > Now, imagine we deprecate the mpif.h/mpi module and add the mpi3 module. It look like we'll upset 90% of the users and prevent other 7% from going to MPI-3. Makes 97%. Bad.
> >
> > Imagine we freeze mpif.h/mpi module and add mpi3 module. We'll probably hit 7% of the users who would otherwise be willing to try MPI-3. Not deadly but still painful.
> >
> > Finally, imagine we do what was suggested earlier and do a reasonable job at maintaining mpif.h/mpi attractive to those in the more conservative upper part, and extending the mpi3 module as we please. I don't think we'll have anyone who gets to the street.
> >
> > See what I mean? Now compare those numbers to the number of people in the MPI Forum. I would say that if there's pain in extending and maintaining the mpif.h/mpi module, we should probably bear it instead of making the user community suffer.
> >
> > As they said once: "Never was so much owed by so many to so few". I'd love this to be said of us. What about you?
> >
> > -----Original Message-----
> > From: mpi3-fortran-bounces at lists.mpi-forum.org [mailto:mpi3-fortran-bounces at lists.mpi-forum.org] On Behalf Of N.M. Maclaren
> > Sent: Thursday, March 04, 2010 8:27 PM
> > To: MPI-3 Fortran working group
> > Subject: Re: [MPI3 Fortran] Deprecate mpif.h?
> >
> > On Mar 4 2010, Supalov, Alexander wrote:
> > >
> > >What is it that you doubt: the peace or the 99%?
> >
> > Both.  As I said, there are quite a lot of people who want a cleaner
> > interface but have no interest in the new features.  And there will
> > always be some people who want every new gimmick but aren't prepared
> > to change from coding in Fortran 77 - I was seeing people who were
> > still coding in the Fortran 66 subset of Fortran 77 (with, very
> > reluctantly, some use of CHARACTER) until quite recently!
> >
> > So we won't get peace.
> >
> > Regards,
> > Nick Maclaren.
> >
> >
> >
> >
> >
> > _______________________________________________
> > mpi3-fortran mailing list
> > mpi3-fortran at lists.mpi-forum.org
> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran
> > ---------------------------------------------------------------------
> > Intel GmbH
> > Dornacher Strasse 1
> > 85622 Feldkirchen/Muenchen Germany
> > Sitz der Gesellschaft: Feldkirchen bei Muenchen
> > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
> > Registergericht: Muenchen HRB 47456 Ust.-IdNr.
> > VAT Registration No.: DE129385895
> > Citibank Frankfurt (BLZ 502 109 00) 600119052
> >
> > This e-mail and any attachments may contain confidential material for
> > the sole use of the intended recipient(s). Any review or distribution
> > by others is strictly prohibited. If you are not the intended
> > recipient, please contact the sender and delete all copies.
> >
> >
> > _______________________________________________
> > mpi3-fortran mailing list
> > mpi3-fortran at lists.mpi-forum.org
> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran
> >
> 
> 
> --
> Jeff Squyres
> jsquyres at cisco.com
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> 
> _______________________________________________
> mpi3-fortran mailing list
> mpi3-fortran at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran
> ---------------------------------------------------------------------
> Intel GmbH
> Dornacher Strasse 1
> 85622 Feldkirchen/Muenchen Germany
> Sitz der Gesellschaft: Feldkirchen bei Muenchen
> Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
> Registergericht: Muenchen HRB 47456 Ust.-IdNr.
> VAT Registration No.: DE129385895
> Citibank Frankfurt (BLZ 502 109 00) 600119052
> 
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
> 
> 
> _______________________________________________
> mpi3-fortran mailing list
> mpi3-fortran at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran
> 

-- 
Jeff Squyres
jsquyres at cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/

_______________________________________________
mpi3-fortran mailing list
mpi3-fortran at lists.mpi-forum.org
http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran
---------------------------------------------------------------------
Intel GmbH
Dornacher Strasse 1
85622 Feldkirchen/Muenchen Germany
Sitz der Gesellschaft: Feldkirchen bei Muenchen
Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
Registergericht: Muenchen HRB 47456 Ust.-IdNr.
VAT Registration No.: DE129385895
Citibank Frankfurt (BLZ 502 109 00) 600119052

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.