[Mpi-forum] MPI user survey

[Richard Treumann]

I know it is tempting to look for public input but for issues as complex as
the ones the MPI Forum is trying to resolve asking a question that
generates a useful answer is so hard it may not really be very useful to
try.

When there are 10 people in the world who have dug deeply enough into an
issue to really understand it, they are unlikely to benefit from the
opinions of hundreds with superficial understanding.

Of course, anyone who has an agenda in politics knows the tactic of taking
a poll with a simplistic question that gets 85% favorable response and then
saying it demonstrates support for the 1000 page bill he is proposing. If
someone else polls with a slightly different description of the same  1000
page bill he may show there is 85% opposition.

I put some comments below too.

               Dick

Dick Treumann  -  MPI Team
IBM Systems & Technology Group
Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601
Tele (845) 433-7846         Fax (845) 433-8363


mpi-forum-bounces at lists.mpi-forum.org wrote on 11/16/2009 10:10:54 AM:

> [image removed]
>
> Re: [Mpi-forum] MPI user survey
>
> Jeff Squyres
>
> to:
>
> Main MPI Forum mailing list
>
> 11/16/2009 10:13 AM
>
> Sent by:
>
> mpi-forum-bounces at lists.mpi-forum.org
>
> Please respond to Main MPI Forum mailing list
>
> On Nov 16, 2009, at 5:15 AM, Richard Treumann wrote:
>
> > I did not see this until Monday morning but here are some late
> > comments.
> >
>
> Thanks for all the time and effort, Dick!  Making a good/useful survey
> is a darn hard process...  (I think we've easily consumed at least
> 20-30 person-hours on this already!)
>
> > How do you plan to give a reasonable range of stake holders a chance
> > to understand the questions and provide their answers?
> >
>
> Are you saying that the questions are difficult to understand?  I'm
> not sure what your indirect assertion is here...?

The HPC customer (and indirectly, an MPI user) is someone who has
science to do that cannot be done on small systems.  Many of the
questions seem directed to people who write MPI applications.

Perhaps putting questions that might apply to HPC users that don't
write MPI code first and telling them their answers to the first few
questions are important even if they do not feel they can respond to
the MPI programmer questions.

This question should not follow lots of questions directed to people
who write MPI apps. It is a critical question for the kind of "user"
we tend to forget - people/businesses with problems to run

I expect to be able to upgrade to an MPI-3 implementation and still be able
to run my legacy MPI applications *without recompiling*.

Questions about the importance of providing a way to use hundreds of
thousands of PEs
effectively should be in the HPC user section. Questions about what must be
done to the
MPI standard in support should be directed to MPI code writers.

>
> > ISVs or "closed source product" providers should be queried. They
> > need to deal with their customers who are delivered an MPI 3
> >
> > Buyers of closed source products are stake holders in this issue. If
> > I am using an MPI based product in my business, how do I feel about
> > needing to buy and validate new software when my MPI provider says
> > his new release breaks my production software?
> >
> > Providers of MPI implementations may want to think about the
> > implications of an influential customer saying "Ship MPI 3 if you
> > want to but to keep my business you must also keep shipping and
> > supporting the MPI 2.2 that my apps still use.
> >
>
> Distilling down your text, are you asking how we plan to get as many
> respondents from as many different categories as possible?
>
> If so, the plan is loosely defined as follows: advertise it heavily at
> the BOF this week.  Further, advertise it as heavily as possible via
> public venues (e.g., open source mailing lists, etc.).  Finally,
> advertise it as heavily as possible with individual contacts (e.g.,
> MPI implementor organizations who have contacts with ISVs and other
> closed source application developers, customers, etc.).
>
> > Typical number of tasks in an MPI job may not be very informative.
> > The question should (also?) try to get at:
> >
> > 1) What is the largest job you tend run today and consider important
> > (vs something run but not critical to the mission / business)
> > 2) What number of tasks do you anticipate wanting to run (and
> > consider important) within the next few years?
> >
> > I do not see anything clearly addressing the question of "How many
> > cores. CPUs. PEs (whatever term) would you hope/expect to be able to
> > apply to your largest problems?"
> >
>
> The rationale for these questions was mainly demographic data
> gathering that can be used to help qualify/quantify further answers
> (e.g., "among respondents who said they run apps at 2049+ MPI procs,
> only 3% use one-sided communications").

OK - it seems to me some of the biggest issues for the future of MPI
are tied up with scaling and with what people hope for or expect in
the future. I strongly suspect that more than  say 32K ranks will be
very hard to do well but using > 32K PEs is coming fast.

Maybe a section on "What you do now" and a section on "what you hope
to do or need to do over the next 5 years" would help.

>
> Do you have suggestions for better wording of those questions, or
> replacement questions?
>
> > Much of the hybrid discussion is about solving problems with so many
> > PEs that MPI cannot scale to enough tasks. It seems likely to me
> > that someone may want to use a million processing elements and that
> > MPI will never scale well to a million tasks.
> >
> > I assume question with the check boxes is asking about "error
> > handlers". It refers to "error handles"
> > http://mpi-forum.questionpro.com/akira/
> TakeSurvey;jsessionid=dcaTO_iOlMbIqcQeW98ts

The question:
How much are each of the following sets of MPI functionality used in your
MPI applications?
mentions "error handles".  If there is anything that could be called an
"error
handle" I suppose it would be the predefined error classes. Are you asking
if
use MPI_Error_class to map an rc to a class or do you mean "do you use
error handlers
vs ERRORS_RETURN vs ERRORS_ARE_FATAL

> >
> > I do not know what this question means.
> >
>
> I'm not quite sure how to parse the above (the link doesn't take me to
> a unique question):
>
> 1. Are you referring to the "How much are each of the following sets
> of MPI functionality used in your MPI applications?" question?
> 2. Are you saying that the phrase should be "Error handlers / error
> checking" (instead of "Error handles")?
> 3. Are you saying that it is unclear what this specific item means?
>
> > MPI one-sided communication performance is more important to me than
> > supporting a rich remote memory access (RMA) feature set.
> >
> > Having been involved in several email discussions and knowing what
> > is being pressed, I can guess the desired answer. We cannot
> > legitimately make decisions based on such an ambiguous question:
> > What expectations of typical MPI communication are included in
> > "rich"? Is using any communicator except MPI_COMM_WORLD part of
> > "rich"? Is getting a non-MPI_SUCCESS return code for an error part
> > of rich?
> >
>
> Keith and I went round and round on this particular question in an off-
> list email thread.  His final suggestion, which I think is a good one,
> is to reword the question thusly:
>
> "MPI one-sided communication performance (e.g., message rate and
> latency) is more important to me than supporting a rich remote memory
> access (RMA) feature set (e.g., communicators, datatypes)."
>
> What do you think?

I think it is so hard to ask a meaningful question to a broad audience on
this topic that we should remove the question. Make the question simple
and get simplistic answers. Make it careful and few will take the time to
understand the implications we try to mention.

>
> > "MPI application control of fault tolerance" -- sounds too easy. Who
> > would say no? I would be happy to have MPI_Tolerate_faults( yes | no)
> >
>
> We agonized over this one as well.  The current text is an attempt to
> imply that MPI won't magically handle faults for you, but rather give
> control of faults to the application.  The implication is that the
> application will have to *do* something rather than have MPI just
> handle faults "better" (which I agree, everyone will just say, "yes,
> gimme that!").
>
> But I can definitely see how a quick read of that would lead to the
> interpretation of what you said.
>
> Can you suggest better wording?

Not really. I am waiting to see what the Fault tolerance group comes up
with but my feeling is that the difficulty of writing fault tolerant
applications and the limitations it will place on general MPI
functionality will restrict its use to a tiny niche market at best. Unless
you can tell the difference between a yes that is based on assuming most
MPI apps can be made fault tolerant with a bit of work vs a yes, I am
willing to work very hard and accept many limitation on what I expect
to be able to do in MPI code, you do not learn much.

>
> > It would be helpful to be able to read the survey questions before
> > starting to answer. I personally dislike answering a survey question
> > without knowing what all the questions are. I must try to pick the
> > best answer even though the question does not really fit my concern
> > or I must ignore my concern on the assumption a better targeted
> > question must be on the survey. (I cannot count the number of times
> > I have started to answer a telephone survey only to say in the
> > middle "We're done" because I have deduced the survey questions will
> > not really help capture my views")
> >

I suppose a note at the intro telling people they can flip through and
look at the questions before starting will be enough.

In this question:
Rate the following in order of importance to your MPI applications

what does performance mean? "hours to solution" and "$$$ to solution"
are both very important. Best case MPI_New_put latency may not tell you
much
about either.

New functions that let libmpi avoid hogging memory can certainly help
"$$$ to solution" but most people would not think of them as a performance
issue.

Ideas that reduce the impact of jitter at scale are clearly important
to both "hours to solution" and "$$$ to solution" and may be far more
important in future than what people often call "performance".


>
>
> Josh and I talked about this quite a bit while putting the questions
> in the survey web thingy.  We balanced your concerns against splitting
> up the survey into seemingly-smaller chunks so that we don't
> intimidate respondents from abandoning the survey in the middle.  In
> the end, we struck a compromise:
>
> - split the survey up into multiple pages so that it looks/feels
> shorter (vs. one, massively-long page)
> - don't use data validation rules
> - provide navigation links in the survey (forward and back)
>
> The latter 2 points allow you to click click click through the entire
> survey, reading everything without entering any answers.  Then, when
> you've read everything, you can navigate back (or just restart) and
> actually take the survey.
>
> That was the best compromise we could think of.  Do you think we
> should have one big, massively-long page with all the questions?
>
> --
> Jeff Squyres
> jsquyres at cisco.com
>
> _______________________________________________
> mpi-forum mailing list
> mpi-forum at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpi-forum/attachments/20091116/2034676d/attachment-0001.html>