[Mpi-forum] [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?

Sun Jul 22 13:30:26 CDT 2012

Torsten and Bill,

Torsten:
> I would agree to Bill. We should have an info argument allowing the user
> to state that all displ_units are the same like we have for count :-).
> A topic for MPI-3.1!

Yes, it is not needed for MPI-3.0, because with two Allreduces with 
MIN and MAX of disp_unit, all processes know in O(log(p)) whether 
it is the same value in all processes.

Bill, yes, here a draft:

  Advice to users. If an application should scale to a large 
  number of processes, then it is recommended for scalability reasons 
  that all processes provide the same disp_unit argument
  on all processes. End of advice to users.

>>>  Advice to implementors. If the user provides the same
>>>  disp_unit argument on all processes of the communicator,
     or a compressible list of values, it is recommended for
>>>  scalability reasons to store this data as compact as possible.
>>>  End of advice to users.

(I substituted necessary by recommended)

Is this text now better?

Best regards
Rolf

----- Original Message -----
> From: "Torsten Hoefler" <htor at illinois.edu>
> To: "Main MPI Forum mailing list" <mpi-forum at lists.mpi-forum.org>
> Cc: "MPI 3.0 Remote Memory Access working group" <mpi3-rma at lists.mpi-forum.org>
> Sent: Sunday, July 22, 2012 8:06:55 PM
> Subject: Re: [Mpi-forum] [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by
> win_shared_query?
> I would agree to Bill. We should have an info argument allowing the
> user
> to state that all displ_units are the same like we have for count :-).
> A
> topic for MPI-3.1!
> 
> Thanks,
> Torsten
> 
> On Sun, Jul 22, 2012 at 01:03:39PM -0500, William Gropp wrote:
> > I think this is a little strong. This isn't necessary for the vast
> > majority of MPI users; just for those at extreme scale. A
> > modification that restricted this to extreme scale might be
> > acceptable.
> >
> > Bill
> >
> > William Gropp
> > Director, Parallel Computing Institute
> > Deputy Director for Research
> > Institute for Advanced Computing Applications and Technologies
> > Paul and Cynthia Saylor Professor of Computer Science
> > University of Illinois Urbana-Champaign
> >
> >
> >
> > On Jul 22, 2012, at 12:28 PM, Rolf Rabenseifner wrote:
> >
> > > On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote:
> > >> I'm concerned with the change (and don't think it should happen
> > >> at this point, as I want to be able to tell other people their
> > >> changes are too big). It's probably not another O(p) structure,
> > >> since a process will have to know the other rank's disp_unit to
> > >> do the memcpy during the communication operations, so we crossed
> > >> the O(p) bridge when we added disp_unit to allocate_shared.
> > >
> > > I would recommend to add two advices:
> > >
> > >  Advice to users. For scalability reasons it is recommended
> > >  that all processes provide the same disp_unit argument
> > >  on all processes. End of advice to users.
> > >
> > >  Advice to implementors. If the user provides the same
> > >  disp_unit argument on all processes of the communicator,
> > >  or a compressible list of values, it is necessary for
> > >  scalability reasons to store this data as compact as possible.
> > >  End of advice to users.
> > >
> > > Of course for all thre MPI_WIN_CREATE, MPI_WIN_AALOCATE, and
> > > MPI_WIN_ALLOCATE_SHARED.
> > >
> > > Is this a good idea?
> > > If yes, can someone find a better wording for this proposal?
> > >
> > > Best regards
> > > Rolf
> > >
> > > ----- Original Message -----
> > >> From: "William Gropp" <wgropp at illinois.edu>
> > >> To: "MPI 3.0 Remote Memory Access working group"
> > >> <mpi3-rma at lists.mpi-forum.org>
> > >> Sent: Sunday, July 22, 2012 7:12:27 PM
> > >> Subject: Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in
> > >> allocate_shared should be returned by win_shared_query?
> > >> What *exactly* are the proposals? I know in general, but some of
> > >> the
> > >> statements make me wonder if I understand the proposal.
> > >>
> > >> Bill
> > >>
> > >> William Gropp
> > >> Director, Parallel Computing Institute
> > >> Deputy Director for Research
> > >> Institute for Advanced Computing Applications and Technologies
> > >> Paul and Cynthia Saylor Professor of Computer Science
> > >> University of Illinois Urbana-Champaign
> > >>
> > >>
> > >>
> > >> On Jul 22, 2012, at 10:55 AM, Torsten Hoefler wrote:
> > >>
> > >>> Dear All,
> > >>>
> > >>> I think we should discuss this more. Right now, we still have
> > >>> more
> > >>> votes
> > >>> in favor of this than against. The change is still committed and
> > >>> part of
> > >>> the "official release". The deadline is 7/26.
> > >>>
> > >>> Please advise what to do!
> > >>>
> > >>> Thanks,
> > >>> Torsten
> > >>>
> > >>> On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W
> > >>> wrote:
> > >>>> I'm concerned with the change (and don't think it should happen
> > >>>> at
> > >>>> this point, as I want to be able to tell other people their
> > >>>> changes
> > >>>> are too big). It's probably not another O(p) structure, since a
> > >>>> process will have to know the other rank's disp_unit to do the
> > >>>> memcpy during the communication operations, so we crossed the
> > >>>> O(p)
> > >>>> bridge when we added disp_unit to allocate_shared.
> > >>>>
> > >>>>
> > >>>> Brian
> > >>>>
> > >>>> --
> > >>>> Brian W. Barrett
> > >>>> Scalable System Software Group
> > >>>> Sandia National Laboratories
> > >>>> ________________________________________
> > >>>> From: mpi3-rma-bounces at lists.mpi-forum.org
> > >>>> [mpi3-rma-bounces at lists.mpi-forum.org] on behalf of Jim Dinan
> > >>>> [dinan at mcs.anl.gov]
> > >>>> Sent: Wednesday, July 18, 2012 12:46 PM
> > >>>> To: mpi3-rma at lists.mpi-forum.org
> > >>>> Subject: [EXTERNAL] Re: [Mpi3-rma] Disp_unit in allocate_shared
> > >>>> should be returned by win_shared_query?
> > >>>>
> > >>>> It's not inherently damaging, but it requires us to
> > >>>> maintain/provide
> > >>>> access to another O(p) structure and suggests to users that
> > >>>> another
> > >>>> process' disp_unit is useful information, which is incorrect.
> > >>>> IIRC,
> > >>>> we
> > >>>> did discuss this in the working group and intentionally omitted
> > >>>> it.
> > >>>>
> > >>>> ~Jim.
> > >>>>
> > >>>> On 7/18/12 1:38 PM, Torsten Hoefler wrote:
> > >>>>> On Wed, Jul 18, 2012 at 01:18:53PM -0500, James Dinan wrote:
> > >>>>>> On 7/18/12 1:06 PM, Torsten Hoefler wrote:
> > >>>>>>> On Wed, Jul 18, 2012 at 01:08:43PM -0500, Pavan Balaji
> > >>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>> On 07/18/2012 01:02 PM, Torsten Hoefler wrote:
> > >>>>>>>>> Jim,
> > >>>>>>>>>> Is one process' disp_unit meaningful at another process?
> > >>>>>>>>>> I
> > >>>>>>>>>> thought that
> > >>>>>>>>>> this is not the case. MPI provides no other routines to
> > >>>>>>>>>> query
> > >>>>>>>>>> another
> > >>>>>>>>>> process' disp_unit.
> > >>>>>>>>> It is something that the user is passing in. I don't know
> > >>>>>>>>> of a
> > >>>>>>>>> particular use-case but we can query all other arguments
> > >>>>>>>>> that
> > >>>>>>>>> each
> > >>>>>>>>> process passed. One more integer seemed safe to me.
> > >>>>>>>>>
> > >>>>>>>>>> Also, what meaning does a disp_unit have in the context
> > >>>>>>>>>> of
> > >>>>>>>>>> load/store
> > >>>>>>>>>> operations that will result from win_shared_query?
> > >>>>>>>>> Nothing.
> > >>>>>>>>>
> > >>>>>>>>>> Maybe I'm missing something, but this doesn't seem
> > >>>>>>>>>> necessary.
> > >>>>>>>>> Well, it's better to have it than to introduce it with a
> > >>>>>>>>> different name
> > >>>>>>>>> later. That was my main motivation. One could also use it
> > >>>>>>>>> in a
> > >>>>>>>>> library
> > >>>>>>>>> context to query. But that's weak, I know.
> > >>>>>>>>
> > >>>>>>>> This doesn't seem critical, and can always be added in
> > >>>>>>>> MPI-3.1
> > >>>>>>>> if we
> > >>>>>>>> find it necessary. No reason to rush it.
> > >>>>>>>
> > >>>>>>> It cannot be added without a new function name. That was the
> > >>>>>>> concern
> > >>>>>>> here. Adding it seems less risk than not adding it. In
> > >>>>>>> addition,
> > >>>>>>> it is
> > >>>>>>> already added ;-).
> > >>>>>>
> > >>>>>> I am concerned that this is a harmful change. The disp_unit
> > >>>>>> from
> > >>>>>> one
> > >>>>>> process has no meaningful use at another process. If the user
> > >>>>>> wants to
> > >>>>>> know it for some reason, they can use a separate
> > >>>>>> communication
> > >>>>>> operation.
> > >>>>> I believe the potential damage is very limited. But we can
> > >>>>> have a
> > >>>>> vote.
> > >>>>>
> > >>>>> So far it's still yes: 5, no: 1
> > >>>>>
> > >>>>> I'm just mediating, not making decisions.
> > >>>>>
> > >>>>> Torsten
> > >>>>>
> > >>>>
> > >>>> _______________________________________________
> > >>>> mpi3-rma mailing list
> > >>>> mpi3-rma at lists.mpi-forum.org
> > >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
> > >>>>
> > >>>>
> > >>>>
> > >>>> _______________________________________________
> > >>>> mpi3-rma mailing list
> > >>>> mpi3-rma at lists.mpi-forum.org
> > >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
> > >>>>
> > >>>
> > >>> --
> > >>> ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/
> > >>> -----
> > >>> Torsten Hoefler | Performance Modeling and Simulation Lead
> > >>> Blue Waters Directorate | University of Illinois (UIUC)
> > >>> 1205 W Clark Street | Urbana, IL, 61801
> > >>> NCSA Building | +01 (217) 244-7736
> > >>> _______________________________________________
> > >>> mpi3-rma mailing list
> > >>> mpi3-rma at lists.mpi-forum.org
> > >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
> > >>
> > >>
> > >> _______________________________________________
> > >> mpi3-rma mailing list
> > >> mpi3-rma at lists.mpi-forum.org
> > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
> > >
> > > --
> > > Dr. Rolf Rabenseifner . . . . . . . . . .. email
> > > rabenseifner at hlrs.de
> > > High Performance Computing Center (HLRS) . phone
> > > ++49(0)711/685-65530
> > > University of Stuttgart . . . . . . . . .. fax ++49(0)711 /
> > > 685-65832
> > > Head of Dpmt Parallel Computing . . .
> > > www.hlrs.de/people/rabenseifner
> > > Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring
> > > 30)
> > > _______________________________________________
> > > mpi-forum mailing list
> > > mpi-forum at lists.mpi-forum.org
> > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
> >
> >
> > _______________________________________________
> > mpi-forum mailing list
> > mpi-forum at lists.mpi-forum.org
> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
> >
> 
> --
> ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ -----
> Torsten Hoefler | Performance Modeling and Simulation Lead
> Blue Waters Directorate | University of Illinois (UIUC)
> 1205 W Clark Street | Urbana, IL, 61801
> NCSA Building | +01 (217) 244-7736
> _______________________________________________
> mpi-forum mailing list
> mpi-forum at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

-- 
Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de
High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530
University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832
Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner
Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)