[Mpi3-rma] [Mpi-forum] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?

Sun Jul 22 13:24:25 CDT 2012

Good point - but what this tells me is that we should leave this out - its something that isn't necessary in specifying the standard, and enumerating the cases where it might apply is non-trivial.

Bill

William Gropp
Director, Parallel Computing Institute
Deputy Director for Research
Institute for Advanced Computing Applications and Technologies
Paul and Cynthia Saylor Professor of Computer Science
University of Illinois Urbana-Champaign

On Jul 22, 2012, at 1:12 PM, Jeff Hammond wrote:

> I think "s/at extreme scale/when memory is limited/".  An adapteva
> chip has 32KB SRAM per core and is pushing 100 cores and farther, so
> having >128*4 bytes used in the window object is no different than
> having >1 of 64 GB used for the window if one has 2^29 ranks.
> 
> Jeff
> 
> On Sun, Jul 22, 2012 at 1:03 PM, William Gropp <wgropp at illinois.edu> wrote:
>> I think this is a little strong.  This isn't necessary for the vast majority of MPI users; just for those at extreme scale.  A modification that restricted this to extreme scale might be acceptable.
>> 
>> Bill
>> 
>> William Gropp
>> Director, Parallel Computing Institute
>> Deputy Director for Research
>> Institute for Advanced Computing Applications and Technologies
>> Paul and Cynthia Saylor Professor of Computer Science
>> University of Illinois Urbana-Champaign
>> 
>> 
>> 
>> On Jul 22, 2012, at 12:28 PM, Rolf Rabenseifner wrote:
>> 
>>> On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote:
>>>> I'm concerned with the change (and don't think it should happen
>>>> at this point, as I want to be able to tell other people their
>>>> changes are too big).  It's probably not another O(p) structure,
>>>> since a process will have to know the other rank's disp_unit to
>>>> do the memcpy during the communication operations, so we crossed
>>>> the O(p) bridge when we added disp_unit to allocate_shared.
>>> 
>>> I would recommend to add two advices:
>>> 
>>> Advice to users. For scalability reasons it is recommended
>>> that all processes provide the same disp_unit argument
>>> on all processes. End of advice to users.
>>> 
>>> Advice to implementors. If the user provides the same
>>> disp_unit argument on all processes of the communicator,
>>> or a compressible list of values, it is necessary for
>>> scalability reasons to store this data as compact as possible.
>>> End of advice to users.
>>> 
>>> Of course for all thre MPI_WIN_CREATE, MPI_WIN_AALOCATE, and
>>> MPI_WIN_ALLOCATE_SHARED.
>>> 
>>> Is this a good idea?
>>> If yes, can someone find a better wording for this proposal?
>>> 
>>> Best regards
>>> Rolf
>>> 
>>> ----- Original Message -----
>>>> From: "William Gropp" <wgropp at illinois.edu>
>>>> To: "MPI 3.0 Remote Memory Access working group" <mpi3-rma at lists.mpi-forum.org>
>>>> Sent: Sunday, July 22, 2012 7:12:27 PM
>>>> Subject: Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
>>>> What *exactly* are the proposals? I know in general, but some of the
>>>> statements make me wonder if I understand the proposal.
>>>> 
>>>> Bill
>>>> 
>>>> William Gropp
>>>> Director, Parallel Computing Institute
>>>> Deputy Director for Research
>>>> Institute for Advanced Computing Applications and Technologies
>>>> Paul and Cynthia Saylor Professor of Computer Science
>>>> University of Illinois Urbana-Champaign
>>>> 
>>>> 
>>>> 
>>>> On Jul 22, 2012, at 10:55 AM, Torsten Hoefler wrote:
>>>> 
>>>>> Dear All,
>>>>> 
>>>>> I think we should discuss this more. Right now, we still have more
>>>>> votes
>>>>> in favor of this than against. The change is still committed and
>>>>> part of
>>>>> the "official release". The deadline is 7/26.
>>>>> 
>>>>> Please advise what to do!
>>>>> 
>>>>> Thanks,
>>>>> Torsten
>>>>> 
>>>>> On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote:
>>>>>> I'm concerned with the change (and don't think it should happen at
>>>>>> this point, as I want to be able to tell other people their changes
>>>>>> are too big). It's probably not another O(p) structure, since a
>>>>>> process will have to know the other rank's disp_unit to do the
>>>>>> memcpy during the communication operations, so we crossed the O(p)
>>>>>> bridge when we added disp_unit to allocate_shared.
>>>>>> 
>>>>>> 
>>>>>> Brian
>>>>>> 
>>>>>> --
>>>>>> Brian W. Barrett
>>>>>> Scalable System Software Group
>>>>>> Sandia National Laboratories
>>>>>> ________________________________________
>>>>>> From: mpi3-rma-bounces at lists.mpi-forum.org
>>>>>> [mpi3-rma-bounces at lists.mpi-forum.org] on behalf of Jim Dinan
>>>>>> [dinan at mcs.anl.gov]
>>>>>> Sent: Wednesday, July 18, 2012 12:46 PM
>>>>>> To: mpi3-rma at lists.mpi-forum.org
>>>>>> Subject: [EXTERNAL] Re: [Mpi3-rma] Disp_unit in allocate_shared
>>>>>> should be returned by win_shared_query?
>>>>>> 
>>>>>> It's not inherently damaging, but it requires us to
>>>>>> maintain/provide
>>>>>> access to another O(p) structure and suggests to users that another
>>>>>> process' disp_unit is useful information, which is incorrect. IIRC,
>>>>>> we
>>>>>> did discuss this in the working group and intentionally omitted it.
>>>>>> 
>>>>>> ~Jim.
>>>>>> 
>>>>>> On 7/18/12 1:38 PM, Torsten Hoefler wrote:
>>>>>>> On Wed, Jul 18, 2012 at 01:18:53PM -0500, James Dinan wrote:
>>>>>>>> On 7/18/12 1:06 PM, Torsten Hoefler wrote:
>>>>>>>>> On Wed, Jul 18, 2012 at 01:08:43PM -0500, Pavan Balaji wrote:
>>>>>>>>>> 
>>>>>>>>>> On 07/18/2012 01:02 PM, Torsten Hoefler wrote:
>>>>>>>>>>> Jim,
>>>>>>>>>>>> Is one process' disp_unit meaningful at another process? I
>>>>>>>>>>>> thought that
>>>>>>>>>>>> this is not the case. MPI provides no other routines to query
>>>>>>>>>>>> another
>>>>>>>>>>>> process' disp_unit.
>>>>>>>>>>> It is something that the user is passing in. I don't know of a
>>>>>>>>>>> particular use-case but we can query all other arguments that
>>>>>>>>>>> each
>>>>>>>>>>> process passed. One more integer seemed safe to me.
>>>>>>>>>>> 
>>>>>>>>>>>> Also, what meaning does a disp_unit have in the context of
>>>>>>>>>>>> load/store
>>>>>>>>>>>> operations that will result from win_shared_query?
>>>>>>>>>>> Nothing.
>>>>>>>>>>> 
>>>>>>>>>>>> Maybe I'm missing something, but this doesn't seem necessary.
>>>>>>>>>>> Well, it's better to have it than to introduce it with a
>>>>>>>>>>> different name
>>>>>>>>>>> later. That was my main motivation. One could also use it in a
>>>>>>>>>>> library
>>>>>>>>>>> context to query. But that's weak, I know.
>>>>>>>>>> 
>>>>>>>>>> This doesn't seem critical, and can always be added in MPI-3.1
>>>>>>>>>> if we
>>>>>>>>>> find it necessary. No reason to rush it.
>>>>>>>>> 
>>>>>>>>> It cannot be added without a new function name. That was the
>>>>>>>>> concern
>>>>>>>>> here. Adding it seems less risk than not adding it. In addition,
>>>>>>>>> it is
>>>>>>>>> already added ;-).
>>>>>>>> 
>>>>>>>> I am concerned that this is a harmful change. The disp_unit from
>>>>>>>> one
>>>>>>>> process has no meaningful use at another process. If the user
>>>>>>>> wants to
>>>>>>>> know it for some reason, they can use a separate communication
>>>>>>>> operation.
>>>>>>> I believe the potential damage is very limited. But we can have a
>>>>>>> vote.
>>>>>>> 
>>>>>>> So far it's still yes: 5, no: 1
>>>>>>> 
>>>>>>> I'm just mediating, not making decisions.
>>>>>>> 
>>>>>>> Torsten
>>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> mpi3-rma mailing list
>>>>>> mpi3-rma at lists.mpi-forum.org
>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> mpi3-rma mailing list
>>>>>> mpi3-rma at lists.mpi-forum.org
>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>>>>> 
>>>>> 
>>>>> --
>>>>> ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/
>>>>> -----
>>>>> Torsten Hoefler | Performance Modeling and Simulation Lead
>>>>> Blue Waters Directorate | University of Illinois (UIUC)
>>>>> 1205 W Clark Street | Urbana, IL, 61801
>>>>> NCSA Building | +01 (217) 244-7736
>>>>> _______________________________________________
>>>>> mpi3-rma mailing list
>>>>> mpi3-rma at lists.mpi-forum.org
>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>>> 
>>>> 
>>>> _______________________________________________
>>>> mpi3-rma mailing list
>>>> mpi3-rma at lists.mpi-forum.org
>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>> 
>>> --
>>> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de
>>> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530
>>> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832
>>> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner
>>> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)
>>> _______________________________________________
>>> mpi-forum mailing list
>>> mpi-forum at lists.mpi-forum.org
>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>> 
>> 
>> _______________________________________________
>> mpi3-rma mailing list
>> mpi3-rma at lists.mpi-forum.org
>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
> 
> 
> 
> -- 
> Jeff Hammond
> Argonne Leadership Computing Facility
> University of Chicago Computation Institute
> jhammond at alcf.anl.gov / (630) 252-5381
> http://www.linkedin.com/in/jeffhammond
> https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond
> _______________________________________________
> mpi-forum mailing list
> mpi-forum at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum