[Mpi3-rma] [Mpi-forum] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?

Sun Jul 22 13:12:51 CDT 2012

I think "s/at extreme scale/when memory is limited/".  An adapteva
chip has 32KB SRAM per core and is pushing 100 cores and farther, so
having >128*4 bytes used in the window object is no different than
having >1 of 64 GB used for the window if one has 2^29 ranks.

Jeff

On Sun, Jul 22, 2012 at 1:03 PM, William Gropp <wgropp at illinois.edu> wrote:
> I think this is a little strong.  This isn't necessary for the vast majority of MPI users; just for those at extreme scale.  A modification that restricted this to extreme scale might be acceptable.
>
> Bill
>
> William Gropp
> Director, Parallel Computing Institute
> Deputy Director for Research
> Institute for Advanced Computing Applications and Technologies
> Paul and Cynthia Saylor Professor of Computer Science
> University of Illinois Urbana-Champaign
>
>
>
> On Jul 22, 2012, at 12:28 PM, Rolf Rabenseifner wrote:
>
>> On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote:
>>> I'm concerned with the change (and don't think it should happen
>>> at this point, as I want to be able to tell other people their
>>> changes are too big).  It's probably not another O(p) structure,
>>> since a process will have to know the other rank's disp_unit to
>>> do the memcpy during the communication operations, so we crossed
>>> the O(p) bridge when we added disp_unit to allocate_shared.
>>
>> I would recommend to add two advices:
>>
>>  Advice to users. For scalability reasons it is recommended
>>  that all processes provide the same disp_unit argument
>>  on all processes. End of advice to users.
>>
>>  Advice to implementors. If the user provides the same
>>  disp_unit argument on all processes of the communicator,
>>  or a compressible list of values, it is necessary for
>>  scalability reasons to store this data as compact as possible.
>>  End of advice to users.
>>
>> Of course for all thre MPI_WIN_CREATE, MPI_WIN_AALOCATE, and
>> MPI_WIN_ALLOCATE_SHARED.
>>
>> Is this a good idea?
>> If yes, can someone find a better wording for this proposal?
>>
>> Best regards
>> Rolf
>>
>> ----- Original Message -----
>>> From: "William Gropp" <wgropp at illinois.edu>
>>> To: "MPI 3.0 Remote Memory Access working group" <mpi3-rma at lists.mpi-forum.org>
>>> Sent: Sunday, July 22, 2012 7:12:27 PM
>>> Subject: Re: [Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?
>>> What *exactly* are the proposals? I know in general, but some of the
>>> statements make me wonder if I understand the proposal.
>>>
>>> Bill
>>>
>>> William Gropp
>>> Director, Parallel Computing Institute
>>> Deputy Director for Research
>>> Institute for Advanced Computing Applications and Technologies
>>> Paul and Cynthia Saylor Professor of Computer Science
>>> University of Illinois Urbana-Champaign
>>>
>>>
>>>
>>> On Jul 22, 2012, at 10:55 AM, Torsten Hoefler wrote:
>>>
>>>> Dear All,
>>>>
>>>> I think we should discuss this more. Right now, we still have more
>>>> votes
>>>> in favor of this than against. The change is still committed and
>>>> part of
>>>> the "official release". The deadline is 7/26.
>>>>
>>>> Please advise what to do!
>>>>
>>>> Thanks,
>>>> Torsten
>>>>
>>>> On Wed, Jul 18, 2012 at 08:11:23PM +0000, Barrett, Brian W wrote:
>>>>> I'm concerned with the change (and don't think it should happen at
>>>>> this point, as I want to be able to tell other people their changes
>>>>> are too big). It's probably not another O(p) structure, since a
>>>>> process will have to know the other rank's disp_unit to do the
>>>>> memcpy during the communication operations, so we crossed the O(p)
>>>>> bridge when we added disp_unit to allocate_shared.
>>>>>
>>>>>
>>>>> Brian
>>>>>
>>>>> --
>>>>> Brian W. Barrett
>>>>> Scalable System Software Group
>>>>> Sandia National Laboratories
>>>>> ________________________________________
>>>>> From: mpi3-rma-bounces at lists.mpi-forum.org
>>>>> [mpi3-rma-bounces at lists.mpi-forum.org] on behalf of Jim Dinan
>>>>> [dinan at mcs.anl.gov]
>>>>> Sent: Wednesday, July 18, 2012 12:46 PM
>>>>> To: mpi3-rma at lists.mpi-forum.org
>>>>> Subject: [EXTERNAL] Re: [Mpi3-rma] Disp_unit in allocate_shared
>>>>> should be returned by win_shared_query?
>>>>>
>>>>> It's not inherently damaging, but it requires us to
>>>>> maintain/provide
>>>>> access to another O(p) structure and suggests to users that another
>>>>> process' disp_unit is useful information, which is incorrect. IIRC,
>>>>> we
>>>>> did discuss this in the working group and intentionally omitted it.
>>>>>
>>>>> ~Jim.
>>>>>
>>>>> On 7/18/12 1:38 PM, Torsten Hoefler wrote:
>>>>>> On Wed, Jul 18, 2012 at 01:18:53PM -0500, James Dinan wrote:
>>>>>>> On 7/18/12 1:06 PM, Torsten Hoefler wrote:
>>>>>>>> On Wed, Jul 18, 2012 at 01:08:43PM -0500, Pavan Balaji wrote:
>>>>>>>>>
>>>>>>>>> On 07/18/2012 01:02 PM, Torsten Hoefler wrote:
>>>>>>>>>> Jim,
>>>>>>>>>>> Is one process' disp_unit meaningful at another process? I
>>>>>>>>>>> thought that
>>>>>>>>>>> this is not the case. MPI provides no other routines to query
>>>>>>>>>>> another
>>>>>>>>>>> process' disp_unit.
>>>>>>>>>> It is something that the user is passing in. I don't know of a
>>>>>>>>>> particular use-case but we can query all other arguments that
>>>>>>>>>> each
>>>>>>>>>> process passed. One more integer seemed safe to me.
>>>>>>>>>>
>>>>>>>>>>> Also, what meaning does a disp_unit have in the context of
>>>>>>>>>>> load/store
>>>>>>>>>>> operations that will result from win_shared_query?
>>>>>>>>>> Nothing.
>>>>>>>>>>
>>>>>>>>>>> Maybe I'm missing something, but this doesn't seem necessary.
>>>>>>>>>> Well, it's better to have it than to introduce it with a
>>>>>>>>>> different name
>>>>>>>>>> later. That was my main motivation. One could also use it in a
>>>>>>>>>> library
>>>>>>>>>> context to query. But that's weak, I know.
>>>>>>>>>
>>>>>>>>> This doesn't seem critical, and can always be added in MPI-3.1
>>>>>>>>> if we
>>>>>>>>> find it necessary. No reason to rush it.
>>>>>>>>
>>>>>>>> It cannot be added without a new function name. That was the
>>>>>>>> concern
>>>>>>>> here. Adding it seems less risk than not adding it. In addition,
>>>>>>>> it is
>>>>>>>> already added ;-).
>>>>>>>
>>>>>>> I am concerned that this is a harmful change. The disp_unit from
>>>>>>> one
>>>>>>> process has no meaningful use at another process. If the user
>>>>>>> wants to
>>>>>>> know it for some reason, they can use a separate communication
>>>>>>> operation.
>>>>>> I believe the potential damage is very limited. But we can have a
>>>>>> vote.
>>>>>>
>>>>>> So far it's still yes: 5, no: 1
>>>>>>
>>>>>> I'm just mediating, not making decisions.
>>>>>>
>>>>>> Torsten
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> mpi3-rma mailing list
>>>>> mpi3-rma at lists.mpi-forum.org
>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> mpi3-rma mailing list
>>>>> mpi3-rma at lists.mpi-forum.org
>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>>>>
>>>>
>>>> --
>>>> ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/
>>>> -----
>>>> Torsten Hoefler | Performance Modeling and Simulation Lead
>>>> Blue Waters Directorate | University of Illinois (UIUC)
>>>> 1205 W Clark Street | Urbana, IL, 61801
>>>> NCSA Building | +01 (217) 244-7736
>>>> _______________________________________________
>>>> mpi3-rma mailing list
>>>> mpi3-rma at lists.mpi-forum.org
>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>>
>>>
>>> _______________________________________________
>>> mpi3-rma mailing list
>>> mpi3-rma at lists.mpi-forum.org
>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>
>> --
>> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de
>> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530
>> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832
>> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner
>> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)
>> _______________________________________________
>> mpi-forum mailing list
>> mpi-forum at lists.mpi-forum.org
>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>
>
> _______________________________________________
> mpi3-rma mailing list
> mpi3-rma at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

-- 
Jeff Hammond
Argonne Leadership Computing Facility
University of Chicago Computation Institute
jhammond at alcf.anl.gov / (630) 252-5381
http://www.linkedin.com/in/jeffhammond
https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond