[Mpi3-rma] [EXTERNAL] Re: Disp_unit in allocate_shared should be returned by win_shared_query?

[Jim Dinan]

I do have a concern that different disp_units can occur unexpectedly 
because of heterogeneous hardware or software -- e.g. MPMD launch with 
different executables or language incompatibility where both C and 
Fortran code are involved in creating the window.

Rather than ruling these cases out (option #1), how about including the 
following advice to users:

Advice to users.  If disp_unit values differ across processes in a 
shared memory window, the programmer may need to use the target's 
disp_unit value in address calculations and use different load/store 
operations depending on the target's disp_unit value. (End of advice to 
users.)

If we included this, I think I would be ok with option #2.

An orthogonal observation -- it would have given us a lot more 
flexibility to define MPI_Win_shared_query as:

int MPI_Win_shared_get_attr(MPI_Win win, int target_rank, int 
win_keyval, void *attribute_val, int *flag)

  ~Jim.

On 7/26/12 9:29 AM, William Gropp wrote:
> Thanks for the support :)
>
> I too vote for #2.  I note that applications do not need to use the disp_unit in accessing data under this model, but will need it if they are acting to replace PUT etc (as they might if the code is designed to work with and without shared memory).
>
> Bill
>
> William Gropp
> Director, Parallel Computing Institute
> Deputy Director for Research
> Institute for Advanced Computing Applications and Technologies
> Paul and Cynthia Saylor Professor of Computer Science
> University of Illinois Urbana-Champaign
>
>
>
> On Jul 25, 2012, at 2:16 PM, Torsten Hoefler wrote:
>
>> Hi Jim,
>>
>> Bill's argument convinced me, I think we should have the displ_unit in
>> the query function because this would support future systems and comes
>> at essentially no cost. If the unit is the same on all processes, it can
>> be stored. This can be determined in O(log(P)) or O(1) time with an info
>> argument. Compression is trivial.
>>
>>> We should try to reach a decision on this issue before the end of the
>>> week.  I think it would be helpful to hear some other opinions.  So far,
>>> we have three options to choose from:
>>>
>>> 1) Require that the displacement unit be the same at all processes for
>>> MPI_Win_allocate_shared().
>> This would never allow us to run across heterogeneous systems. I don't
>> know any heterogeneous system that supports the unified model but GPUs
>> are coming :-). I agree to Jim that coherency would be tricky, so this
>> hardware is unlikely. We could also always lift this restriction in
>> future MPI versions, so I'm not opposed.
>>
>>> 2) Allow different displacement units at each process in shared memory
>>> windows and allow processes to determine the displacement unit at remote
>>> processes via MPI_Win_shared_query().  This is in addition to the MPI-2
>>> self-query already possible with MPI_Win_get_attr(MPI_WIN_DISP_UNIT).
>> Yes, this is what we have at this point.
>>
>>> 3) Allow different displacement units at each process in shared memory
>>> windows and provide only the MPI-2 self-query already possible with
>>> MPI_Win_get_attr(MPI_WIN_DISP_UNIT).  Manual communication would be
>>> needed to exchange disp_unit information.
>> This would require a wording change.
>>
>>> A (very) rough summary of the discussion on this issue:
>>>
>>> * Given that non-uniform disp_unit values are currently allowed,
>>> processes may need to use disp_unit in address calculations.
>> They may check if all processes' disp_unit is the same, but that's
>> another complication in the interface.
>>
>>> * Requiring programmers to use the disp_unit at each process in address
>>> calculations diminishes the usefulness of the shared memory programming
>>> model.  It's also not clear if there is a use case to support the
>>> non-uniform disp_unit model.
>> Agreed.
>>
>> I vote for #2.
>>
>> Best,
>>   Torsten
>>
>> --
>> ### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ -----
>> Torsten Hoefler         | Performance Modeling and Simulation Lead
>> Blue Waters Directorate | University of Illinois (UIUC)
>> 1205 W Clark Street     | Urbana, IL, 61801
>> NCSA Building           | +01 (217) 244-7736
>> _______________________________________________
>> mpi3-rma mailing list
>> mpi3-rma at lists.mpi-forum.org
>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>
>
> _______________________________________________
> mpi3-rma mailing list
> mpi3-rma at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>