[Mpi3-rma] [EXTERNAL] Re: Ticket for WG discussion
Jim Dinan
dinan at mcs.anl.gov
Mon Apr 8 16:30:03 CDT 2013
Hi Hubert,
Sorry it has taken me a long time to respond. I talked with Dave
Goodell out here and he has (I think successfully) sorted me out with
respect to how MPI_Aint addresses relative to different MPI_BOTTOM
values work when passed around. So, I apologize for headaches relating
to my misunderstandings of how these work.
The function we would like to propose is what was originally discussed:
MPI_Aint MPI_Aint_add(MPI_Aint base, int disp)
This function produces a new MPI_Aint value that is equal to base +
disp. "base" may be relative to a non-zero value of MPI_BOTTOM that is
unknown at the process performing the call to MPI_Aint_add. Therefore,
the addition is performed in a manner that results in the correct
address, as if the process that originally produced "base" had called:
MPI_Get_address((char *) base + disp, &out_addr)
Interested to hear your thoughts on this.
Cheers,
~Jim.
On 3/15/13 11:57 AM, Hubert Ritzdorf wrote:
> Hi,
>
> I think,
> several kinds of addresses and displacements to MPI_BOTTOM are currently mixed up or
> we are discussing in different terminologies (or I missed some information).
>
> After attaching memory, one process can sent AINT's (displacement to MPI_BOTTOM) to other processes
> and other processes can use these AINT's (displacements) in order to perform one sided operations:
>
>
> Process A: Process B
>
> MPI_Win_create_dynamic (info, comm, win) MPI_Win_create_dynamic (info, comm, win)
>
> base_a = MALLOC (size_a) base_b = MALLOC (size_b)
> MPI_Win_attach (win, base_a, size_a); MPI_Win_attach (win, base_b, size_b)
>
> // Get Displacement relative to MPI_BOTTOM(A) // Get Displacement relative to MPI_BOTTOM(B)
> MPI_Get_address (base_a, &disp_a); MPI_Get_address (base_b, &disp_b);
>
> // Recv displacement from Process B // Recv displacement from A
> MPI_Irecv (&disp_b, 1, MPI_Aint, ...) MPI_Irecv (&disp_a, 1, MPI_Aint, ...)
>
> MPI_Send (&disp_a, 1, MPI_Aint, ...) MPI_Send (&disp_b, 1, MPI_Aint, ....)
> MPI_Wait () MPI_Wait ()
>
>
> MPI_Put (origin_addr_a, .... MPI_Put (origin_addr_b, ...,
> B, disp_b, count_a, type_a, win) A, disp_a, count_b, type_b, win);
>
> This work's in C and Fortran, if MPI_Aint is sufficiently large.
>
> Your original proposal doesn't work for Fortran :
>
> Process A:
>
> disp2 = MPI_Aint_index (base_b, index2)
>
> MPI_Put (origin_addr_a2, ...
> B, disp2, count_a2, type_a2, win)
>
> since you would use MPI_BOTTOM(Process A) instead of MPI_BOTTOM(Process B) in the computation of
> disp2.
>
> In the heterogeneous case and using real displacements (note disp_b might be negative),
>
> disp2 = MPI_Aint_index_disp (disp_b, index2)
>
> might not work since the specific data (on heterogeneity) of the Process B is missing.
>
> What's about (called in process A)
>
> disp2 = MPI_Win_get_disp (win, B, disp_b, index2)
>
> which computes the displacement for address (MPI_BOTTOM(B) + disp_b + index2) for Process B.
> The window win knows the address of MPI_BOTTOM(B) for dynamic windows or other bases for other
> window flavors.
>
> A function
>
> disp2 = MPI_Win_get_addr_disp (win, B, base_b, index2)
>
> (base_b is an address stored in an MPI_Aint) would be also possible and
> would return the displacement for address ((char *) base_b + index2) in Process B
> relative to MPI_BOTTOM(B) (or the base address of other window flavors).
>
>
> I hope, that I understood the problems correctly.
>
> Hubert
>
>
> ________________________________________
> From: mpi3-rma-bounces at lists.mpi-forum.org [mpi3-rma-bounces at lists.mpi-forum.org] on behalf of Jim Dinan [dinan at mcs.anl.gov]
> Sent: Friday, March 15, 2013 4:24 PM
> To: mpi3-rma at lists.mpi-forum.org
> Subject: Re: [Mpi3-rma] [EXTERNAL] Re: Ticket for WG discussion
>
> Instead, how about a routine that converts MPI_Aints to be relative to a
> common MPI_BOTTOM so they mean something on other processes. We could
> define:
>
> MPI_Aint_rebase(MPI_Aint in, MPI_Aint in_bottom, MPI_Aint *out, MPI_Aint
> out_bottom);
>
> So, one could take the address from MPI_Win_attach and perform:
>
> MPI_Aint_rebase(attach_addr, MPI_BOTTOM, &portable_addr, MPI_BOTTOM_COMMON);
>
> The recipient would reverse this operation:
>
> MPI_Aint_rebase(recv_addr, MPI_BOTTOM_COMMON, &addr, MPI_BOTTOM);
>
> ~Jim.
>
> On 3/15/13 10:16 AM, Dave Goodell wrote:
>> We currently require MPI_Aint to have the same size and representation in both C and Fortran. So this would be a somewhat major departure...
>>
>> -Dave
>>
>> On Mar 15, 2013, at 10:04 AM CDT, Jim Dinan <dinan at mcs.anl.gov> wrote:
>>
>>> Given this MPI_BOTTOM awfulness, it seems like we need to add MPI_Aint_f2c/c2f() in order for dynamic windows to be reasonable from Fortran. To use dynamic windows, users must send aints from one process to another (after attaching memory). Given that Fortran aints are not portable across processes and C aints are, it seems like we need a way to make this conversion. And we will have to warn Fortran users that only C aints can be interpreted correctly in a different process.
>>>
>>> ~Jim.
>>>
>>> On 3/14/13 12:18 PM, Hubert Ritzdorf wrote:
>>>> Ok,
>>>>
>>>> this was one of the questions which was open to me.
>>>> If you want to create an address (better displacement to MPI_BOTTOM) for another process
>>>> you have to include the id (i.e. win, rank) of this process into
>>>> MPI_Aint_index(). MPI_BOTTOM may be different in each process in Fortran
>>>> and the local value of MPI_BOTTOM cannot be used.
>>>>
>>>> Hubert
>>>> ________________________________________
>>>> From: mpi3-rma-bounces at lists.mpi-forum.org [mpi3-rma-bounces at lists.mpi-forum.org] on behalf of Barrett, Brian W [bwbarre at sandia.gov]
>>>> Sent: Thursday, March 14, 2013 6:04 PM
>>>> To: MPI 3.0 Remote Memory Access working group
>>>> Subject: Re: [Mpi3-rma] [EXTERNAL] Re: Ticket for WG discussion
>>>>
>>>> Hubert -
>>>>
>>>> The issue with MPI_Get_address() in your usage model is one of heterogeneity. If process A is 32 bit and trying to create an address to talk to process B (which is 64 bit), there's now a difference (this, of course, assumes that MPI_Aint is an int64_t in that MPI).
>>>>
>>>> Brian
>>>>
>>>> --
>>>> Brian W. Barrett
>>>> Scalable System Software Group
>>>> Sandia National Laboratories
>>>> ________________________________________
>>>> From: mpi3-rma-bounces at lists.mpi-forum.org [mpi3-rma-bounces at lists.mpi-forum.org] on behalf of Hubert Ritzdorf [Hubert.Ritzdorf at emea.nec.com]
>>>> Sent: Thursday, March 14, 2013 10:56 AM
>>>> To: MPI 3.0 Remote Memory Access working group
>>>> Subject: [EXTERNAL] Re: [Mpi3-rma] Ticket for WG discussion
>>>>
>>>> Hi Jim,
>>>>
>>>> this sounds to me that
>>>>
>>>> addr2 = MPI_Aint_index (base, disp)
>>>>
>>>> is equivalent to
>>>>
>>>> MPI_Get_Address ((char *) base + disp, &addr2)
>>>>
>>>> Is this correct or where is the difference ?
>>>>
>>>> Or is "base" also a displacement relative to MPI_BOTTOM
>>>> (in which case it might be signed in Fortran) and
>>>>
>>>> addr2 = MPI_Aint_index (base, disp)
>>>>
>>>> is equivalent to
>>>>
>>>> MPI_Get_address ((char *) MPI_BOTTOM + base + disp, &addr2)
>>>>
>>>> Hubert
>>>> ________________________________________
>>>> From: mpi3-rma-bounces at lists.mpi-forum.org [mpi3-rma-bounces at lists.mpi-forum.org] on behalf of Jim Dinan [dinan at mcs.anl.gov]
>>>> Sent: Thursday, March 14, 2013 4:05 PM
>>>> To: mpi3-rma at lists.mpi-forum.org
>>>> Subject: Re: [Mpi3-rma] Ticket for WG discussion
>>>>
>>>> A little clearer:
>>>>
>>>> The value returned is the address, relative to MPI_BOTTOM, resulting
>>>> from the sum of base and disp.
>>>>
>>>> ~Jim.
>>>>
>>>> On 3/14/13 9:42 AM, Jim Dinan wrote:
>>>>> Hi Hubert,
>>>>>
>>>>> My apologies, the ticket is light on details. The proposed function is:
>>>>>
>>>>> MPI_Aint MPI_Aint_index(MPI_Aint base, MPI_Aint disp);
>>>>>
>>>>> This function interprets the base argument as an address relative to
>>>>> MPI_BOTTOM and the disp argument as a signed integer displacement. The
>>>>> value returned is the sum of these two values.
>>>>>
>>>>> ~Jim.
>>>>>
>>>>> On 3/13/13 7:12 AM, Hubert Ritzdorf wrote:
>>>>>> Hi Jim,
>>>>>>
>>>>>> Sorry, I have some problems with the interpretation of function
>>>>>> MPI_Aint_index().
>>>>>>
>>>>>> (*) What does MPI_Aint_index() return ?
>>>>>> A target displacement which can be used in RMA functions such
>>>>>> as MPI Put/Get or
>>>>>> does it return an address such as MPI_Get_address () ?
>>>>>>
>>>>>> (*) What is argument "base" ?
>>>>>> An address returned by MPI_Get_address() of any address or
>>>>>> a window base returned by
>>>>>> MPI_Win_get_attr(win, MPI_WIN_BASE, &base, &flag) ?
>>>>>> May "base" (as MPI_Aint) be transferred from a different
>>>>>> process ?
>>>>>>
>>>>>> Hubert
>>>>>> ________________________________________
>>>>>> From: mpi3-rma-bounces at lists.mpi-forum.org
>>>>>> [mpi3-rma-bounces at lists.mpi-forum.org] on behalf of Jim Dinan
>>>>>> [dinan at mcs.anl.gov]
>>>>>> Sent: Tuesday, March 12, 2013 9:08 PM
>>>>>> To: mpi3-rma at lists.mpi-forum.org
>>>>>> Subject: Re: [Mpi3-rma] Ticket for WG discussion
>>>>>>
>>>>>> Hubert,
>>>>>>
>>>>>> We have added a solution #3 to the ticket based on discussion in the
>>>>>> working group today. That's our current proposal. Could you take a
>>>>>> look and see if that will be Fortran compatible?
>>>>>>
>>>>>> Thanks,
>>>>>> ~Jim.
>>>>>>
>>>>>> On 3/12/13 2:28 PM, Hubert Ritzdorf wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> concerning Ticket 349.
>>>>>>> (*) In Fortran, the displacement for dynamic windows may be signed
>>>>>>> since MPI_BOTTOM of Fortran is not (void *) NULL..
>>>>>>> (*) Fortran doesn't have unsigned integers, thus you cannot use
>>>>>>> a Fortran version of MPI_Disp for arithmetics (such as disp++).
>>>>>>> (*) MPI Displacements in general may be signed.
>>>>>>> They might be relative to any address.
>>>>>>>
>>>>>>>
>>>>>>> Therefore, I can't see that the Proposed Solution 2 solves the problem.
>>>>>>>
>>>>>>> Hubert
>>>>>>>
>>>>>>> ________________________________________
>>>>>>> From: mpi3-rma-bounces at lists.mpi-forum.org
>>>>>>> [mpi3-rma-bounces at lists.mpi-forum.org] on behalf of Jim Dinan
>>>>>>> [dinan at mcs.anl.gov]
>>>>>>> Sent: Tuesday, March 12, 2013 5:12 PM
>>>>>>> To: MPI 3.0 Remote Memory Access working group
>>>>>>> Subject: [Mpi3-rma] Ticket for WG discussion
>>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> I would like to propose the following two tickets for discussion in
>>>>>>> today's WG meeting:
>>>>>>>
>>>>>>> https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/349
>>>>>>> https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/344
>>>>>>>
>>>>>>> #349 is a bugfix that definitely needs to be addressed. #344 is an
>>>>>>> "elegance of the interface" ticket where I would like to poll the WG to
>>>>>>> determine the level of support and determine if this should be pursued.
>>>>>>>
>>>>>>> I unfortunately need to duck out at 4pm today, so if possible, I propose
>>>>>>> that we discuss these at the beginning of the meeting.
>>>>>>>
>>>>>>> Thanks!
>>>>>>> ~Jim.
>>>>>>> _______________________________________________
>>>>>>> mpi3-rma mailing list
>>>>>>> mpi3-rma at lists.mpi-forum.org
>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>>>>>>
>>>>>>>
>>>>>>> Click
>>>>>>> https://www.mailcontrol.com/sr/kHf7UuVmhqHGX2PQPOmvUu5zZAYN1MosNz2OH0bOQrlt5gIrfPQ8MyK+!tr+rBycX!uZHqHqrIVkxH4nYzTQYQ==
>>>>>>> to report this email as spam.
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> mpi3-rma mailing list
>>>>>>> mpi3-rma at lists.mpi-forum.org
>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>>>>>>
>>>>>> _______________________________________________
>>>>>> mpi3-rma mailing list
>>>>>> mpi3-rma at lists.mpi-forum.org
>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>>>>>
>>>>>> _______________________________________________
>>>>>> mpi3-rma mailing list
>>>>>> mpi3-rma at lists.mpi-forum.org
>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>>>>>
>>>>> _______________________________________________
>>>>> mpi3-rma mailing list
>>>>> mpi3-rma at lists.mpi-forum.org
>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>>> _______________________________________________
>>>> mpi3-rma mailing list
>>>> mpi3-rma at lists.mpi-forum.org
>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>>>
>>>> _______________________________________________
>>>> mpi3-rma mailing list
>>>> mpi3-rma at lists.mpi-forum.org
>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> mpi3-rma mailing list
>>>> mpi3-rma at lists.mpi-forum.org
>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>>>
>>>> _______________________________________________
>>>> mpi3-rma mailing list
>>>> mpi3-rma at lists.mpi-forum.org
>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>>>
>>> _______________________________________________
>>> mpi3-rma mailing list
>>> mpi3-rma at lists.mpi-forum.org
>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>
>>
>> _______________________________________________
>> mpi3-rma mailing list
>> mpi3-rma at lists.mpi-forum.org
>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>>
> _______________________________________________
> mpi3-rma mailing list
> mpi3-rma at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>
> _______________________________________________
> mpi3-rma mailing list
> mpi3-rma at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma
>
More information about the mpiwg-rma
mailing list