[mpiwg-rma] same_op_no_op

Balaji, Pavan balaji at anl.gov
Thu Mar 13 12:21:34 CDT 2014 

Well, I think they are not allowed.  If the implementation can assume that they are the same, if a single process does two accumulate operations, the implementation is allowed to add an assert saying first_op == second_op.  That’ll blow up the program if they are not the same.

  — Pavan

On Mar 13, 2014, at 12:16 PM, Jeff Hammond <jeff.science at gmail.com> wrote:

> I don't think MPI-3 says they are not allowed, just that the
> implementation "will assume" that >1 operations (except for NO_OP) are
> never used concurrently.  Since the effect of the implementation's
> assumptions cannot be known, I would like to assume that concurrent
> different operations are not necessarily atomic i.e. implementation
> defined, and most certainly not erroneous.
> 
> As to screwing this up, yes, I agree on multiple levels :-)
> 
> Jeff
> 
> On Thu, Mar 13, 2014 at 12:02 PM, Balaji, Pavan <balaji at anl.gov> wrote:
>> 
>> MPI-2.2 says that accumulate with different ops are not atomic.
>> 
>> MPI-3 says that accumulate with different ops are not allowed (since same_op_no_op is default).
>> 
>> I think we screwed that up?
>> 
>>  — Pavan
>> 
>> On Mar 13, 2014, at 11:48 AM, Jeff Hammond <jeff.science at gmail.com> wrote:
>> 
>>> It is extremely difficult to see that this is what the MPI-3 standard says.
>>> 
>>> First we have this:
>>> 
>>> "The outcome of concurrent accumulate operations to the same location
>>> with the same predefined datatype is as if the accumulates were done
>>> at that location in some serial order. Additional restrictions on the
>>> operation apply; see the info key accumulate_ops in Section 11.2.1.
>>> Concurrent accumulate operations with different origin and target
>>> pairs are not ordered. Thus, there is no guarantee that the entire
>>> call to an accumulate operation is executed atomically. The effect of
>>> this lack of atomicity is limited: The previous correctness conditions
>>> imply that a location updated by a call to an accumulate operation
>>> cannot be accessed by a load or an RMA call other than accumulate
>>> until the accumulate operation has completed (at the target).
>>> Different interleavings can lead to different results only to the
>>> extent that computer arithmetics are not truly associative or
>>> commutative. The outcome of accumulate operations with overlapping
>>> types of different sizes or target displacements is undefined."
>>> [11.7.1 Atomicity]
>>> 
>>> Then we have this:
>>> 
>>> "accumulate_ops — if set to same_op, the implementation will assume
>>> that all concurrent accumulate calls to the same target address will
>>> use the same operation. If set to same_op_no_op, then the
>>> implementation will assume that all concurrent accumulate calls to the
>>> same target address will use the same operation or MPI_NO_OP. This can
>>> eliminate the need to protect access for certain operation types where
>>> the hardware can guarantee atomicity. The default is same_op_no_op."
>>> [11.2.1 Window Creation]
>>> 
>>> I was not aware that the definition of info keys was normative, given
>>> that implementations are free to ignore them.  Even if info key text
>>> is normative, one has to infer from the fact that same_op_no_op is the
>>> default info behavior - and thus RMA semantic - that accumulate
>>> atomicity is restricted to the case where one uses the same op or noop
>>> but not replace.
>>> 
>>> The MPI-2.2 spec is unambiguous because it explicitly requires the
>>> same operation in 11.7.1 Atomicity.  This text was removed in MPI-3.0
>>> in favor of the info key text.
>>> 
>>> Best,
>>> 
>>> Jeff
>>> 
>>> On Tue, Mar 11, 2014 at 12:04 AM, Balaji, Pavan <balaji at anl.gov> wrote:
>>>> 
>>>> MPI-2 defines atomicity only for the same operation, not any operation for MPI_ACCUMULATE.
>>>> 
>>>> — Pavan
>>>> 
>>>> On Mar 10, 2014, at 11:22 PM, Jeff Hammond <jeff.science at gmail.com> wrote:
>>>> 
>>>>> So MPI-2 denied compatibility between replace and not-replace?
>>>>> 
>>>>> Jeff
>>>>> 
>>>>> Sent from my iPhone
>>>>> 
>>>>>> On Mar 11, 2014, at 12:06 AM, "Balaji, Pavan" <balaji at anl.gov> wrote:
>>>>>> 
>>>>>> 
>>>>>> It doesn’t break backward compatibility.  The info argument is still useful when you don’t want to use replace.  I don’t see anything wrong with it.
>>>>>> 
>>>>>>> On Mar 10, 2014, at 11:01 PM, Jeff Hammond <jeff.science at gmail.com> wrote:
>>>>>>> 
>>>>>>> Does this or does this not break BW compatibility w.r.t. MPI-2.2 and
>>>>>>> did we do it intentionally?  Unless we did so intentionally and
>>>>>>> explicitly, I will argue that the WG screwed up and the info key+val
>>>>>>> is invalid.
>>>>>>> 
>>>>>>> Jeff
>>>>>>> 
>>>>>>>> On Mon, Mar 10, 2014 at 11:03 PM, Balaji, Pavan <balaji at anl.gov> wrote:
>>>>>>>> 
>>>>>>>> If a hardware can implement MPI_SUM, it should be able to implement MPI_SUM with 0 as well.
>>>>>>>> 
>>>>>>>> But that’s not a generic solution.
>>>>>>>> 
>>>>>>>> Jeff: at some point you were planning to bring in a ticket which does more combinations of operations than just same_op and no_op.  Maybe it’s worthwhile bringing that up again?
>>>>>>>> 
>>>>>>>> — Pavan
>>>>>>>> 
>>>>>>>>> On Mar 10, 2014, at 9:26 PM, Jim Dinan <james.dinan at gmail.com> wrote:
>>>>>>>>> 
>>>>>>>>> Maybe there's a loophole that I'm forgetting?
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Mon, Mar 10, 2014 at 9:43 PM, Jeff Hammond <jeff.science at gmail.com> wrote:
>>>>>>>>> How the hell can I do GA or SHMEM then? Roll my own mutexes and commit perf-suicide?
>>>>>>>>> 
>>>>>>>>> Jeff
>>>>>>>>> 
>>>>>>>>> Sent from my iPhone
>>>>>>>>> 
>>>>>>>>>> On Mar 10, 2014, at 8:32 PM, Jim Dinan <james.dinan at gmail.com> wrote:
>>>>>>>>>> 
>>>>>>>>>> You can't use replace and sum concurrently at a given target address.
>>>>>>>>>> 
>>>>>>>>>> ~Jim.
>>>>>>>>>> 
>>>>>>>>>> On Mon, Mar 10, 2014 at 4:30 PM, Jeff Hammond <jeff.science at gmail.com> wrote:
>>>>>>>>>> Given the following, how do I use MPI_NO_OP, MPI_REPLACE and MPI_SUM
>>>>>>>>>> in accumulate/atomic operations in a standard-compliant way?
>>>>>>>>>> 
>>>>>>>>>> accumulate_ops — if set to same_op, the implementation will assume
>>>>>>>>>> that all concurrent accumulate calls to the same target address will
>>>>>>>>>> use the same operation. If set to same_op_no_op, then the
>>>>>>>>>> implementation will assume that all concurrent accumulate calls to the
>>>>>>>>>> same target address will use the same operation or MPI_NO_OP. This can
>>>>>>>>>> eliminate the need to protect access for certain operation types where
>>>>>>>>>> the hardware can guarantee atomicity. The default is same_op_no_op.
>>>>>>>>>> 
>>>>>>>>>> We discuss this before and the resolution was not satisfying to me.
>>>>>>>>>> 
>>>>>>>>>> Thanks,
>>>>>>>>>> 
>>>>>>>>>> Jeff
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> Jeff Hammond
>>>>>>>>>> jeff.science at gmail.com
>>>>>>>>>> _______________________________________________
>>>>>>>>>> mpiwg-rma mailing list
>>>>>>>>>> mpiwg-rma at lists.mpi-forum.org
>>>>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma
>>>>>>>>>> 
>>>>>>>>>> _______________________________________________
>>>>>>>>>> mpiwg-rma mailing list
>>>>>>>>>> mpiwg-rma at lists.mpi-forum.org
>>>>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma
>>>>>>>>> 
>>>>>>>>> _______________________________________________
>>>>>>>>> mpiwg-rma mailing list
>>>>>>>>> mpiwg-rma at lists.mpi-forum.org
>>>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma
>>>>>>>>> 
>>>>>>>>> _______________________________________________
>>>>>>>>> mpiwg-rma mailing list
>>>>>>>>> mpiwg-rma at lists.mpi-forum.org
>>>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> mpiwg-rma mailing list
>>>>>>>> mpiwg-rma at lists.mpi-forum.org
>>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> Jeff Hammond
>>>>>>> jeff.science at gmail.com
>>>>>>> _______________________________________________
>>>>>>> mpiwg-rma mailing list
>>>>>>> mpiwg-rma at lists.mpi-forum.org
>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma
>>>>>> 
>>>>>> _______________________________________________
>>>>>> mpiwg-rma mailing list
>>>>>> mpiwg-rma at lists.mpi-forum.org
>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma
>>>>> _______________________________________________
>>>>> mpiwg-rma mailing list
>>>>> mpiwg-rma at lists.mpi-forum.org
>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma
>>>> 
>>>> _______________________________________________
>>>> mpiwg-rma mailing list
>>>> mpiwg-rma at lists.mpi-forum.org
>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma
>>> 
>>> 
>>> 
>>> --
>>> Jeff Hammond
>>> jeff.science at gmail.com
>>> _______________________________________________
>>> mpiwg-rma mailing list
>>> mpiwg-rma at lists.mpi-forum.org
>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma
>> 
>> _______________________________________________
>> mpiwg-rma mailing list
>> mpiwg-rma at lists.mpi-forum.org
>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma
> 
> 
> 
> -- 
> Jeff Hammond
> jeff.science at gmail.com
> _______________________________________________
> mpiwg-rma mailing list
> mpiwg-rma at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

More information about the mpiwg-rma mailing list