[mpiwg-rma] Default value of accumulate_ops info key
balaji at anl.gov
Sat Nov 3 11:21:54 CDT 2018
It’ll be good to understand in the next call why Open MPI is losing performance with any_op compared with same_op_no_op. In MPICH, they would both perform equally mainly because of three reasons:
1. The process calling “no op” doesn’t know what the “same op” is, so it has to assume that it could be some op that is not supported in hardware, and hence rely on active messages.
2. There’s no compatibility between shared memory atomics and network atomics.
3. Derived datatypes might go through active messages (for pack/unpack reasons) even if the op is known. So even if I have contiguous datatypes I might have to assume that other processes could use derived datatypes. Issuing each contiguous segment independently is losing too much performance.
With the proposed hints for MPI-4, we can certainly improve all these cases, but without any hint the default case should be just as good (or just as bad) for MPI-3 and MPI-4.
Sent from my iPhone
On Nov 3, 2018, at 10:56 AM, Nathan Hjelm <hjelmn at me.com<mailto:hjelmn at me.com>> wrote:
I plan to implement the new keys in Open MPI for the next release (not v4.0.x). Just keep in mind any_op will slow accumulate, fetch and op, etc down a lot in some cases as the implemention may be unable to use network atomics. Pavan's proposal adds another info key where you can list the atomics that will be used to hopefully reduce the number of cases where things slow down.
On Nov 2, 2018, at 11:02 PM, Fujita, Hajime via mpiwg-rma <mpiwg-rma at lists.mpi-forum.org<mailto:mpiwg-rma at lists.mpi-forum.org>> wrote:
Got it. Thanks!
From: "Balaji, Pavan" <balaji at anl.gov<mailto:balaji at anl.gov>>
Date: Friday, November 2, 2018 at 3:03 PM
To: "Fujita, Hajime" <hajime.fujita at intel.com<mailto:hajime.fujita at intel.com>>
Cc: "Balaji, Pavan" <balaji at anl.gov<mailto:balaji at anl.gov>>, MPI WG Remote Memory Access working group <mpiwg-rma at lists.mpi-forum.org<mailto:mpiwg-rma at lists.mpi-forum.org>>
Subject: Re: [mpiwg-rma] Default value of accumulate_ops info key
The invite went out to the entire WG. I've attached it again as an ics file with this email.
RMA WG recurring telecon
Scheduled: Nov 5, 2018 at 2:00 PM to 3:00 PM
To join the Meeting:
To join via Room System:
Video Conferencing System: bjn.vc -or-188.8.131.52
Meeting ID : 173205081
Participant Passcode : 3141592
To join via phone :
+1.408.317.9254 (US (San Jose))
+1.866.226.4650 (US Toll Free)
(see all numbers - http://bluejeans.com/numbers)
2) Enter Conference ID : 173205081
3) Enter Participant Passcode : 3141592
> On Nov 2, 2018, at 2:37 PM, Fujita, Hajime <hajime.fujita at intel.com<mailto:hajime.fujita at intel.com>> wrote:
> Hi Pavan,
> So the next meeting is at 11/5 (Mon) 2pm central? I don't see a calendar invite.
> On 11/2/18, 12:11 PM, "mpiwg-rma on behalf of Balaji, Pavan via mpiwg-rma" <mpiwg-rma-bounces at lists.mpi-forum.org<mailto:mpiwg-rma-bounces at lists.mpi-forum.org> on behalf of mpiwg-rma at lists.mpi-forum.org<mailto:mpiwg-rma at lists.mpi-forum.org>> wrote:
> Hi Joseph,
> I don't know about MPI-3.2, but if the Forum accepts the proposal, I believe we have started pushing out draft standards which can be used. Having said that, I don't know if MPI implementations will implement draft standards or wait till the final standard is out.
> The last telecon was canceled. We have one more this coming Monday.
> -- Pavan
>> On Nov 2, 2018, at 10:08 AM, Joseph Schuchart via mpiwg-rma <mpiwg-rma at lists.mpi-forum.org<mailto:mpiwg-rma at lists.mpi-forum.org>> wrote:
>> While working on a code that makes use of MPI accumulate operations, I stumbled upon reading the description of the `accumulate_ops` info key. The standard (v3.1) states "The default is same_op_no_op." My interpretation is that this is the value used if the info key is not specified (based on my understanding of the term `default`). How would I specify my intent to use any accumulate operation? From looking at one implementation, the default is to expect the use of any operation but that seems inconsistent and confusing for users.
>> I remember the discussion of a proposal for v4.0 to add `any_op` in a recent phone conference but I couldn't find that proposal anywhere online. Would it be possible to fix this issue in v3.2 by removing the sentence quoted above?
>> On another note: Based on the invitation sent out by Pavan Balaji I tried to join the biweekly phone conference on Monday last week (10/22) at 2pm central time but the conference room was empty. Is this still the current schedule? Maybe I just got the timezones messed up?
>> Many thanks in advance,
>> Joseph Schuchart
>> Dipl.-Inf. Joseph Schuchart
>> High Performance Computing Center Stuttgart (HLRS)
>> Nobelstr. 19
>> D-70569 Stuttgart
>> Tel.: +49(0)711-68565890
>> Fax: +49(0)711-6856832
>> E-Mail: schuchart at hlrs.de<mailto:schuchart at hlrs.de>
>> mpiwg-rma mailing list
>> mpiwg-rma at lists.mpi-forum.org<mailto:mpiwg-rma at lists.mpi-forum.org>
> mpiwg-rma mailing list
> mpiwg-rma at lists.mpi-forum.org<mailto:mpiwg-rma at lists.mpi-forum.org>
mpiwg-rma mailing list
mpiwg-rma at lists.mpi-forum.org<mailto:mpiwg-rma at lists.mpi-forum.org>
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the mpiwg-rma