[Mpi-forum] MPI One-Sided Communication

[Underwood, Keith D]

For the record, I'm an advocate of providing one-sided semantics and one-sided hardware support.  Heck, I help work on the Portals specification.  I just don't want to see that support to the exclusion of MPI and MPI hardware support.  I also want good arguments and examples to use to justify that support to get it in more hardware.  Scalable, fast (low latency, high message rate) one-sided hardware is not so prevalent as you seem to think.

>> Ahem.  I think you just proved Greg's point.  Jaguar is a Cray XT
>> series.  While I rather like that machine (I was part of the team at 
>> Sandia at the time), it has atrocious one-sided performance for small
>> messages.  If you can do it with big messages, you can do it with MPI-1
>> two sided.
>
>I repeat: show me the code.  By claiming that I could do it with
>MPI-1, you're saying that $100 million worth of one-sided-based
>quantum chemistry codes were a waste of time and money and that some
>very well regarded people are, in fact, quite stupid.

I'm claiming nothing of the sort.  I'm simply pointing out that if you set the one-sided performance of the XT as your bar, you are setting an awfully low bar.  It's got nice one-sided SEMANTICS (I like Portals, too), but the performance...

>>>See the attached papers for the details of the algorithm.  The
>>>GA/ARMCI implementation is less than 1000 lines of code.  Please let
>>>me know when you can match this with MPI-1.
>>
>> At this point, we are just bickering about where the work is, right?  So,
>the app wants to program in a GA model for ease of use.  Great.  That isn't
>a performance argument.
>
>If programmability isn't part of the equation, why have MPI?  Why have
>C?  We can all just write in assembly, right?
>
>We need to have an optimal balance between programmability and
>performance.  For the HPC applications I work on, one-sided beats
>two-sided hands down.  That's not a philosophy, it's 20 years of
>results on machines from the IBM SP-2 to the Cray XT5.

Where did I say I was opposed to programmability?!?  If you want to work in GA for ease of use, Great! If you need it for performance, Great!  However, don't tell me that you need it for performance and show me a system that takes an interrupt for EVERY SINGLE MESSAGE as an example.   

>>>Frankly, this is just a religious war already waged in HPCWire
>>>(Myricom versus IBM).  It appears you're saying that IBM and Cray
>>>supporting powerful one-sided hardware is a waste of money.  Is that
>>>why their machines, SGI's and those running Infiniband (excellent
>>>RDMA) compose the first 26 slots on the Top500 list?  Anti-one-sided
>>>Myricom's best showing is #27.
>>
>> Um, yeah, Infiniband isn't up there because of its one-sided
>> support.  Neither is the Cray machine.  Oh, wait, neither are the IBM
>> machines.
>
>So good hardware support for one-sided correlates strongly with Top500
>performance?  Why then argue against one-sided hardware?

No, it doesn't.  And, no, I'm not.  The XT series is a great product with great semantic support for one-sided and relatively poor one-sided performance.  I've never worked with a BG machine, so can't say.    

>Honestly, I can't follow this moving target argument against
>one-sided.  What is bad about one-sided, the hardware support for it,
>the programming model itself, or the use of implementation layers for
>programming within the model?  It seems you guys want to pick and
>choose what aspect of one-sided you want to trash whenever I attempt
>to pin down what your trying to say.

I'm not arguing against one-sided hardware, software, or implementations.  I was simply frustrated by your arguments against Greg.  It is very hard to create new one-sided/RMA semantics for MPI when we can't zero in on what the semantic and performance requirements are.  I would love to see a quantitative study of the one-sided performance needs of an application.  Telling me that Jaguar is great for the purpose would lead me to believe that those requirements aren't particularly strict.

Keith