[Mpi-forum] MPI One-Sided Communication

[Patrick Geoffray]

Dear Jeff,

Jeff Hammond wrote:
> Frankly, this is just a religious war already waged in HPCWire
> (Myricom versus IBM).  It appears you're saying that IBM and Cray

Ok, this is my invitation to participate in this sterile discussion.

First of all, it was not Myricom versus IBM, it was me, as someone who 
implemented MPI-1 and one-sided interfaces such as VIA, versus a guy who 
sat on IB/iWarp/iSCSI/FC committees knowing zip about communication 
middlewares. It looks like a religion war for people who do not 
understand the issues. Fortunately, a lot of people in this forum do, as 
least the implementors.

 From your background and comments, you are a product of the PNNL Church 
of One-Siding. Nothing wrong with that, free country. However, you 
should step outside of the NWchem/GA/ARMCI community and see what the 
rest of the world is doing. How many apps use GA/ARMCI except NWchem and 
Molpro ? I happen to believe that there is no difference in performance 
between one and two-sided interfaces on most apps, as long as the app 
and middleware codes are optimized. I had this discussion with Vinod a 
few years ago, when he was presenting an ARMCI vs MPI paper: if you 
spend the same care when writing the MPI version of the code, it would 
perform the same as the freshly written ARMCI version. It should not be 
a performance discussion, it should be about ease of use.

Ease of use is subjective of course, but the fact that there are a 
gazillions more MPI codes than one-sided codes out there could be a hint.

> supporting powerful one-sided hardware is a waste of money.  Is that
> why their machines, SGI's and those running Infiniband (excellent
> RDMA) compose the first 26 slots on the Top500 list?  Anti-one-sided
> Myricom's best showing is #27.

If you think that people buy machines because of one-sided interface, 
you are greatly mistaken. If you think the large IB machines use 
one-sided operations for MPI small/medium messages at scale, you are 
wrong again. Finally, if you think that Myricom is anti-one-sided, you 
don't know the differences between RDMA and one-sided. We happen to have 
ported ARMCI on top of asynchronous Active Messages in MX, like 
Pathscale did. It walks like one-sided, it quacks like one-sided, but it 
scales well.

This is essentially Greg's point: you can implement one-sided semantics 
on Active Messages and it does scale.


Now, it doesn't help in the discussion about MPI-3 one-sided interface. 
There is simply no incentive for implementors to spend time on it if 
nobody uses it. Today, almost nobody does. Why ?

Patrick