[Mpi-forum] MPI One-Sided Communication
patrick at myri.com
Fri Apr 24 23:34:14 CDT 2009
Jeff Hammond wrote:
> Frankly, this is just a religious war already waged in HPCWire
> (Myricom versus IBM). It appears you're saying that IBM and Cray
Ok, this is my invitation to participate in this sterile discussion.
First of all, it was not Myricom versus IBM, it was me, as someone who
implemented MPI-1 and one-sided interfaces such as VIA, versus a guy who
sat on IB/iWarp/iSCSI/FC committees knowing zip about communication
middlewares. It looks like a religion war for people who do not
understand the issues. Fortunately, a lot of people in this forum do, as
least the implementors.
From your background and comments, you are a product of the PNNL Church
of One-Siding. Nothing wrong with that, free country. However, you
should step outside of the NWchem/GA/ARMCI community and see what the
rest of the world is doing. How many apps use GA/ARMCI except NWchem and
Molpro ? I happen to believe that there is no difference in performance
between one and two-sided interfaces on most apps, as long as the app
and middleware codes are optimized. I had this discussion with Vinod a
few years ago, when he was presenting an ARMCI vs MPI paper: if you
spend the same care when writing the MPI version of the code, it would
perform the same as the freshly written ARMCI version. It should not be
a performance discussion, it should be about ease of use.
Ease of use is subjective of course, but the fact that there are a
gazillions more MPI codes than one-sided codes out there could be a hint.
> supporting powerful one-sided hardware is a waste of money. Is that
> why their machines, SGI's and those running Infiniband (excellent
> RDMA) compose the first 26 slots on the Top500 list? Anti-one-sided
> Myricom's best showing is #27.
If you think that people buy machines because of one-sided interface,
you are greatly mistaken. If you think the large IB machines use
one-sided operations for MPI small/medium messages at scale, you are
wrong again. Finally, if you think that Myricom is anti-one-sided, you
don't know the differences between RDMA and one-sided. We happen to have
ported ARMCI on top of asynchronous Active Messages in MX, like
Pathscale did. It walks like one-sided, it quacks like one-sided, but it
This is essentially Greg's point: you can implement one-sided semantics
on Active Messages and it does scale.
Now, it doesn't help in the discussion about MPI-3 one-sided interface.
There is simply no incentive for implementors to spend time on it if
nobody uses it. Today, almost nobody does. Why ?
More information about the mpi-forum