[Mpi3-rma] non-contiguous support in RMA & one-sided pack/unpack (?)

Wed Sep 16 16:29:55 CDT 2009

Do you really think it is just a couple of ifs?  If this is not MPI_COMM_WORLD... if this is a derived datatype but not contiguous locally... if this is a derived datatype but not contiguous remotely... if these are not the same datatypes... if this is not heterogeneous...

You also get into the trade-offs (that we obviously have to make even with two routines as well)... so, should contiguous derived datatypes be fast or slow?  Well, obviously the NIC based agent can handle them if you do the math in the MPI library, but that math takes instructions.   One of the things that was put on the table in Chicago was "should we have specialized functions for each native datatype?"

Ultimately, it is all about what your performance targets are in terms of latency and issue rate.  On one end of the spectrum, you have MPI_Isend().  On the other you have the speed of a native processor store.  Which end of the spectrum do we want to be on?  If we want broader penetration for RMA, we need to move toward the latter.  So, yeah, "a handful of ifs" may be an issue... Particularly on any architecture where a given thread isn't all that speedy...

Keith

From: mpi3-rma-bounces at lists.mpi-forum.org [mailto:mpi3-rma-bounces at lists.mpi-forum.org] On Behalf Of Richard Treumann
Sent: Wednesday, September 16, 2009 3:05 PM
To: MPI 3.0 Remote Memory Access working group
Subject: Re: [Mpi3-rma] non-contiguous support in RMA & one-sided pack/unpack (?)

Hi Jeff

I do not understand the need for two distinct origin side operations if you will assume the agency infrastructure must always be standing by ready to be activated.

The MPI implementation will be able to determine at the origin side if the parameters passed into a complex RMA fit the capabilities of its NIC. It can decide whether to use the intrinsic NIC mode or the software agent mode. If it has a NIC that does integer accumulate but not floating point accumulate, the implementation can pick between NIC mode and agent mode. When a new NIC that does floating point accumulate comes along, the implementation can exploit it with no need for application rewrite.

On the other hand, the RAW RMA can be used by by applications that want to exploit NIC capabilities that are widely available today and do not want to drag in overheads for an agent waiting on the sidelines. An agent that they may know will never be needed. If someday in the future the typical NIC has certain additional capabilities we can add a richer RAW operation to the MPI standard.

Can you explain to me what you expect to have always available as a software agent that will be there at no significant cost in memory or harm to performance of non-RMA operations?

Are you thinking we need two distinct routines just to avoid a couple "if" tests to decide inside libmpi whether to use NIC based or agent based RMA?

Dick

Dick Treumann - MPI Team
IBM Systems & Technology Group
Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601
Tele (845) 433-7846 Fax (845) 433-8363

[cid:image001.gif at 01CA36E1.7F111D50]Jeff Hammond ---09/16/2009 04:37:33 PM---I was having the same thoughts during my hideous commute today. Clearly, future systems may have ver

From:

Jeff Hammond <jeff.science at gmail.com>

To:

"MPI 3.0 Remote Memory Access working group" <mpi3-rma at lists.mpi-forum.org>

Date:

09/16/2009 04:37 PM

Subject:

Re: [Mpi3-rma] non-contiguous support in RMA & one-sided pack/unpack (?)

Sent by:

mpi3-rma-bounces at lists.mpi-forum.org

________________________________

<snip>

I'm not sure I understand all this talk about assertions upon
initialization, since I was hoping that Raw_xfer and general-purpose
xfer (Gen_xfer) would both always be available but that the former
would be much faster in certain contexts, since implementing the Raw
version close to hardware doesn't require much work and would not
interfere with how Gen_xfer operates.  Am I missing something?

<snip>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-rma/attachments/20090916/02ba44d8/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 105 bytes
Desc: image001.gif
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-rma/attachments/20090916/02ba44d8/attachment-0001.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 168 bytes
Desc: image003.png
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-rma/attachments/20090916/02ba44d8/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 166 bytes
Desc: image004.png
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-rma/attachments/20090916/02ba44d8/attachment-0003.png>