[Mpi-forum] MPI One-Sided Communication

[Underwood, Keith D]

<sigh> I meant the copy in the application that comes before MPI is called.  A lot of apps don't have all of the data they want to send in a contiguous buffer.  And, yes, derived datatypes can help this, but aren't commonly used and still involve a copy on a lot of hardware.

Keith


----- Original Message -----
From: mpi-forum-bounces at lists.mpi-forum.org <mpi-forum-bounces at lists.mpi-forum.org>
To: Main MPI Forum mailing list <mpi-forum at lists.mpi-forum.org>
Sent: Mon Apr 27 06:32:53 2009
Subject: Re: [Mpi-forum] MPI One-Sided Communication

On Apr 27, 2009, at 9:14 AM, Underwood, Keith D wrote:

> For many (most?) science and engineering codes, the trade-off  
> between matching for two sided and synchronization for one sided is  
> at best a wash, and often would fall in favor of two sided. There  
> is, of course, an exception....
>
> If one sided delivered truly stellar performance in terms of message  
> rate and latency, you could (in some cases) eliminate the cost of  
> the copy at the sender that is tyically done to send long messages  
> to cover the overhead. The hardware to deliver that level of  
> performance is truly rare...
>
> Keith

"cost of the copy at the sender that is tyically done to send long  
messages to cover the overhead" ?

Typically, any HPC NIC uses zero-copy for large messages. Copy is only  
used for small messages (where small depends on the fabric/NIC/ 
software stack).

It is even possible to avoid copy on send for large messages over  
standard Ethernet using sendfile() and other options.

Scott
_______________________________________________
mpi-forum mailing list
mpi-forum at lists.mpi-forum.org
http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum