<html><body>
<p>On any MPI implementation the author of this application sequence:<br>
<br>
MPI_ISEND<br>
compute<br>
MPI_WAIT<br>
modify or free buffer<br>
<br>
will hope that the MPI_WAIT takes as little time as possible. <br>
<br>
The standard tells the user he can safely "reuse" the send buffer when the MPI_WAIT returns. If there are things which must happen before it is safe to free the buffer that do not need to happen before it is safe to modify its content, must an MPI implementation with the kind of issues Erez is suggesting delay the return from MPI_WAIT until both modify and free are safe?<br>
<br>
I suppose so - but it is unfortunate to need to delay the return from MPI_WAIT when it is safe for the user to do what he most likely has in mind (modify).<br>
<br>
As someone else pointed out, once the destination receive has finished, any packet retransmissions that are caused by ACK loss can harmlessly contain modified data. The send buffer does not need to be protected from updates. The destination will simply discard these packets them anyway.<br>
<br>
Also, in most systems support for malloc/free, the free() call does not change the ability of the adapter or VMM to modify the status of the address range. The address range that had been exposed to application use before the free() is simply added back to the heap and stays there until another malloc needs it. Doing a store to a memory range after a free() is likely to cause trouble but doing a load from ii is normally harmless. (I suppose someone will now tell me about a system where "free(ptr); x=*ptr;" causes segfaults. It is not the norm)<br>
<br>
Dick<br>
<br>
<br>
Dick Treumann - MPI Team/TCEM <br>
IBM Systems & Technology Group<br>
Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601<br>
Tele (845) 433-7846 Fax (845) 433-8363<br>
<br>
<br>
<tt>mpi-22-bounces@lists.mpi-forum.org wrote on 07/17/2008 02:01:24 PM:<br>
<br>
> <br>
> <br>
> -----Original Message-----<br>
> From: mpi-22-bounces@lists.mpi-forum.org [<a href="mailto:mpi-22-">mailto:mpi-22-</a><br>
> bounces@lists.mpi-forum.org] On Behalf Of Underwood, Keith D<br>
> Sent: Wednesday, July 16, 2008 4:32 PM<br>
> To: MPI 2.2<br>
> Subject: Re: [Mpi-22] [Mpi-forum] MPI 2.2 proposal:resolving <br>
> MPI_Request_free issues<br>
> <br>
> >> I don't quite understand examples 1 and 2 (how would they cause segv's<br>
> >> in the TCP stack). It is permissible to (pseudocode):<br>
> >><br>
> >> while (bytes_to_send > 0) {<br>
> >> rc = write(fd, buffer, bytes_to_send);<br>
> >> if (rc > 0) {<br>
> >> buffer += rc;<br>
> >> bytes_to_send -= rc;<br>
> >> } else {<br>
> >> ...error...<br>
> >> }<br>
> >> }<br>
> >> free(buffer);<br>
> >><br>
> >> regardless of what the receiver does. I'm not a kernel guy; does<br>
> >> updating TCP sequence numbers also interact with the payload buffer?<br>
> ><br>
> >[erezh] it will never happen with your code above; but you are not using<br>
> >async zcopy.<br>
> >The pattern in windows is to use overlapped send (write) which is still<br>
> >active when the function returns, and is the most efficient way to send<br>
> >your buffer. I know it's possible with Linux but I don't' have the exact<br>
> >pattern.<br>
> <br>
> > [Keith]<br>
> > So, how does this work for other users of zero copy? How do they know<br>
> > when the send buffer is truly free?<br>
> <br>
> [erezh] you get a notification from your local resource manager; on <br>
> windows with the overlapped mechanism you have few ways to get <br>
> notified. 1) The overlapped event get set, the thread can wait on <br>
> it. 2) a completion port notification that the thread can dequeue. <br>
> 3) an APC (similar to unix signal) get queued to the thread (thread <br>
> need to be in an alertable state)<br>
> <br>
> _______________________________________________<br>
> mpi-22 mailing list<br>
> mpi-22@lists.mpi-forum.org<br>
> <a href="http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22">http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22</a><br>
</tt></body></html>