[Mpi-22] [Mpi-forum] MPI 2.2 proposal: resolving MPI_Request_free issues

[Richard Treumann]

On any MPI implementation the author of this application sequence:

MPI_ISEND
compute
MPI_WAIT
modify or free buffer

will hope that the MPI_WAIT takes as little time as possible.

The standard tells the user he can safely "reuse" the send buffer when the
MPI_WAIT returns.  If there are things which must happen before it is safe
to free the buffer that do not need to happen before it is safe to modify
its content, must an MPI implementation with the kind of issues Erez is
suggesting delay the return from MPI_WAIT until both modify and free are
safe?

I suppose so - but it is unfortunate to need to delay the return from
MPI_WAIT when it is safe for the user to do what he most likely has in mind
(modify).

As someone else pointed out, once the destination receive has finished, any
packet retransmissions that are caused by ACK loss can harmlessly contain
modified data.  The send buffer does not need to be protected from updates.
The destination will simply discard these packets them anyway.

Also, in most systems support for malloc/free, the free() call does not
change the ability of the adapter or VMM to modify the status of the
address range.  The address range that had been exposed to application use
before the free() is simply added back to the heap and stays there until
another malloc needs it. Doing a store to a memory range after a free() is
likely to cause trouble but doing a load from ii is normally harmless. (I
suppose someone will now tell me about a system where "free(ptr); x=*ptr;"
causes segfaults. It is not the norm)

               Dick

Dick Treumann  -  MPI Team/TCEM
IBM Systems & Technology Group
Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601
Tele (845) 433-7846         Fax (845) 433-8363

mpi-22-bounces_at_[hidden] wrote on 07/17/2008 02:01:24 PM:

>
>
> -----Original Message-----
> From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-
> bounces_at_[hidden]] On Behalf Of Underwood, Keith D
> Sent: Wednesday, July 16, 2008 4:32 PM
> To: MPI 2.2
> Subject: Re: [Mpi-22] [Mpi-forum] MPI 2.2 proposal:resolving
> MPI_Request_free issues
>
> >> I don't quite understand examples 1 and 2 (how would they cause segv's
> >> in the TCP stack).  It is permissible to (pseudocode):
> >>
> >>   while (bytes_to_send > 0) {
> >>      rc = write(fd, buffer, bytes_to_send);
> >>      if (rc > 0) {
> >>         buffer += rc;
> >>         bytes_to_send -= rc;
> >>      } else {
> >>         ...error...
> >>      }
> >>   }
> >>   free(buffer);
> >>
> >> regardless of what the receiver does.  I'm not a kernel guy; does
> >> updating TCP sequence numbers also interact with the payload buffer?
> >
> >[erezh] it will never happen with your code above; but you are not using
> >async zcopy.
> >The pattern in windows is to use overlapped send (write) which is still
> >active when the function returns, and is the most efficient way to send
> >your buffer. I know it's possible with Linux but I don't' have the exact
> >pattern.
>
> > [Keith]
> > So, how does this work for other users of zero copy?  How do they know
> > when the send buffer is truly free?
>
> [erezh] you get a notification from your local resource manager; on
> windows with the overlapped mechanism you have few ways to get
> notified. 1) The overlapped event get set, the thread can wait on
> it. 2) a completion port notification that the thread can dequeue.
> 3) an APC (similar to unix signal) get queued to the thread (thread
> need to be in an alertable state)
>
> _______________________________________________
> mpi-22 mailing list
> mpi-22_at_[hidden]
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22





* 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpi-22/attachments/20080717/073038ab/attachment.html>