[Mpi-forum] MPI_Request_free restrictions

[Jeff Hammond]

You are right that I am assuming that if an NBC completes at one rank, I
can reason about the state of that operation at another.  I can certainly
come up with scenarios where that is dangerous.

Jeff

On Sat, Aug 15, 2020 at 9:57 AM Richard Graham <richgraham at nvidia.com>
wrote:

> I think you start going down a slippery slope when you start to guess
> about the state of a resource that has been put under the control on
> another entity, without that entity explicitly declaring something.
>
> Rich
>
>
>
> Sent via the Samsung Galaxy S9, an AT&T 5G Evolution capable smartphone
>
>
>
> -------- Original message --------
> From: Jeff Hammond via mpi-forum <mpi-forum at lists.mpi-forum.org>
> Date: 8/15/20 12:51 (GMT-05:00)
> To: "Skjellum, Anthony" <Tony-Skjellum at utc.edu>
> Cc: Jeff Hammond <jeff.science at gmail.com>, Main MPI Forum mailing list <
> mpi-forum at lists.mpi-forum.org>
> Subject: Re: [Mpi-forum] MPI_Request_free restrictions
>
> *External email: Use caution opening links or attachments*
> Yes, but do we think the use case of never detecting completion is
> likely?  I am not arguing for that, but rather that users might free the
> request but then detect completion another way, such as by waiting on the
> request on a subset of processes.
>
> Jeff
>
> On Wed, Aug 12, 2020 at 5:40 PM Skjellum, Anthony <Tony-Skjellum at utc.edu>
> wrote:
>
>> FYI, one argument (also used to force us to add restrictions on MPI
>> persistent collective initialization to be blocking)... The
>> MPI_Request_free on an NBC poses a problem for the cases where there are
>> array types
>> posed (e.g., Alltoallv/w)... It will not be knowable to the application
>> if the vectors are in use by MPI still after
>> the  free on an active request.  We do *not* mandate that the MPI
>> implementation copy such arrays currently, so they are effectively "held as
>> unfreeable" by the MPI implementation till MPI_Finalize.  The user
>> cannot deallocate them in a correct program till after MPI_Finalize.
>>
>> Another effect for NBC of releasing an active request, IMHO,  is that you
>> don't know when send buffers are free to be deallocated or receive buffers
>> are free to be deallocated... since you don't know when the transfer is
>> complete OR the buffers are no longer used by MPI (till after MPI_Finalize).
>>
>> Tony
>>
>>
>>
>>
>> Anthony Skjellum, PhD
>>
>> Professor of Computer Science and Chair of Excellence
>>
>> Director, SimCenter
>>
>> University of Tennessee at Chattanooga (UTC)
>>
>> tony-skjellum at utc.edu  [or skjellum at gmail.com]
>>
>> cell: 205-807-4968
>>
>>
>> ------------------------------
>> *From:* mpi-forum <mpi-forum-bounces at lists.mpi-forum.org> on behalf of
>> Jeff Hammond via mpi-forum <mpi-forum at lists.mpi-forum.org>
>> *Sent:* Saturday, August 8, 2020 12:07 PM
>> *To:* Main MPI Forum mailing list <mpi-forum at lists.mpi-forum.org>
>> *Cc:* Jeff Hammond <jeff.science at gmail.com>
>> *Subject:* Re: [Mpi-forum] MPI_Request_free restrictions
>>
>> We should fix the RMA chapter with an erratum. I care less about NBC but
>> share your ignorance of why it was done that way.
>>
>> Sent from my iPhone
>>
>> On Aug 8, 2020, at 6:51 AM, Balaji, Pavan via mpi-forum <
>> mpi-forum at lists.mpi-forum.org> wrote:
>>
>>  Folks,
>>
>> Does someone remember why we disallowed users from calling
>> MPI_Request_free on nonblocking collective requests?  I remember the
>> reasoning for not allowing cancel (i.e., the operation might have completed
>> on some processes, but not all), but not for Request_free.  AFAICT,
>> allowing the users to free the request doesn’t make any difference to the
>> MPI library.  The MPI library would simply maintain its own refcount to the
>> request and continue forward till the operation completes.  One of our
>> users would like to free NBC requests so they don’t have to wait for the
>> operation to complete in some situations.
>>
>> Unfortunately, when I added the Rput/Rget operations in the RMA chapter,
>> I copy-pasted that text into RMA as well without thinking too hard about
>> it.  My bad!  Either the RMA committee missed it too, or they thought of a
>> reason that I can’t think of now.
>>
>> Can someone clarify or remind me what the reason was?
>>
>> Regards,
>>
>>   — Pavan
>>
>> MPI-3.1 standard, page 197, lines 26-27:
>>
>> “It is erroneous to call MPI_REQUEST_FREE or MPI_CANCEL for a request
>> associated with a nonblocking collective operation.”
>>
>> _______________________________________________
>> mpi-forum mailing list
>> mpi-forum at lists.mpi-forum.org
>> https://lists.mpi-forum.org/mailman/listinfo/mpi-forum
>> <https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mpi-forum.org%2Fmailman%2Flistinfo%2Fmpi-forum&data=02%7C01%7Crichgraham%40nvidia.com%7Cda892ae3f6da411c982908d8413b8181%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637331071121370760&sdata=L1UxfurOJSMZu0VlEgq5tl7sqghnlzVx%2Fr%2BOxVq4tdQ%3D&reserved=0>
>>
>>
>
> --
> Jeff Hammond
> jeff.science at gmail.com
> http://jeffhammond.github.io/
> <https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fjeffhammond.github.io%2F&data=02%7C01%7Crichgraham%40nvidia.com%7Cda892ae3f6da411c982908d8413b8181%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637331071121380752&sdata=UW5krprJGbRnlqB3YHXrt82QRM8fJ5gn3bo%2B63E1D6g%3D&reserved=0>
>


-- 
Jeff Hammond
jeff.science at gmail.com
http://jeffhammond.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpi-forum/attachments/20200815/ad5482d1/attachment.html>