[Mpi-forum] MPI_WAIT/MPI_TEST and generalized, I/O, and NBC requests

Jeff Hammond jhammond at alcf.anl.gov
Tue Feb 14 13:49:53 CST 2012


I favor chapter committee, as it is obvious even to a chemist (1) that
these changes are necessary and (2) how the appropriate members of the
Forum can/should/will implement them.

While I am not a chapter author, I have been the most vocal proponent
of RMA request-based operations, and both (1) and (2) remain obvious
in this context.

Jeff

On Tue, Feb 14, 2012 at 1:10 PM, Fab Tillier <ftillier at microsoft.com> wrote:
> I made a ticket for this, 321, so I could track it and have a way of posting a PDF for folks to take a look (e.g. I'd like the chapter authors for NBC, I/O, and generalized requests to agree to the wording).  Beyond that, I don't care how this eventually gets checked into the approved branch, and am happy just merging it and marking it as ticket0 and omitting the changelog entry if that's what folks want.
>
> So far we have:
>
> In favor of full ticket:
> - Rolf
>
> In favor of chapter committee:
> - Bill
> - Dave
> - Bronis
>
> I'd like to see at least Rich (pt2pt), Torsten (coll), and Rajeev (I/O) chime in with their preference.
>
> -Fab
>
> Dave Goodell wrote on Tue, 14 Feb 2012 at 10:41:51
>
>> No.  This is the sort of thing that should be handled by the chapter
>> committee.  This issue is far too small to warrant the attention of the
>> full Forum.
>>
>> We need to stay away from this mentality of making a ticket for everything
>> by default.
>>
>> -Dave
>>
>> On Feb 14, 2012, at 2:57 AM CST, Rolf Rabenseifner wrote:
>>
>>> A change that needs a change-log entry is not a ticket-0
>>> change. Therefore a ticket with ticket number is needed and
>>> it should be done now, added to the agenda, and read in Chicago.
>>>
>>> This change definitely is a correction that changes
>>> the standard, but hopefully no implementation,
>>> because all existing implementations may have
>>> it already implemented (i.e. are currently wrong
>>> if we look at the wording of MPI-2.2).
>>>
>>> Because it is very clear, it should need only 10 minutes
>>> of the Forum time.
>>>
>>> Rolf
>>>
>>> ----- Original Message -----
>>>> From: "William Gropp" <wgropp at illinois.edu> To: "Main MPI Forum
>>>> mailing list" <mpi-forum at lists.mpi-forum.org> Sent: Monday, February
>>>> 13, 2012 7:38:16 PM Subject: Re: [Mpi-forum] MPI_WAIT/MPI_TEST and
>>>> generalized, I/O, and NBC requests My recommendation is that the
>>>> chapter committee fold this into the chapter, rather than add this to
>>>> the long list of infinitesimal updates.
>>>>
>>>> Bill
>>>>
>>>> On Feb 13, 2012, at 11:42 AM, Fab Tillier wrote:
>>>>
>>>>> So are we back to needing a ticket for this then? Or is it still at
>>>>> the discretion of the chapter authors?
>>>>>
>>>>> -Fab
>>>>>
>>>>> Rolf Rabenseifner wrote on Mon, 13 Feb 2012 at 08:32:07
>>>>>
>>>>>> Rajeev and Fab,
>>>>>>
>>>>>> yes, you are both right.
>>>>>> Then we need a change-log like
>>>>>>
>>>>>>>> Section 3.7 on page 48.[[BR]]
>>>>>>>> It is clarified that MPI_Wait and MPI_Test set the request handle
>>>>>>>> to
>>>>>>>> MPI_REQUEST_NULL if also a generalized or I/O request is
>>>>>>>> completed.
>>>>>> ----- Original Message -----
>>>>>>> From: "Rajeev Thakur" <thakur at mcs.anl.gov> To: "Main MPI Forum
>>>>>>> mailing
>>>>>>> list" <mpi-forum at lists.mpi-forum.org> Sent: Monday, February 13,
>>>>>>> 2012
>>>>>>> 3:32:47 PM Subject: Re: [Mpi-forum] MPI_WAIT/MPI_TEST and
>>>>>>> generalized,
>>>>>>> I/O, and NBC requests Nonblocking *independent* I/O requests
>>>>>>> existed in
>>>>>>> MPI-2. Nonblocking collective I/O may come in MPI-3.
>>>>>>>
>>>>>>> Rajeev
>>>>>>>
>>>>>>> On Feb 13, 2012, at 7:20 AM, Rolf Rabenseifner wrote:
>>>>>>>
>>>>>>>>> Non-blocking I/O requests weren't covered by the text in the 2.2
>>>>>>>>> standard, so it's not just generalized requests that were
>>>>>>>>> missed.
>>>>>>>>
>>>>>>>> Non-blocking I/O requests did not exist in MPI-2.2.
>>>>>>>> They may come with MPI-3.0.
>>>>>>>>
>>>>>>>> Yes, your text is perfect.
>>>>>>>> Do you file the ticket?
>>>>>>>> Please use official MPI-2.2 page and line numbers,
>>>>>>>> as in nearly all other tickets.
>>>>>>>> I would say, it is enough when you copy the paragraphes
>>>>>>>> mentioned in may email into the ticket.
>>>>>>>>
>>>>>>>> As Change-log, I would recommend:
>>>>>>>>
>>>>>>>> Section 3.7 on page 48.[[BR]]
>>>>>>>> It is clarified that MPI_Wait and MPI_Test set the request handle
>>>>>>>> to
>>>>>>>> MPI_REQUEST_NULL if a generalized request is completed.
>>>>>>>>
>>>>>>>> Best regards
>>>>>>>> Rolf
>>>>>>>>
>>>>>>>>
>>>>>>>> ----- Original Message -----
>>>>>>>>> From: "Fab Tillier" <ftillier at microsoft.com> To: "Main MPI Forum
>>>>>>>>> mailing list" <mpi-forum at lists.mpi-forum.org> Sent: Monday,
>>>>>>>>> February 13, 2012 12:45:13 AM Subject: Re: [Mpi-forum]
>>>>>>>>> MPI_WAIT/MPI_TEST and generalized, I/O, and NBC requests
>>>>>>>>>
>>>>>>>>> Non-blocking I/O requests weren't covered by the text in the 2.2
>>>>>>>>> standard, so it's not just generalized requests that were
>>>>>>>>> missed.
>>>>>>>>> Other than that, you have it correct Rolf. I would word it
>>>>>>>>> slightly
>>>>>>>>> differently, though, something along the lines of:
>>>>>>>>>
>>>>>>>>> "If the operation associated with this request was a persistent
>>>>>>>>> communication operation, the persistent communication request is
>>>>>>>>> marked as inactive. Other nonblocking operations are deallocated
>>>>>>>>> and
>>>>>>>>> the request handle is set to MPI_REQUEST_NULL."
>>>>>>>>>
>>>>>>>>> Basically, mention the special case of persistent requests
>>>>>>>>> first,
>>>>>>>>> so
>>>>>>>>> as to avoid the negative (non-persistent requests).
>>>>>>>>>
>>>>>>>>> -Fab
>>>>>>>>>
>>>>>>>>> Rolf Rabenseifner wrote on Sun, 12 Feb 2012 at 05:46:17
>>>>>>>>>
>>>>>>>>>> When I understand correctly, then we have two problems:
>>>>>>>>>>
>>>>>>>>>> 1. MPI-2.2 has the "bug" that MPI-2.2
>>>>>>>>>> page 53 line 47 - page 54 line 3, and page 54 lines 40-45 do
>>>>>>>>>> not
>>>>>>>>>> mention the generalize requests although it was intended that
>>>>>>>>>> MPI_Wait or MPI_Test of a generalized request acts as it would
>>>>>>>>>> have been a isend or irecv request. 2. Nobody has checked for
>>>>>>>>>> nonblocking collectives and nonblocking I/O that this text
>>>>>>>>>> must
>>>>>>>>>> now also include those routines.
>>>>>>>>>> When I also understand correctly, then we currently say
>>>>>>>>>>
>>>>>>>>>> "If the communication object associated with this request
>>>>>>>>>> was created by a nonblocking send or receive all,
>>>>>>>>>> then ..."
>>>>>>>>>> but we wanted to say
>>>>>>>>>>
>>>>>>>>>> "If the communication object associated with this request
>>>>>>>>>> is not a persistent communication request (see Section 3.9 on
>>>>>>>>>> page 69),
>>>>>>>>>> then ..."
>>>>>>>>>> Yes?
>>>>>>>>>>
>>>>>>>>>> This change would solve both problems. Yes?
>>>>>>>>>>
>>>>>>>>>> Best regards
>>>>>>>>>> Rolf
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>> From: "Bronis R. de Supinski" <bronis at llnl.gov> To: "Fab
>>>>>>>>>>> Tillier"
>>>>>>>>>>> <ftillier at microsoft.com> Cc: "Main MPI Forum mailing list"
>>>>>>>>>>> <mpi-forum at lists.mpi-forum.org> Sent: Saturday, February 11,
>>>>>>>>>>> 2012
>>>>>>>>>>> 7:57:30 PM Subject: Re: [Mpi-forum] MPI_WAIT/MPI_TEST and
>>>>>>>>>>> generalized,
>>>>>>>>>>> I/O, and NBC requests I will leave it to you to decide. I am
>>>>>>>>>>> sure
>>>>>>>>>>> I
>>>>>>>>>>> will be very pressed for time...
>>>>>>>>>>>
>>>>>>>>>>> On Sat, 11 Feb 2012, Fab Tillier wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I think it would make more sense to fix the text for the WAIT
>>>>>>>>>>>> and
>>>>>>>>>>>> TEST routines, rather than spread the information around. The
>>>>>>>>>>>> generalized request section might not need changes if we do
>>>>>>>>>>>> this.
>>>>>>>>>>>>
>>>>>>>>>>>> I don't know if I'll have time to put the text together
>>>>>>>>>>>> before
>>>>>>>>>>>> the
>>>>>>>>>>>> meeting, though.
>>>>>>>>>>>>
>>>>>>>>>>>> -Fab
>>>>>>>>>>>>
>>>>>>>>>>>> Bronis R. de Supinski wrote on Sat, 11 Feb 2012 at 10:13:48
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am fine with the decision to make it the chapter committee
>>>>>>>>>>>>> responsibility. I suppose that means I need to draft text
>>>>>>>>>>>>> for the generalized requests?
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sat, 11 Feb 2012, William Gropp wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> It looks like this is a result of adding new request types
>>>>>>>>>>>>>> without
>>>>>>>>>>>>>> revisiting
>>>>>>>>>>>>>> the original text. Under generalized requests, there is
>>>>>>>>>>>>>> text
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> seems to
>>>>>>>>>>>>>> imply that the request is set to MPI_REQUEST_NULL on
>>>>>>>>>>>>>> completion,
>>>>>>>>>>>>>> but it
>>>>>>>>>>>>>> isn't explicit and should be.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This is a chapter committee correction.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Bill
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Feb 10, 2012, at 6:04 PM, Fab Tillier wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Folks,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Looking at the text for MPI_WAIT (page 54, line 15) and
>>>>>>>>>>>>>> MPI_TEST
>>>>>>>>>>>>>> (page 55, line 7), it seems to imply that the request
>>>>>>>>>>>>>> handle is
>>>>>>>>>>>>>> only set to MPI_REQUEST_NULL for non-blocking send and
>>>>>>>>>>>>>> receive
>>>>>>>>>>>>>> requests. Are generalized, I/O, and NBC requests not
>>>>>>>>>>>>>> completed
>>>>>>>>>>>>>> the same way? Are users required to free such requests
>>>>>>>>>>>>>> explicitly after they complete using MPI_REQUEST_FREE?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I would have expected that all request handles except those
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> persistent
>>>>>>>>>>>>>> requests are set to MPI_REQUEST_NULL, and persistent
>>>>>>>>>>>>>> request
>>>>>>>>>>>>>> just
>>>>>>>>>>>>>> rather
>>>>>>>>>>>>>> marked inactive. If that is indeed the intent, shouldn't we
>>>>>>>>>>>>>> update
>>>>>>>>>>>>>> the text to
>>>>>>>>>>>>>> reflect this? Is this a ticket-0 level change?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -Fab
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________ mpi-forum
>>>>>>>>>>>>>> mailing list mpi-forum at lists.mpi-forum.org<mailto:mpi-
>>>>>>>>>>>>>> forum at lists.mpi- forum.org>
>>>>>>>>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> William Gropp
>>>>>>>>>>>>>> Director, Parallel Computing Institute
>>>>>>>>>>>>>> Deputy Director for Research
>>>>>>>>>>>>>> Institute for Advanced Computing Applications and
>>>>>>>>>>>>>> Technologies
>>>>>>>>>>>>>> Paul and Cynthia Saylor Professor of Computer Science
>>>>>>>>>>>>>> University of Illinois Urbana-Champaign
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> mpi-forum mailing list
>>>>>>>>>>>>> mpi-forum at lists.mpi-forum.org
>>>>>>>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> mpi-forum mailing list
>>>>>>>>>>> mpi-forum at lists.mpi-forum.org
>>>>>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> mpi-forum mailing list
>>>>>>>>> mpi-forum at lists.mpi-forum.org
>>>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>>>>>>>>
>>>>>>>> --
>>>>>>>> Dr. Rolf Rabenseifner . . . . . . . . . .. email
>>>>>>>> rabenseifner at hlrs.de
>>>>>>>> High Performance Computing Center (HLRS) . phone
>>>>>>>> ++49(0)711/685-65530
>>>>>>>> University of Stuttgart . . . . . . . . .. fax ++49(0)711 /
>>>>>>>> 685-65832
>>>>>>>> Head of Dpmt Parallel Computing . . .
>>>>>>>> www.hlrs.de/people/rabenseifner
>>>>>>>> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring
>>>>>>>> 30)
>>>>>>>> _______________________________________________
>>>>>>>> mpi-forum mailing list
>>>>>>>> mpi-forum at lists.mpi-forum.org
>>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> mpi-forum mailing list
>>>>>>> mpi-forum at lists.mpi-forum.org
>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> mpi-forum mailing list
>>>>> mpi-forum at lists.mpi-forum.org
>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>>>>
>>>> William Gropp
>>>> Director, Parallel Computing Institute
>>>> Deputy Director for Research
>>>> Institute for Advanced Computing Applications and Technologies
>>>> Paul and Cynthia Saylor Professor of Computer Science
>>>> University of Illinois Urbana-Champaign
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> mpi-forum mailing list
>>>> mpi-forum at lists.mpi-forum.org
>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>>>
>>> --
>>> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de
>>> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530
>>> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832
>>> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner
>>> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)
>>> _______________________________________________
>>> mpi-forum mailing list
>>> mpi-forum at lists.mpi-forum.org
>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>>
>>
>> _______________________________________________
>> mpi-forum mailing list
>> mpi-forum at lists.mpi-forum.org
>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>
>
> _______________________________________________
> mpi-forum mailing list
> mpi-forum at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum



-- 
Jeff Hammond
Argonne Leadership Computing Facility
University of Chicago Computation Institute
jhammond at alcf.anl.gov / (630) 252-5381
http://www.linkedin.com/in/jeffhammond
https://wiki.alcf.anl.gov/old/index.php/User:Jhammond




More information about the mpi-forum mailing list