<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body>
<div>
<div>
<div dir="ltr">
<div dir="ltr">Hi all — we can make a ticket —- Iets discuss in The WG as soon as next slot for this on a Wednesday … </div>
<div dir="ltr"><br>
</div>
<div dir="ltr">Tony </div>
</div>
</div>
<div id="ms-outlook-mobile-signature">
<div><br>
</div>
<span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif; font-size: inherit;">Anthony Skjellum, PhD</span>
<div dir="auto">
<p style="caret-color:rgb(0, 0, 0);color:rgb(0, 0, 0);font-family:Calibri, Helvetica, sans-serif;margin-top:0px;margin-bottom:0px">
Professor of Computer Science and Chair of Excellence</p>
<p style="caret-color:rgb(0, 0, 0);color:rgb(0, 0, 0);font-family:Calibri, Helvetica, sans-serif;margin-top:0px;margin-bottom:0px">
Director, SimCenter</p>
<p style="caret-color:rgb(0, 0, 0);color:rgb(0, 0, 0);font-family:Calibri, Helvetica, sans-serif;margin-top:0px;margin-bottom:0px">
University of Tennessee at Chattanooga (UTC)</p>
<p style="caret-color:rgb(0, 0, 0);color:rgb(0, 0, 0);font-family:Calibri, Helvetica, sans-serif;margin-top:0px;margin-bottom:0px">
<span dir="ltr" style="color:rgb(0, 120, 212)">tony-skjellum@utc.edu</span><span> </span> [or<span> </span><span dir="ltr" style="color:rgb(0, 120, 212)">skjellum@gmail.com</span>]</p>
<p style="caret-color:rgb(0, 0, 0);color:rgb(0, 0, 0);font-family:Calibri, Helvetica, sans-serif;margin-top:0px;margin-bottom:0px">
cell:<span> </span><span dir="ltr" style="color:rgb(0, 120, 212)">205-807-4968</span></p>
<br>
</div>
</div>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> mpiwg-persistence <mpiwg-persistence-bounces@lists.mpi-forum.org> on behalf of Holmes, Daniel John via mpiwg-persistence <mpiwg-persistence@lists.mpi-forum.org><br>
<b>Sent:</b> Tuesday, January 24, 2023 6:59:52 AM<br>
<b>To:</b> Joachim Protze <protze@itc.rwth-aachen.de>; mpiwg-persistence@lists.mpi-forum.org <mpiwg-persistence@lists.mpi-forum.org><br>
<b>Subject:</b> Re: [mpiwg-persistence] Completion call before all partitions marked ready</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">Hi Joachim,<br>
<br>
That exact question was raised and discussed in the Forum during the standardisation procedure for partitioned communication.<br>
<br>
The resulting intent, IMHO, was that calling a completion procedure before all partitions are marked ready is permitted, but will always respond with its "not completed" semantic -- MPI_TEST will set flag==false and MPI_WAIT will not return (until something
changes -- in a multithreaded situation, another thread could make calls to MPI_Pready and release MPI_WAIT from what would otherwise be a deadlock).<br>
<br>
This modus operandi may be important for offload to accelerators, where the "other thread" that calls MPI_Pready is actually a kernel running on an accelerator device (once we have accelerator bindings for MPI_Pready and can guarantee sufficient preparedness
of the partitioned communication operation that such a device call makes sense, which are both work in progress).<br>
<br>
As this question has arisen several times from different people, I would say that the intent is not stated in the document clearly enough. We should work on writing some better wording. Given that the intended semantics would not be changing, we might still
have time to get a wording fix proposal into MPI-4.1 -- it needs to be read at the March voting meeting to hit that goal.<br>
<br>
Best wishes,<br>
Dan.<br>
<br>
-----Original Message-----<br>
From: mpiwg-persistence <mpiwg-persistence-bounces@lists.mpi-forum.org> On Behalf Of Joachim Protze via mpiwg-persistence<br>
Sent: 21 September 2022 08:21<br>
To: mpiwg-persistence@lists.mpi-forum.org<br>
Subject: [mpiwg-persistence] Completion call before all partitions marked ready<br>
<br>
Hello wg-persistence,<br>
<br>
Looking at the MPI 4.0 document, it is not clear to us, whether it is allowed to call a completion call for a partitioned communication request before all partitions are marked ready. A simple single-threaded example would be:<br>
<br>
```C<br>
MPI_Psend_init(message, partitions, COUNT, MPI_DOUBLE, dest, tag,<br>
MPI_COMM_WORLD, MPI_INFO_NULL, &request); MPI_Start(&request); for(i = 0; i < partitions-1; ++i) {<br>
MPI_Pready(i, request);<br>
}<br>
MPI_Test(&request, &flag, MPI_STATUS_IGNORE); // flag will always be 0 MPI_Pready(partitions-1, request); MPI_Wait(&request, MPI_STATUS_IGNORE); MPI_Request_free(&request); ```<br>
<br>
The question becomes more relevant in a multi-threaded context. One thread could finish the work early and call MPI_Wait to detect when all partitions were sent.<br>
<br>
From my understanding, the only requirement is that all partitions must be marked ready with explicit ready calls before the operation can complete. Replacing the test in above example with a wait call would result in deadlock.<br>
<br>
Best<br>
Joachim<br>
<br>
--<br>
Dr. rer. nat. Joachim Protze<br>
<br>
IT Center<br>
Group: High Performance Computing<br>
Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany)<br>
Tel: +49 241 80- 24765<br>
Fax: +49 241 80-624765<br>
protze@itc.rwth-aachen.de<br>
<a href="https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.itc.rwth-aachen.de%2F&data=05%7C01%7Ctony-skjellum%40utc.edu%7Cc34dfba827dd46a52fee08dafe028d2f%7C515813d9717d45dd9eca9aa19c09d6f9%7C0%7C0%7C638101584165642512%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hvTklQ2nEO1VtbS2eo2kBu2PZ7XW56mo%2Ba5r3Vevf28%3D&reserved=0">https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.itc.rwth-aachen.de%2F&data=05%7C01%7Ctony-skjellum%40utc.edu%7Cc34dfba827dd46a52fee08dafe028d2f%7C515813d9717d45dd9eca9aa19c09d6f9%7C0%7C0%7C638101584165642512%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hvTklQ2nEO1VtbS2eo2kBu2PZ7XW56mo%2Ba5r3Vevf28%3D&reserved=0</a><br>
_______________________________________________<br>
mpiwg-persistence mailing list<br>
mpiwg-persistence@lists.mpi-forum.org<br>
<a href="https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mpi-forum.org%2Fmailman%2Flistinfo%2Fmpiwg-persistence&data=05%7C01%7Ctony-skjellum%40utc.edu%7Cc34dfba827dd46a52fee08dafe028d2f%7C515813d9717d45dd9eca9aa19c09d6f9%7C0%7C0%7C638101584165642512%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=M2uWuXaMjgG%2BBeIA8F349wMQxgDMG1J3xzVP2k4YYcE%3D&reserved=0">https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mpi-forum.org%2Fmailman%2Flistinfo%2Fmpiwg-persistence&data=05%7C01%7Ctony-skjellum%40utc.edu%7Cc34dfba827dd46a52fee08dafe028d2f%7C515813d9717d45dd9eca9aa19c09d6f9%7C0%7C0%7C638101584165642512%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=M2uWuXaMjgG%2BBeIA8F349wMQxgDMG1J3xzVP2k4YYcE%3D&reserved=0</a><br>
</div>
</span></font></div>
</body>
</html>