<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-CA" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">This is correct, you’re allowed to call test/wait whenever you want as long as you abide by the rules that all pready calls must be made in order for completion to happen. So don’t MPI_Wait on an operation to complete when you haven’t called
all of the pready calls (that the process/thread is responsible for) from that process/thread yet (similar to your responsibilities to make sure Send/Recv don’t deadlock, but obviously a different semantic for completion).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">-Ryan<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">James Dinan <james.dinan@gmail.com><br>
<b>Date: </b>Wednesday, October 12, 2022 at 6:43 AM<br>
<b>To: </b>Joachim Jenke <protze@itc.rwth-aachen.de>, Ryan Grant <ryan.grant@queensu.ca><br>
<b>Cc: </b>"mpiwg-persistence@lists.mpi-forum.org" <mpiwg-persistence@lists.mpi-forum.org><br>
<b>Subject: </b>Re: [mpiwg-persistence] Completion call before all partitions marked ready<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">This came up recently and I believe it is allowed to wait/test a send request before all partitions are marked as ready. It's not explicitly stated, rather there is no restriction disallowing this usage. It may be good to clarify this semantic
(e.g. in Section 4.2.2) for MPI 4.1 since this is a topic that the WG debated at length.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><a href="mailto:ryan.grant@queensu.ca">@Ryan Grant</a> Please fact check.<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> ~Jim.<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Mon, Oct 10, 2022 at 7:22 AM Joachim Jenke via mpiwg-persistence <<a href="mailto:mpiwg-persistence@lists.mpi-forum.org">mpiwg-persistence@lists.mpi-forum.org</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">Hello wg-persistence,<br>
<br>
Looking at the MPI 4.0 document, it is not clear to us, whether it is <br>
allowed to call a completion call for a partitioned communication <br>
request before all partitions are marked ready. A simple single-threaded <br>
example would be:<br>
<br>
```C<br>
MPI_Psend_init(message, partitions, COUNT, MPI_DOUBLE, dest, tag,<br>
MPI_COMM_WORLD, MPI_INFO_NULL, &request);<br>
MPI_Start(&request);<br>
for(i = 0; i < partitions-1; ++i)<br>
{<br>
MPI_Pready(i, request);<br>
}<br>
MPI_Test(&request, &flag, MPI_STATUS_IGNORE); // flag will always be 0<br>
MPI_Pready(partitions-1, request);<br>
MPI_Wait(&request, MPI_STATUS_IGNORE);<br>
MPI_Request_free(&request);<br>
```<br>
<br>
The question becomes more relevant in a multi-threaded context. One <br>
thread could finish the work early and call MPI_Wait to detect when all <br>
partitions were sent.<br>
<br>
From my understanding, the only requirement is that all partitions must <br>
be marked ready with explicit ready calls before the operation can <br>
complete. Replacing the test in above example with a wait call would <br>
result in deadlock.<br>
<br>
Best<br>
Joachim<br>
<br>
-- <br>
Dr. rer. nat. Joachim Jenke<br>
<br>
IT Center<br>
Group: High Performance Computing<br>
Division: Computational Science and Engineering<br>
RWTH Aachen University<br>
Seffenter Weg 23<br>
D 52074 Aachen (Germany)<br>
Tel: +49 241 80- 24765<br>
Fax: +49 241 80-624765<br>
<a href="mailto:protze@itc.rwth-aachen.de" target="_blank">protze@itc.rwth-aachen.de</a><br>
<a href="https://can01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.itc.rwth-aachen.de%2F&data=05%7C01%7Cryan.grant%40queensu.ca%7C6464a2e4284646255bf008daac3e940c%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C638011682013760361%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=n%2B4uUA704BGsh4zIbm7w0Fzs0S8rzlWv13LQX%2BzXF0I%3D&reserved=0" target="_blank">www.itc.rwth-aachen.de</a><br>
_______________________________________________<br>
mpiwg-persistence mailing list<br>
<a href="mailto:mpiwg-persistence@lists.mpi-forum.org" target="_blank">mpiwg-persistence@lists.mpi-forum.org</a><br>
<a href="https://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mpi-forum.org%2Fmailman%2Flistinfo%2Fmpiwg-persistence&data=05%7C01%7Cryan.grant%40queensu.ca%7C6464a2e4284646255bf008daac3e940c%7Cd61ecb3b38b142d582c4efb2838b925c%7C1%7C0%7C638011682013760361%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iOVtiUL9aSL7Bu%2BmeC6dRWfAARYfpi5pxGbGFjTf4uM%3D&reserved=0" target="_blank">https://lists.mpi-forum.org/mailman/listinfo/mpiwg-persistence</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</body>
</html>