<!-- BaNnErBlUrFlE-BoDy-start -->
<!-- Preheader Text : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">
Hi all, My notes from the meeting today (more detail and/or explanation available on request, or watch the recording): Notified-RMA feedback 2024_10_10 Jeff: notification for fetch_and_op and compare_and_op? Overlapping windows. Fine-grained</div>
<!-- Preheader Text : END -->
<!-- Email Banner : BEGIN -->
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerStart</div>
<!--[if ((ie)|(mso))]>
<table border="0" cellspacing="0" cellpadding="0" width="100%" style="padding: 16px 0px 16px 0px; direction: ltr" ><tr><td>
<table border="0" cellspacing="0" cellpadding="0" style="padding: 0px 10px 5px 6px; width: 100%; border-radius:4px; border-top:4px solid #90a4ae;background-color:#D0D8DC;"><tr><td valign="top">
<table align="left" border="0" cellspacing="0" cellpadding="0" style="padding: 4px 8px 4px 8px">
<tr><td style="color:#000000; font-family: 'Arial', sans-serif; font-weight:bold; font-size:14px; direction: ltr">
This Message Is From an External Sender
</td></tr>
<tr><td style="color:#000000; font-weight:normal; font-family: 'Arial', sans-serif; font-size:12px; direction: ltr">
This message came from outside your organization.
</td></tr>
</table>
</td></tr></table>
</td></tr></table>
<![endif]-->
<![if !((ie)|(mso))]>
<div dir="ltr" id="pfptBannerzy5mx7b" style="all: revert !important; display:block !important; text-align: left !important; margin:16px 0px 16px 0px !important; padding:8px 16px 8px 16px !important; border-radius: 4px !important; min-width: 200px !important; background-color: #D0D8DC !important; background-color: #D0D8DC; border-top: 4px solid #90a4ae !important; border-top: 4px solid #90a4ae;">
<div id="pfptBannerzy5mx7b" style="all: unset !important; float:left !important; display:block !important; margin: 0px 0px 1px 0px !important; max-width: 600px !important;">
<div id="pfptBannerzy5mx7b" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-weight:bold !important; font-weight:bold; font-size:14px !important; line-height:18px !important; line-height:18px">
This Message Is From an External Sender
</div>
<div id="pfptBannerzy5mx7b" style="all: unset !important; display:block !important; visibility: visible !important; background-color: #D0D8DC !important; color:#000000 !important; color:#000000; font-weight:normal; font-family: 'Arial', sans-serif !important; font-family: 'Arial', sans-serif; font-size:12px !important; line-height:18px !important; line-height:18px; margin-top:2px !important;">
This message came from outside your organization.
</div>
</div>
<div style="clear: both !important; display: block !important; visibility: hidden !important; line-height: 0 !important; font-size: 0.01px !important; height: 0px"> </div>
</div>
<![endif]>
<div style="display:none !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ffffff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">ZjQcmQRYFpfptBannerEnd</div>
<!-- Email Banner : END -->
<!-- BaNnErBlUrFlE-BoDy-end -->
<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head><!-- BaNnErBlUrFlE-HeAdEr-start -->
<style>
#pfptBannerzy5mx7b { all: revert !important; display: block !important;
visibility: visible !important; opacity: 1 !important;
background-color: #D0D8DC !important;
max-width: none !important; max-height: none !important }
.pfptPrimaryButtonzy5mx7b:hover, .pfptPrimaryButtonzy5mx7b:focus {
background-color: #b4c1c7 !important; }
.pfptPrimaryButtonzy5mx7b:active {
background-color: #90a4ae !important; }
</style>
<!-- BaNnErBlUrFlE-HeAdEr-end -->
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Hi all,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">My notes from the meeting today (more detail and/or explanation available on request, or watch the recording):<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Notified-RMA feedback 2024_10_10<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Jeff: notification for fetch_and_op and compare_and_op?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Overlapping windows.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Fine-grained completion of fetch-and-op without requirement to progress other things<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Target or origin side? Origin s/be request-based?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Joseph: is the extra flexibility of changing the number of notifications valuable enough?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Does it compromise the performance of the implementation?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Add MPI_Win_allocate_with_notifications initially; add others later, if needed and proven to work?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Dmitry – allocation and window creation might be in different levels of the application; needs separate APIs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Jeff: MPI_Win_allocate_shared should only be allowed for getting shared memory, not for RMA operations!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">This API exists as a wrapper over POSIX shmem; app should do language atomics, not RMA<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Dmitry: making the user implement notifications in shared memory is a bad idea<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Efficiency is secondary to portability, correctness, etc.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Jeff: Fortran has no memory model, MPI can help by defining notification memory semantics<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Personal feeling – Forum will demand all four for orthogonality<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Maybe an advice – one of these is better than the others<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Joseph: implement "during constructor" and see if it is faster, then decide<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Burden of proof – need to prove a negative; hmm<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Jeff: no law preventing pre-allocation of N counters during constructor<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">If user asks for less, great! Otherwise, allocate more.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Alexander: INFO key for constructor to specify expected max number of counters<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Also, window memory on device and notifications on host<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Joseph: agree that notifications should be at message boundaries<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Maybe we can have both kinds [ED: yes, that's orthogonal]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Jeff: can you use IB "send immediate" for put-notify?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Needs recent implementor experience -- maybe there's a size limit?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Ken – size is not consistent across providers, also not exposed in some low-level APIs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Jeff – if network is in-order then two writes is ordered for free<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Ken – if out-of-order network, we'll look at it<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Jeff: add request-based notified-RMA, restrict from device-initiated when that happens<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Alexander – we proposed what we implemented, other things are possible<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Jeff – implementation of request-based not necessary for evaluation of this proposal<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Best wishes,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US">Dan.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> mpiwg-rma <mpiwg-rma-bounces@lists.mpi-forum.org>
<b>On Behalf Of </b>Joseph Schuchart via mpiwg-rma<br>
<b>Sent:</b> Tuesday, October 8, 2024 9:34 PM<br>
<b>To:</b> mpiwg-rma <mpiwg-rma@lists.mpi-forum.org><br>
<b>Cc:</b> Joseph Schuchart <schuchart@icl.utk.edu><br>
<b>Subject:</b> [mpiwg-rma] Meeting Oct 10: Notified communication discussion<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">Dear all, The RMA WG will meet again this Thursday (October 10) at 10am US Central / 11am US Eastern / 5pm Central European time to discuss the notified communication
proposal put forward by Intel [1]. Dan will lead the discussion. The connection<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">ZjQcmQRYFpfptBannerStart<o:p></o:p></span></p>
</div>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="100%" style="width:100.0%;border-radius:4px">
<tbody>
<tr>
<td style="padding:12.0pt 0cm 12.0pt 0cm">
<table class="MsoNormalTable" border="1" cellspacing="0" cellpadding="0" width="100%" style="width:100.0%;background:#D0D8DC;border:none;border-top:solid #90A4AE 3.0pt">
<tbody>
<tr>
<td valign="top" style="border:none;padding:0cm 7.5pt 3.75pt 4.5pt">
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" align="left">
<tbody>
<tr>
<td style="padding:3.0pt 6.0pt 3.0pt 6.0pt">
<p class="MsoNormal"><b><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:black">This Message Is From an External Sender
<o:p></o:p></span></b></p>
</td>
</tr>
<tr>
<td style="padding:3.0pt 6.0pt 3.0pt 6.0pt">
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:black">This message came from outside your organization.
<o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
<div>
<p class="MsoNormal" style="mso-line-height-alt:.75pt"><span style="font-size:1.0pt;color:white">ZjQcmQRYFpfptBannerEnd<o:p></o:p></span></p>
</div>
<pre style="white-space:pre-wrap"><span style="font-size:12.0pt;font-family:"Arial",sans-serif">Dear all,<o:p></o:p></span></pre>
<pre><span style="font-size:12.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></pre>
<pre><span style="font-size:12.0pt;font-family:"Arial",sans-serif">The RMA WG will meet again this Thursday (October 10) at 10am US Central <o:p></o:p></span></pre>
<pre><span style="font-size:12.0pt;font-family:"Arial",sans-serif">/ 11am US Eastern / 5pm Central European time to discuss the notified <o:p></o:p></span></pre>
<pre><span style="font-size:12.0pt;font-family:"Arial",sans-serif">communication proposal put forward by Intel [1]. Dan will lead the <o:p></o:p></span></pre>
<pre><span style="font-size:12.0pt;font-family:"Arial",sans-serif">discussion.<o:p></o:p></span></pre>
<pre><span style="font-size:12.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></pre>
<pre><span style="font-size:12.0pt;font-family:"Arial",sans-serif">The connection information can be found at <o:p></o:p></span></pre>
<pre><span style="font-size:12.0pt;font-family:"Arial",sans-serif"><a href="https://urldefense.us/v3/__https:/github.com/mpiwg-rma/mpi-standard/wiki__;!!G_uCfscf7eWS!fViEO5aL-_8ZRp44IBYKrhwZ9k3R4PeNVKx-N249TvuDUkTTP_EECU_oTellYSddl0cpOx-Otmmw0RkV6SeEgcV2GShjxfqK$">https://urldefense.us/v3/__https://github.com/mpiwg-rma/mpi-standard/wiki__;!!G_uCfscf7eWS!fViEO5aL-_8ZRp44IBYKrhwZ9k3R4PeNVKx-N249TvuDUkTTP_EECU_oTellYSddl0cpOx-Otmmw0RkV6SeEgcV2GShjxfqK$</a>. We started using a new <o:p></o:p></span></pre>
<pre><span style="font-size:12.0pt;font-family:"Arial",sans-serif">zoom room last time so please make sure you use the link on the github wiki.<o:p></o:p></span></pre>
<pre><span style="font-size:12.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></pre>
<pre><span style="font-size:12.0pt;font-family:"Arial",sans-serif">Cheers<o:p></o:p></span></pre>
<pre><span style="font-size:12.0pt;font-family:"Arial",sans-serif">Joseph<o:p></o:p></span></pre>
<pre><span style="font-size:12.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></pre>
<pre><span style="font-size:12.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></pre>
<pre><span style="font-size:12.0pt;font-family:"Arial",sans-serif">[1] <a href="https://urldefense.us/v3/__https:/github.com/mpiwg-rma/mpi-standard/pull/11__;!!G_uCfscf7eWS!fViEO5aL-_8ZRp44IBYKrhwZ9k3R4PeNVKx-N249TvuDUkTTP_EECU_oTellYSddl0cpOx-Otmmw0RkV6SeEgcV2GbG3jyzA$">https://urldefense.us/v3/__https://github.com/mpiwg-rma/mpi-standard/pull/11__;!!G_uCfscf7eWS!fViEO5aL-_8ZRp44IBYKrhwZ9k3R4PeNVKx-N249TvuDUkTTP_EECU_oTellYSddl0cpOx-Otmmw0RkV6SeEgcV2GbG3jyzA$</a><o:p></o:p></span></pre>
</div>
</body>
</html>