<html>
<head>
<style>
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 10pt;
font-family:Verdana
}
</style>
</head>
<body class='hmmessage'>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><FONT face="Times New Roman, serif"><FONT size=3>Here is the draft of the FAQ that I will add to the proposal</FONT></FONT></P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><FONT face="Times New Roman" size=3></FONT> </P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><FONT face="Times New Roman, serif"><FONT size=3>1. Will this work on non-cache coherent systems?</FONT></FONT></P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><FONT face="Times New Roman, serif"><FONT size=3>This will require either a remote locking mechanism (some remote atomic operation) or support for threads that can receive messages to work on a non-cache coherent system. The latter would certainly be more efficient. In the absence of either, it will have to rely on MPI progress and hence semantics will be very difficult to guarantee. A presentation about this to follow in the December forum meeting</FONT></FONT></P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><BR><BR></P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><FONT face="Times New Roman, serif"><FONT size=3>2. Will it be sufficient to support PGAS models?</FONT></FONT></P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><FONT face="Times New Roman, serif"><FONT size=3>In addition to put/get/acc for different data types, and remote atomic operations many pgas models requires some kind of remote method invocation (like AM in GasNET and GPC in ARMCI). This proposal doesn't explicitly propose remote methods. However it is not far fetched to think that an RMA_xfer call that is support PUT/GET/ACC op types may in the future support RMI optype as well. An RMI optype for this call was in consideration but was not proposed yet as we had issues guaranteeing ubiquity. With additional discussions in the forum we can certainly include something like RMI. into this call. Once RMI is added it will be sufficient to easily replace most existing RMA models. (Note that it may have to rely on other things like non-blocking collectives that have been proposed). Once we do implement a prototype, we will demonstrate how this can replace some of the existing RMA models.</FONT></FONT></P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><BR><BR></P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><FONT face="Times New Roman, serif"><FONT size=3>3. Support for heterogeneous architectures</FONT></FONT></P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><FONT face="Times New Roman, serif"><FONT size=3>This is tied into MPI's support for heterogeneous architectures. </FONT></FONT></P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><BR><BR></P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><FONT face="Arial, sans-serif"><FONT size=2>4. </FONT></FONT><FONT face="Times New Roman, serif"><FONT size=3>does the target know of completion</FONT></FONT></P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><FONT face="Times New Roman, serif"><FONT size=3>A collective remote complete will complete all outstanding one-sided messsages. Peer-to-peer remote completion notification is however not directly addressed in the proposal. Cray also brought up remote notification as an option. Certainly this can be implemented with a flag that is updated remotely to indicate remote completion. But to make this a part of an RMA call (ie, making it an attribute) will require contexts. Hence one may need to use tags in RMA calls. This interface can certainly be modified to include an tag as a parameter. The remote side can then use this tag to check for completion.</FONT></FONT></P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"> </P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><FONT face="Times New Roman, serif"><FONT size=3>5. how does it interplay with existing RMA spec</FONT></FONT></P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><FONT face="Arial, sans-serif"><FONT size=2>We didn't intend this to be compared to rDma spec's, they I think are more low level. DAPL, VERBS, OpenRDMA etc have a different purpose.</FONT></FONT></P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><BR><BR></P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><FONT face="Arial, sans-serif"><FONT size=2>6. memory allocation</FONT></FONT></P>
<P style="MARGIN-TOP: 0.07in; MARGIN-BOTTOM: 0.07in"><FONT face="Arial, sans-serif"><FONT size=2>The proposal has a section about how special memory allocators may be useful</FONT></FONT></P>
<P style="MARGIN-BOTTOM: 0in"><BR></P><BR><BR>
<HR id=stopSpelling>
From: thakur@mcs.anl.gov<BR>To: mpi3-rma@lists.mpi-forum.org<BR>Date: Mon, 8 Dec 2008 12:04:14 -0600<BR>Subject: [Mpi3-rma] FW: draft of a proposal for RMA interfaces<BR><BR>
<DIV dir=ltr align=left><SPAN class=EC_608520218-08122008><FONT face=Arial color=#0000ff size=2>Just resending this as a reminder. It would be good to have an FAQ that answers commonly asked questions.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=EC_608520218-08122008><FONT face=Arial color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=EC_608520218-08122008><FONT face=Arial color=#0000ff size=2>Rajeev</FONT></SPAN></DIV><BR>
<DIV class=EC_OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR>
<FONT face=Tahoma size=2><B>From:</B> Rajeev Thakur [mailto:thakur@mcs.anl.gov] <BR><B>Sent:</B> Sunday, October 19, 2008 6:43 PM<BR><B>To:</B> 'MPI 3.0 Remote Memory Access working group'<BR><B>Subject:</B> RE: [Mpi3-rma] draft of a proposal for RMA interfaces<BR></FONT><BR></DIV>
<DIV></DIV>
<FONT size=2>I think it would be good to add an FAQ section at the end, containing answers to questions that will be asked of any RMA proposal, such as non-cache-coherent, does it meet the needs of PGAS/Global Arrays, support for heterogeneous, how does the target know of completion, how does it interplay with existing RMA spec, etc. It will make sure that the proposal addresses those issues, that we ourselves are clear of the answers, and that they are not repeatedly raised at each meeting.</FONT><BR>
<FONT size=2>Rajeev </FONT><BR><BR>
<FONT size=2>> Richard Graham wrote:</FONT> <BR><FONT size=2>></FONT> <BR><FONT size=2>>> Just to get discussion going again. Talking with several folks I have </FONT><BR><FONT size=2>>> heard several concerns expressed about the proposal. I think it would </FONT><BR><FONT size=2>>> be good if these (and others) could be raised on the list, so we can </FONT><BR><FONT size=2>>> start discussion. We can continue this next week in Chicago, but </FONT><BR><FONT size=2>>> Vinod will not be able to make this meeting, so an e-mail discussion </FONT><BR><FONT size=2>>> will help.</FONT> <BR><FONT size=2>>></FONT> <BR><FONT size=2>>> Here are the issues I have hear of so far:</FONT> <BR><FONT size=2>>> - May not work well on current h/w that is not cache coherent, as it </FONT><BR><FONT size=2>>> requires a remote thread in this case. I believe this is for the SX </FONT><BR><FONT size=2>>> series of machines, but Jesper please correct me if I am wrong here. </FONT><BR><FONT size=2>>> What would be an alternative approach that could provide expected </FONT><BR><FONT size=2>>> performance on platforms that may require work on the remote end for </FONT><BR><FONT size=2>>> RMA for correctness, and work well on platforms that do require very </FONT><BR><FONT size=2>>> specific remote cache management (or other actions) for correctness ?</FONT> <BR><FONT size=2>>> - Concern about future high-end platforms, under that assumption that </FONT><BR><FONT size=2>>> these will not be cache coherent (and will actually have caches – if </FONT><BR><FONT size=2>>> they don’t this is not a concern), and therefore this proposal is </FONT><BR><FONT size=2>>> aimed at a short-lived technical capability.</FONT> <BR><FONT size=2>>> - What is missing ?</FONT> <BR></body>
</html>