[mpiwg-rma] Meeting Oct 10: Notified communication discussion

[Holmes, Daniel John]

Hi all,

My notes from the meeting today (more detail and/or explanation available on request, or watch the recording):

Notified-RMA feedback 2024_10_10

Jeff: notification for fetch_and_op and compare_and_op?
Overlapping windows.
Fine-grained completion of fetch-and-op without requirement to progress other things
Target or origin side? Origin s/be request-based?

Joseph: is the extra flexibility of changing the number of notifications valuable enough?
Does it compromise the performance of the implementation?
Add MPI_Win_allocate_with_notifications initially; add others later, if needed and proven to work?
Dmitry - allocation and window creation might be in different levels of the application; needs separate APIs

Jeff: MPI_Win_allocate_shared should only be allowed for getting shared memory, not for RMA operations!
This API exists as a wrapper over POSIX shmem; app should do language atomics, not RMA

Dmitry: making the user implement notifications in shared memory is a bad idea
Efficiency is secondary to portability, correctness, etc.

Jeff: Fortran has no memory model, MPI can help by defining notification memory semantics
Personal feeling - Forum will demand all four for orthogonality
Maybe an advice - one of these is better than the others

Joseph: implement "during constructor" and see if it is faster, then decide
Burden of proof - need to prove a negative; hmm

Jeff: no law preventing pre-allocation of N counters during constructor
If user asks for less, great! Otherwise, allocate more.

Alexander: INFO key for constructor to specify expected max number of counters
Also, window memory on device and notifications on host

Joseph: agree that notifications should be at message boundaries
Maybe we can have both kinds [ED: yes, that's orthogonal]

Jeff: can you use IB "send immediate" for put-notify?
Needs recent implementor experience -- maybe there's a size limit?
Ken - size is not consistent across providers, also not exposed in some low-level APIs
Jeff - if network is in-order then two writes is ordered for free
Ken - if out-of-order network, we'll look at it

Jeff: add request-based notified-RMA, restrict from device-initiated when that happens
Alexander - we proposed what we implemented, other things are possible
Jeff - implementation of request-based not necessary for evaluation of this proposal

Best wishes,
Dan.

From: mpiwg-rma <mpiwg-rma-bounces at lists.mpi-forum.org> On Behalf Of Joseph Schuchart via mpiwg-rma
Sent: Tuesday, October 8, 2024 9:34 PM
To: mpiwg-rma <mpiwg-rma at lists.mpi-forum.org>
Cc: Joseph Schuchart <schuchart at icl.utk.edu>
Subject: [mpiwg-rma] Meeting Oct 10: Notified communication discussion

Dear all, The RMA WG will meet again this Thursday (October 10) at 10am US Central / 11am US Eastern / 5pm Central European time to discuss the notified communication proposal put forward by Intel [1]. Dan will lead the discussion. The connection
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
ZjQcmQRYFpfptBannerEnd

Dear all,



The RMA WG will meet again this Thursday (October 10) at 10am US Central

/ 11am US Eastern / 5pm Central European time to discuss the notified

communication proposal put forward by Intel [1]. Dan will lead the

discussion.



The connection information can be found at

https://urldefense.us/v3/__https://github.com/mpiwg-rma/mpi-standard/wiki__;!!G_uCfscf7eWS!fViEO5aL-_8ZRp44IBYKrhwZ9k3R4PeNVKx-N249TvuDUkTTP_EECU_oTellYSddl0cpOx-Otmmw0RkV6SeEgcV2GShjxfqK$<https://urldefense.us/v3/__https:/github.com/mpiwg-rma/mpi-standard/wiki__;!!G_uCfscf7eWS!fViEO5aL-_8ZRp44IBYKrhwZ9k3R4PeNVKx-N249TvuDUkTTP_EECU_oTellYSddl0cpOx-Otmmw0RkV6SeEgcV2GShjxfqK$>. We started using a new

zoom room last time so please make sure you use the link on the github wiki.



Cheers

Joseph





[1] https://urldefense.us/v3/__https://github.com/mpiwg-rma/mpi-standard/pull/11__;!!G_uCfscf7eWS!fViEO5aL-_8ZRp44IBYKrhwZ9k3R4PeNVKx-N249TvuDUkTTP_EECU_oTellYSddl0cpOx-Otmmw0RkV6SeEgcV2GbG3jyzA$<https://urldefense.us/v3/__https:/github.com/mpiwg-rma/mpi-standard/pull/11__;!!G_uCfscf7eWS!fViEO5aL-_8ZRp44IBYKrhwZ9k3R4PeNVKx-N249TvuDUkTTP_EECU_oTellYSddl0cpOx-Otmmw0RkV6SeEgcV2GbG3jyzA$>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-rma/attachments/20241010/179d5542/attachment-0001.html>