[mpiwg-tools] question about MPI_T_EVENT_SET_DROPPED_HANDLER

Tue Jan 26 11:28:50 CST 2021

Hi Ken,

>     Oh, so if I am reading your clarification correctly above, you are saying I should not invoke the dropped handler until a callback for that safety level has been registered. It is possible then that I might never call the dropped handler. Should I be managing dropped event counts for all safety levels?
>     
> Let me see if I can give a more concrete example. Let's say I have an event registration handle allocated, and a callback registered for the lowest safety level (NONE). I also have a dropped handler registered. The event itself is generating instances for higher safety levels (MPI_RESTRICTED, THREAD_SAFE), but because there are no handlers registered for those levels, the events are being dropped.
> 
> Question 1: Should the implementation be recording those dropped events for higher safety levels? The text leads me to believe yes.
> Question 2: Should the implementation delay calling the dropped handler until a callback satisfying the safety level of the dropped events has been registered? Your clarification leads me to believe yes.
> Question 3: If I register a MPI_RESTRICTED callback, will the dropped handler then execute to inform me of all the dropped events at that safety level?
> Question 4: If I instead register a THREAD_SAFE callback, will the dropped handler execute to inform me of all the dropped events at that safety level **as well as MPI_RESTRICTED**?

Let me see whether I can answer you questions, as some are indeed
unclear with the current text/defined behavior.

1) The implementation should (where possible) record whenever an event
could not be recorded and had to be dropped.

2) The implementation does not have to keep separate counters for each
level.

3) The cb_safety argument of the handler is to define "what is
possible for the tool inside this handler at the time of invocation"
and does not imply the safety level events were dropped.

Specifically, the implementation does not have to invoke the handler
with the same safety level as where the event was dropped but (in the
words of Capt. Jack Sparrow) "wait for the opportune moment" to invoke
the handle with the highest leeway for the tool manageable.

As higher level callbacks can fill in for lower levels (i.e.,
ASYNC_SIGNAL_SAFE can be invoked with level THREAD_SAFE) the
implementation can always safely use the one with the lowest level
still higher or equal to the level used when invoking the dropped handler.

If an event is registered for a level that the implementation supports
in general, there should be states in the execution where the dropped
handler can be invoked with the matching safety level, and the
implementation can delay until then.

I hope this already clarifies most of the questions, yet you pointed
out a severe under-specification that we need to fix:

---\/---\/---\/--- here be dragons ---\/---\/---\/---\/---

What happens when a user only registers a callback for NONE, and the
library does not support these in the first place? Then, the library
will never get to a place where it can issue the callback handler with
a user provided user_data pointer.

There could be different solutions for this situation, and each of
them has a problem in our current situation:

1) Add a "user_data" element to the "dropped_handler" registration call.

2) If the MPI implementation does not support a certain level at all,
have the callback registration function return MPI_T_ERR_NOT_SUPPORTED
and refuse the registration in this case.

3) Use the 'user_data' registered with a callback at a safety level
closest to the one where the dropped handler is invoked with.

4) Just accept that the dropped handler will never be called in this case.

Option 1 would be in symmetry with the "free" handler registration,
but would need an API change and a no-no vote to be included in 4.0.
Is that likely to happen?

Option 2 would be possible, yet MPI_T_ERR_NOT_SUPPORTED is not yet
listed in Table 15.7 on page 778 in the correct row (it exists, but is
listed only for MPI_T_SOURCE_*) ... We might handle this as an
oversight and clarify it basically with the release of MPI 4.0 and
include it in 4.1.

Option 3 feels odd, as the we assume something about the
characteristics of 'user_data' for a safety level that might not be
the case and might not be safe.

Option 4 would be possible, yet lacks the feedback to the user that
something went wrong.

What's your opinion? Are there other/better ways to deal with the
situation?

Are there other opinions/ideas in the group on how to resolve this?

Cheers,
Marc-Andre

-- 
Dr. rer. nat. Marc-André Hermanns

IT Center
Group: High Performance Computing
Division: Computational Science and Engineering
RWTH Aachen University
Seffenter Weg 23
52074 Aachen
Phone: +49 241 80-24381
hermanns at itc.rwth-aachen.de
www.itc.rwth-aachen.de

Social Media Kanäle des IT Centers:
https://blog.rwth-aachen.de/itc/
https://www.facebook.com/itcenterrwth
https://www.linkedin.com/company/itcenterrwth
https://twitter.com/ITCenterRWTH
https://www.youtube.com/channel/UCKKDJJukeRwO0LP-ac8x8rQ

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5336 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-tools/attachments/20210126/498c4caf/attachment-0001.p7s>