<html><body>
<p>Proposal for MPI Initialization time assertions in MPI 2.2<br>
<br>
Define two new MPI functions:<br>
<br>
MPI_INIT_ASSERTED(assertions, threads, provided)<br>
[ IN assertions ] assertions about application behavior<br>
[ IN threads] desired level of thread support (integer)<br>
[ OUT provided] provided level of thread support (integer)<br>
<br>
int MPI_Init_asserted(int *argc, char *((*argv)[]), int assertions, int threads, int *provided)<br>
<br>
MPI_INIT_ASSERTED(ASSERTIONS, THREADS, PROVIDED, IERROR)<br>
INTEGER ASSERTIONS, THREADS, PROVIDED, IERROR<br>
<br>
(I will need help for C++ bindings )<br>
<br>
The assertions arguments is to be provided as the logical OR of any subset of the assertions defined <br>
by the MPI standard. Zero is valid for an application that does not wish to make any assertion. The <br>
assertions are constants defined in mpi.h. <br>
<br>
MPI_QUERY_ASSERTIONS(stated, exploited)<br>
[ OUT stated] assertions as give in the MPI_INIT_ASSERTED call<br>
[ OUT exploited ] assertions the MPI implementation is exploiting<br>
<br>
int MPI_Query_assertions(int *stated, int *exploited)<br>
<br>
MPI_QUERY_ASSERTIONS(STATED, EXPLOITED, IERROR)<br>
INTEGER STATED, EXPLOITED, IERROR<br>
<br>
The outputs, STATED and EXPLOITED, can be checked for specific assertions with logical<br>
masks. For example to see if the application has used MPI_NO_SEND_CANCEL, library code<br>
could use:<br>
MPI_Query_assertions(&stated, &exploited);<br>
if (stated & MPI_NO_SEND_CANCEL) .... <br>
To confirm that no assertions are being exploited the library could use<br>
if (exploited) ..... error<br>
<br>
====================================================================<br>
Rationale <br>
<br>
There are certain semantic guarantees or features required by the MPI standard that can add significant<br>
memory overheads or performance costs. If an application does not depend on one of these features <br>
and can declare that fact at MPI initialization, the MPI implementation may be able to provide better <br>
performance. Assertions on MPI_INIT_ASSERTED let an application declare it does not require a <br>
specific feature and allow the implementation to omit support for that feature if there is a savings <br>
from doing so.<br>
<br>
When an application calls MPI_INIT_ASSERTED with a particular assertion the MPI implementation may <br>
depend on the application to act as the assertion promises. The behavior of an MPI application that<br>
uses or depends on some feature of MPI after asserting it does not is undefined. The application cannot <br>
depend on the assertion being exploited. The MPI application must be valid for an MPI implementation <br>
that ignores the assertion. <br>
<br>
Advise to implementors -<br>
When it is practical, a high quality MPI implementation will raise an error if an application violates an <br>
assertion that was provided at initialization It will not be practical for the MPI implementation to detect<br>
all assertion violations.<br>
End advise to implementors -<br>
<br>
The following assertions are available (defined in mpi.h) <br>
<br>
MPI_NO_EAGER_THROTTLE <br>
<br>
The MPI standard allows an implementation to send some messages eagerly, meaning the message can<br>
complete at the sender before a matching receive is posted. The implementation is required to hold the<br>
message data until the matching receive gets posted. If there is no space to buffer the data the <br>
send must block. The MPI job must not fail because it has run out of space or lose the data. Providing this <br>
guarantee can demand an expensive flow control.<br>
<br>
In many well structured applications there will never be a need to buffer significant numbers of eager <br>
messages but eager sends do help performance. These applications can tell the MPI implementation the <br>
guarantee is optional. <br>
<br>
<br>
MPI_NO_REDUCTION_DETERMINISM<br>
<br>
The MPI standard either requires or strongly urges that an MPI_REDUCE/MPI_ALLREDUCE give exactly the <br>
same answer every time. There are applications that do not need a "same answer" promise as long as<br>
they can assume they will get a "correct" answer. Maybe they can be provided a faster reduction algorithm.<br>
<br>
MPI_NO_DATATYPE_HETEROGENEITY<br>
<br>
MPI_Datatypes that are made from multiple types can be optimized if it is known there will never be <br>
a need to translate the data between heterogeneous nodes. A structure made of an MPI_INT followed by an <br>
MPI_FLOAT is likely to be 8 contiguous bytes with an extent of 8. An MPI implementation that "knows" it will <br>
not need to deal with data conversion can simplify the datatype commit logic and resulting internal representation <br>
by discarding the MPI_INT/MPI_FLOAT distinctions and just recording that the type is 8 bytes with a stride of 8.<br>
<br>
==============================<br>
<br>
If we add MPI_INIT_ASSERTED to the MPI standard, MPI_INIT and MPI_INIT_THREAD continue to work<br>
as before. MPI_INIT_THREAD is equivalent to MPI_INIT_ASSERTED with zero as the assertion. We can <br>
consider deprecating the older initialization calls.<br>
<br>
Above are the assertions I have decided make most sense for the first pass. There are others that may <br>
be justified but a rationale is needed. If MPI 2.2 defines support for the assertion above we can consider <br>
others as part of MPI 3.0, Below are some That may be justified but will need a better rationale. <br>
<br>
MPI_NO_THREAD_CONTENTION <br>
<br>
(I am not sure this different from asking for MPI_THREAD_SERIALIZED but here is what it would mean)<br>
<br>
If the MPI library may be called by contending threads it must do something to protect internal state<br>
from conflicting updates. Whether this involves mutex locks or some other scheme, it is likely to have<br>
a performance cost. This assertion promises that the application will not make conflicting MPI calls.<br>
The MPI implementation can bypass any logic that defends state against multiple concurrent calls to MPI.<br>
The application may be multi-threaded in other aspects. <br>
<br>
MPI_NO_SEND_CANCEL<br>
<br>
To support MPI_CANCEL of a send, the MPI implementation may need to add extra information to every <br>
message header. If the application can assert that it does not try to cancel any sends then perhaps<br>
less state must be maintained and smaller message headers can be used.<br>
<br>
MPI_NO_ANY_SOURCE<br>
<br>
<br>
<br>
<br>
Dick Treumann - MPI Team/TCEM <br>
IBM Systems & Technology Group<br>
Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601<br>
Tele (845) 433-7846 Fax (845) 433-8363<br>
</body></html>