[Mpi-forum] user-defined reductions and re-entrancy

[N.M. Maclaren]

On Oct 20 2015, William Gropp wrote:

>The standard doesn't explicitly address this.
>
> However, in the spirit of the standard, I would say yes, multiple threads 
> may be used. The behavior of side effects of the function are not 
> defined, nor is it required to apply to elements 0..n-1 in that order in 
> an n-element vector (this is a vector on each process; the is/is-not 
> commutative applies between processes). Rather than try to enumerate and 
> disallow certain actions, I think this is the right solution - the action 
> of the user function on the input parameters is defined; nothing else 
> should be. An implementation might warn users that the function may be 
> called by multiple threads concurrently (e.g., to use atomic memory 
> operations to count invocations or increment "evil").

While I agree in principle, that is disastrous in practice.  None of C,
Fortran nor POSIX are fit for such an implementation.

I can see no sign that C11 is ever going to become established (even C99
took over a decade).  Even if it did, its memory model is seriously and
incomprehensibly ambiguous because (unlike C++11), it has left much of
the old, ambiguous sequence point description in place.  And (again unlike
C11), it doesn't specify anything about any of the library macros and
functions.

Fortran doesn't say anything about threading.  Its concept of purity is
not enough, because there are two separate such concepts and it is the
other that is really needed.  I.e. a function call does not just not
affect anything outside, but nothing outside affects it.  It gets a long
way there, but provides far too many loopholes.

POSIX threading is a well-known disaster area, not least because it
contains no description of how its rules relate to C's, because its list
of thread-safe functions is quite simply broken, and because (God help us
all) it says NOTHING about how to synchronise signalling and state control.
And, yes, I have seen that produce logical impossibilities in the field,
despite using some of its more heavyweight thread synchronisation.

At the very least, it should be possible to ensure that any threading
is transparent using MPI_Init_thread.

Regards,
Nick Maclaren.