[Mpi-forum] user-defined reductions and re-entrancy

[Jeff Hammond]

Thanks.  I think this is a reasonable conclusion.  One might expect an
implementation that does this to expose control of internal concurrency by
environment variables (or that fancy MPI_T business about which I know
almost nothing) so as to ensure an "evil" user only blows their leg off
once.

Best,

Jeff

On Tue, Oct 20, 2015 at 11:19 AM, William Gropp <wgropp at illinois.edu> wrote:

> The standard doesn’t explicitly address this.
>
> However, in the spirit of the standard, I would say yes, multiple threads
> may be used.  The behavior of side effects of the function are not defined,
> nor is it required to apply to elements 0..n-1 in that order in an
> n-element vector (this is a vector on each process; the is/is-not
> commutative applies between processes).  Rather than try to enumerate and
> disallow certain actions, I think this is the right solution - the action
> of the user function on the input parameters is defined; nothing else
> should be.  An implementation might warn users that the function may be
> called by multiple threads concurrently (e.g., to use atomic memory
> operations to count invocations or increment “evil”).
>
> Bill
>
> William Gropp
> Director, Parallel Computing Institute
> Thomas M. Siebel Chair in Computer Science
> Chief Scientist, NCSA
> University of Illinois Urbana-Champaign
>
>
>
>
>
> On Oct 20, 2015, at 1:04 PM, Jeff Hammond <jeff.science at gmail.com> wrote:
>
> > Thanks Jed for clarifying. I agree with your modified example.
> >
> > The fundamental question here remains: can an implementation internally
> call user-defined reduction operators from multiple threads at the same
> time?
> >
> > Jeff
> >
> > Sent from my iPhone
> >
> >> On Oct 20, 2015, at 10:50 AM, Jed Brown <jedbrown at mcs.anl.gov> wrote:
> >>
> >> "Balaji, Pavan" <balaji at anl.gov> writes:
> >>
> >>> You can't call MPI functions in there, but you can replace the abort
> with an exit(1).
> >>
> >> MPI-3.0 p184: "MPI_ABORT may be called inside the function in case of
> an error."
> >>
> >>> Each user function is called once per process, so "evil" will always
> be 1, isn't it?  I'm not sure why reentrance is an issue here.
> >>
> >> Jeff said "user code increments evil once between every call to this
> >> function so that it should never fail in single-threaded execution".
> >>
> >> A self-contained rendition:
> >>
> >> void User_reduce(void* invec, void* inoutvec, int *len, MPI_Datatype
> *datatype)
> >> {
> >> static int evil = 0;
> >> double * in = (double*)invec;
> >> double * out = (double*)inoutvec;
> >> if (*datatype == MPI_DOUBLE && (++evil%2) == 1) {
> >>   for (int i=0; i<*len; i++) out[i] = evil*in[i];
> >>   evil++;
> >> } else {
> >>   MPI_Abort(1,MPI_COMM_WORLD);
> >> }
> >> }
> >>
> >> The results certainly depend on how an implementation segments a buffer
> >> and on which process the reduction is called.  Additionally, calling the
> >> reduction from multiple threads may cause a non-deterministic abort.
> >> _______________________________________________
> >> mpi-forum mailing list
> >> mpi-forum at lists.mpi-forum.org
> >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
> > _______________________________________________
> > mpi-forum mailing list
> > mpi-forum at lists.mpi-forum.org
> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>
> _______________________________________________
> mpi-forum mailing list
> mpi-forum at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>



-- 
Jeff Hammond
jeff.science at gmail.com
http://jeffhammond.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpi-forum/attachments/20151020/4318fbd0/attachment-0001.html>