<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<small><font size="+2"><small>Hi </small></font><font size="+2"><small>Matthew,
All,<br>
<br>
Indeed, it is a very good idea, it should be possible to chain
calls using such opaque data-structure.<br>
<br>
Just some random ideas here, complementary to what has already
been introduced:<br>
<br>
I consider that we want to make the tool independent from the
underlying MPI to allow "drop-in" tools.<br>
I was thinking during the webex of<i> MPI_Type_get_envelope</i>
and <i>MPI_Type_get_contents</i> which provide such<br>
abstraction for data-types </small></font></small><small><font
size="+2"><small><small><font size="+2"><small>(p.121 of 3.1) </small></font></small>wouldn't
it be possible to do it for all MPI functions, packing on the
stack before chaining ?<br>
<br>
<small><small>/* Internal runtime representation<br>
passed as an opaque handler<br>
should be a common denominator<br>
between function footprints<br>
(not the case here !) */</small></small><br>
<small><small>typedef struct<br>
{<br>
char * function_name;<br>
int ints[16];<br>
void * pointers[8];<br>
MPI_Comm comms[2];<br>
MPI_Datatype types[10];<br>
MPI_Status statuses[10]; </small></small></small></font><small><small><br>
<font size="+2"><small><small><small><font size="+2"><small><small><small>
MPI_Status requests[10];</small></small></small></font>
...<br>
/* Some pointers to handle calls with arrays
(Waitall, Testall, ... no need to copy just reference
here getters will know where to look) */<br>
MPI_Status * statusses_overflow;</small></small></small></font><br>
</small></small><font size="+2"><small><small><small><font
size="+2"><small><small><small> MPI_Status *
requests_overflow;<br>
/* QMPI Call Chain CTX */<br>
int depth_in_the_chain;<br>
int (*handler_chain[10])( void *);<br>
</small></small></small></font>}QMPI_Context;</small><br>
</small><br>
<i>param_t</i> could point to such context which would be
filled prior to the call and<br>
passed through the handlers until reaching the actual MPI
implementation<br>
which would be able to reference packed elements (directly).<br>
</small></font><big><br>
/* The actual call at the bottom of the chain (with
implementation side knowledge) */<br>
</big></small>
<pre>__INTERNAL__MPI_Send(ctx->pointers[0], ctx->ints[0], ctx->types[0], ctx->ints[1], ctx->ints[2], ctx->comms[0])
</pre>
<small><font size="+2"><small>Could be used like this in the tool,
as in the example, all wrappers would have the same footprint
just a "void *"<br>
making bindings very simple:<br>
<br>
<small><small>static void mpisend_wrapper(param_t param)<br>
{<br>
<br>
printf("START mpisend tool wrapper. Param is %s\n",
(param == MPI_PARAM_VAL) ? "correct" : "wrong");<br>
</small></small></small></font><font size="+2"><small><small><small><font
size="+2"><small><small><small>QMPI_Wrapper_chain(param)</small></small></small></font>;</small></small></small></font><br>
</small><font size="+2"><small><small><small><font size="+2"><small><small><small>
printf("DONE mpisend tool wrapper. Param is %s\n",
(param == MPI_PARAM_VAL) ? "correct" : "wrong");<br>
</small></small></small></font>}</small></small><br>
<br>
Stacking would allow a tool to inspect the result of a call for
functions outputting<br>
elements, as in PMPI. Also the context could be used to resolve
the next function to call (QMPI chain, incrementing an integer
and calling).<br>
<br>
With the same infrastructure, it is possible to imagine that the
tool submits "generic" calls, filling a QMPI_Context<br>
(this is why I've put a <i>name</i> field). For example if you
do a profile and want to reduce performance data at the end,<br>
you could do it without linking to MPI, you just need the qmpi.h
header (too much ? sort of "UMPI - Universal MPI"):<br>
</small></font><br>
<font size="+2"><small><small><small>/* Allocate runtime side
storage (you don't know MPI handle sizes !)*/<br>
QMPI_Context_init( &ctx, int sizes, int sizes, ... );</small><br>
<small>/* Function to call */</small></small></small></font><br>
<font size="+2"><small><small><small><font size="+2"><small><small><small>QMPI_Context_set_name(
</small></small></small></font></small></small><small><small>&ctx,
"MPI_Reduce");<br>
/* Sendbuf */</small></small></small></font><br>
<font size="+2"><small><small><small><font size="+2"><small><small><small><font
size="+2"><small><small><small>QMPI_Context_set_pointer(
</small></small></small></font></small></small><small><small>&ctx,
0 /*index*/, sendbuf);</small></small></small></font><br>
/* Recvbuf */</small></small></small></font><br>
<font size="+2"><small><small><small><font size="+2"><small><small><small><font
size="+2"><small><small><small><font size="+2"><small><small><small>QMPI_Context_set_pointer(
</small></small></small></font></small></small><small><small>&ctx,
1 /*index*/, recvbuf);</small></small></small></font></small></small></small></font><br>
/* Count */</small></small></small></font><br>
<font size="+2"><small><small><small><font size="+2"><small><small><small><font
size="+2"><small><small><small><font size="+2"><small><small><small>QMPI_Context_set_int(
</small></small></small></font></small></small><small><small>&ctx,
0 /*index*/, count);</small></small></small></font></small></small></small></font><br>
/* Datatype */<br>
QMPI_Context_set_type( &ctx, 0 /* Index */, QMPI_CHAR );<br>
/* OP */</small></small></small></font><small><small><br>
<font size="+2"><small><small><small><font size="+2"><small><small><small>QMPI_Context_set_op(
&ctx, 0 /* Index */, QMPI_SUM );<br>
</small></small></small></font>/* Root */</small></small></small></font></small></small><br>
<small><small><font size="+2"><small><small><small><font size="+2"><small><small><small><font
size="+2"><small><small><small><font size="+2"><small><small><small><font
size="+2"><small><small><small>QMPI_Context_set_int(
</small></small></small></font></small></small><small><small>&ctx,
1 /*index*/, 0);</small></small></small></font></small></small></small></font></small></small></small></font><br>
/* Comm */</small></small></small></font><br>
</small></small><font size="+2"><small><small><font size="+2"><small><font
size="+2"><small><small><small>QMPI_Context_set_comm(
&ctx, 0 /* Index */, QMPI_COMM_WORLD );</small></small><br>
</small></font></small></font><small>QMPI_Submit(
&ctx );<br>
/* There would be a GIANT switch behind this ! */</small></small><br>
<br>
A small subset of data-types, Ops and Comms could be exposed as
standardized QMPI_* defines (for example a dup of COMM_WORLD per
plugin QMPI_COMM_PLUGIN).<br>
<br>
However, if the opaque handle is used, one would need to be able
to inspect <i>MPI_Status</i>, <i>MPI_Comm</i>,<i> MPI_Co</i><i>unt</i>
and <i>MPI_Datatype</i> in an abstract manner:<br>
<small><small><br>
/* Exposed interface relative to a context as you cannot
expose the size of the Comm */<br>
int QMPI_Comm_size( QMPI_Context * ctx, int comm_index, int
* size )<br>
{<br>
/* In the MPI implem */<br>
MPI_Comm_size( ctx->coms[ comm_index], size );<br>
}<br>
<br>
... where do we stop ?<br>
</small></small></small><br>
<small>No problem for integers, you just need to know what is
where (standardized as for get_contents), idem for void *<br>
you just need something like this on tool side (but only C types
this way):<br>
</small></font><br>
<font size="+2"><small><small><small><font size="+2"><small><small><small>int
QMPI_Context_get_envelope( QMPI_Context * ctx , int
* int_count, int * addr_count, int * datatype_count,
int * comm_count... ); <br>
</small></small></small></font>int QMPI_Context_get_contents(
QMPI_Context * ctx , int * ints, void * addr, ... );<br>
</small></small><br>
If you are interested, I'll try do draft a first implementation
later in the week to see the performance overhead.<br>
<br>
Thank you very much for the interesting discussions examples and
ideas,<br>
<br>
Jean-Baptiste.<br>
</small></font><br>
<div class="moz-cite-prefix">Le 22/10/2015 19:34, Matthew LeGendre a
écrit :<br>
</div>
<blockquote
cite="mid:alpine.LRH.2.00.1510220959250.9851@auk60.llnl.gov"
type="cite">
<br>
At the end of the call, Jeff asked about a simple example that
illustrates building a tool with QMPI that works with multiple MPI
implementations. I've attached a tarball that illustrates what I
was picturing.
<br>
<br>
This example builds a pair of libraries, libfakeopenmpi.so and
libfakempich.so, which each export "mpisend" functions with
different signatures as well as a pair of fake apps, app_openmpi
and app_mpich, that use each library. It also builds a single
libtool.so, which contains wrapper functions for each mpisend and
dynamically chooses which one is used.
<br>
<br>
Note that the tool library still has files built against each MPI
implementation, but produces a single library that works with
either implementation.
<br>
<br>
On the phone I speculated that you'd need the callback
registration functions to be ABI compatible across all MPI
implementations, but this didn't turn up being necessary. You can
get away with only the MPI identification function having a
standard ABI. In this example I called the function
get_mpi_implementation().
<br>
<br>
It also turned up being very convenient if the MPI header exported
the return value of get_mpi_implementation() though a #define
(which I called MPI_IMPL). This allowed for an easy runtime test
of whether the a tool implementation was compiled for the running
MPI.
<br>
<br>
Also, On the phone I speculated that getting per-compiler tools
would be difficult. You actually could solve part of the problem
by including a compiler name and version in the MPI_IMPL
#define--most compilers have preprocessor macros that exports that
info. Though this still wouldn't solve any tool incompatibilities
with runtime libraries like libstdc++.so.
<br>
<br>
-Matt
<br>
<br>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
mpiwg-tools mailing list
<a class="moz-txt-link-abbreviated" href="mailto:mpiwg-tools@lists.mpi-forum.org">mpiwg-tools@lists.mpi-forum.org</a>
<a class="moz-txt-link-freetext" href="http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools">http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools</a></pre>
</blockquote>
<br>
</body>
</html>