[mpiwg-tools] Per the QMPI/PMPI+1/whatever discussion yesterday

Martin Schulz schulzm at llnl.gov
Tue Dec 9 13:10:26 CST 2014

On 12/9/14, 8:38 AM, "Bland, Wesley B." <wbland at anl.gov> wrote:

>> On Dec 9, 2014, at 5:31 AM, Jeff Squyres (jsquyres)
>><jsquyres at cisco.com> wrote:
>> This morning, I have a bunch of follow-on thoughts to the
>>QPMI/PMPI+1/whatever tools WG face-to-face discussion yesterday.  We
>>were all getting tired at the end, and I don't think we documented
>>everything well.  Here are my thoughts this morning, in no particular
>> 1. New names: QMPI_Tool_add / QMPI_Tool_get.  In general, using the
>>"QMPI_" prefix should be sufficient for distinguishing any new functions
>>we create here from both the PMPI_ world and from the MPI_T_ world.
>>E.g., even though the 2 names I propose here include the word "tool", I
>>think the QMPI_ prefix is sufficient to distinguish them from the MPI_T_
>>tool genre.
>Q* seems fine to me. I don¹t really care a lot about that end of the
>name, but I think perhaps the QMPI_Tool_get name isn¹t right anymore
>since we¹ve changed the definition of the function from the beginning of
>the discussion. Now that it¹s aimed at providing the tool(s) that is(are)
>initialized after MPI_Init, perhaps it should be something like

Is this intended as temporary internal name or for the standard at the
end. For the former, it¹s fine, but I don¹t think it¹s a good idea to have
a special name space in the MPI standard. The forum already hated the idea
of MPIT and we had to change it to MPI_T. Also, most of these functions
are also intended for users, so why shouldn¹t they be normal MPI functions?

>> 2. Things we didn't document yesterday:
>> - Need to decide a free-memory model for QMPI_Tool_get (i.e., how is
>>the argv that is returned via QMPI_Tool_get returned).
>> - Need to specify that the strings passed in to QMPI_Tool_add are safe
>>to be freed after QMPI_Tool_add returns (i.e., imply that the MPI
>>implementation will make a copy of the string if it needs it)
>> - Need Fortran bindings for the QMPI_* functions.
>> - In the union of fields passed to the tool-provided QMPI_multiplex
>>function for each, some functions will need to pass an enum to indicate
>>which MPI binding the intercept came from (e.g, C, mpif.h, use mpi, use

We should also talk more about the union type and should try to spell it
out for a few functions.

Another concern: we need to find a way to properly represent (and
distinguish) Fortran buffer descriptors.

>> - In all cases, the tool-provided QMPI_multiplex function will be
>>passed the C versions of MPI handles.  I.e., if a Fortran program calls
>>MPI_Send which ends up invoking a tool's QMPI_multiplex function, the
>>QMPI_multiplex function will receive C versions of the communicator and
>>datatype handles.
>> 3. Random thought -- do we need QMPI_Control?  This would be analogous
>>to MPI_Control -- it would be used for communicating directly with
>>tools.  Its first argument could be a (const char*) that corresponds to
>>the strings that you get back from QMPI_Tool_get (i.e., allowing an app
>>to communicate with a specific tool).
>What is the benefit of having this in lieu of just having tool specific
>functions? If you (the application) know enough about the tool to know
>what to pass to QMPI_Control, why not just call a function directly from
>the tool¹s library?

We definitely need a QMPI_Qcontrol - two reasons: for one, some of the
information provided should be tool independent (where to start or stop
profiling or how to mark phases is application characteristics and not
tool specific); second, we need to be able to leave these annotations in
the code and be able to link the application without the tool. Only f the
tool is attached, the Qcontrol gets called. This could be as simple as
including the function in the list of wrappable functions.

>> 4. We haven't crisply laid out the goals for QMPI/PMPI+1/whatever.
>>That is -- to paraphrase Kathryn's sentiments from yesterday: how is
>>this interface *better* than the current PMPI world?  I think the goals
>> - Avoid relying on linker tricks (although this is kinda weak -- linker
>>tricks work just fine in many cases, and, indeed, we're not outlawing
>>linker tricks like LD_PRELOAD)
>> - Solve all language bindings issues (e.g., allow tools to be written
>>in C)
>> - *Eventually* allow supporting N tools simultaneously
>>  --> Although this is just the first step in supporting that eventual

We conquered (I think?) the biggest problem we had with multiple tools,
the specification of which tools to load (this was the main reason why I
suggested to limit the proposal). I think the dependencies are easy, which
leaves ordering. Perhaps we should have at least some support for this
already in the proposal.

>> And again paraphrasing Kathryn (who was channeling Adam): given that
>>the plan is that MPI-4 won't support N simultaneous tools, are these
>>goals enough "better" than the current PMPI to be worth doing?
>> I think we need a crisp, irrefutable answer for that question before
>>going before the entire Forum.
>This seems like a good start. We should also try not to disallow multiple
>tools as things are now. We can just stay that it¹s undefined behavior,
>and the implementation is free to support it if desired with random
>ordering (or something).

I agree - there is no reason not to support an ordered list of tools at
this point already.


>> -- 
>> Jeff Squyres
>> jsquyres at cisco.com
>> For corporate legal information go to:
>> _______________________________________________
>> mpiwg-tools mailing list
>> mpiwg-tools at lists.mpi-forum.org
>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools
>mpiwg-tools mailing list
>mpiwg-tools at lists.mpi-forum.org

More information about the mpiwg-tools mailing list