[mpiwg-tools] Meeting reminder: Sep 19, 2019 (today)

[Marc-André Hermanns]

Hi Martin,

> Thanks for the notes and sorry I couldn’t make it last week. Looking
> over the notes I have a few questions/comments on QMPI (not
> necessarily for you, but for the group):
> 
> Re: "Implementations define an enum that can be used to query.”
> 
> That certainly works and is probably relatively clean. If we go that
> way, though, we should prescribe the enum values in the standard. 

From what we saw in the prototype, it should be enough to define the
enum *keys*, not the values, as the tool wrappers will always use
those to interact with QMPI (it would replace the explicit numbers
currently used in the wrappers of the prototype tool).

> We should also leave the option for MPI implementations to add more
> than the existing standardized calls (e.g., now MPIX functions) to
> that array, e.g., after a cutoff index.
As we should only rely on the enum keys, I think there is no need to
define any cutoff index, the additional symbol names would be there.
It would be part of the documentation to know which additional keys
are defined.

> Re: "What about symbol name length”
> 
> What speaks against just using the existing MPI function names as enum
> elements? Then we at least don’t introduce any new problems.

My point for the notes was that we already have function names that
are right on the maximum length that _have_ to be supported by C, C++,
and Fortran, so we should keep this in mind.

Yes, I'd advocate for a short prefix and the function name, e.g., in
lieu of a better name "QMPI_INIT".

> Re: "How would QMPI know the name of the wrappers of a linked tool?”
> 
> I had thought about this before as well, but I think this can lead to
> problems - if you end up with two tools using the same prefix (like
> the “tool” in the example - I imagine we put this into the examples
> and there every 2nd tool will be called “tool”, no matter how many
> advice to users we write). It would also require MPI to do a dlsym on
> every node at startup for every tool for every MPI function, which
> probably causes scalability issues. 

You would have your real world tool return "tool" as the symbol prefix?!?

For Score-P I can certainly guarantee that we would have "SCOREP" as
the prefix. For MUST I'd expect something like "MUST" or "must" and
for OpenSpeedShop I'd expect something like "OSS". The list goes on. I
think it would be a matter of tool quality to _not_ use some generic
name like "tool".

After all, we are advocating the same level of user/vendor
responsibility in our MPI_T UID proposal.

Did I misunderstand you here?

> I am not sure we need it though: if MPI passes a void* array with the
> index enum from above during the init function, then the tool can fill
> it out and just write all its own function pointers into the array and
> then hand that back to MPI for the creation of the larger tables.

I am not a fan of handing out some array and directly setting values
in there. It seems like exposing too much implementation detail (where
MPI usually does a good job hiding these things behind an opaque handle).

If we use the approach above, I could imagine something along the
lines of:

int tool_id;
QMPI_Tool_register(&tool_id);
QMPI_Tool_set_prefix(tool_id, "unique_name_for_tool")
QMPI_Tool_intercept(tool_id, QMPI_INIT);
QMPI_Tool_intercept(tool_id, QMPI_FINALIZE);

the MPI implementation would then search for the symbols
"unique_name_for_tool_MPI_Init" and
"unique_name_for_tool_MPI_Finalize" and register the wrappers.


> Re: "Current prototype focuses/relies on dynamic libraries”
> 
> With the above function table method, I think the system should
> already support static libraries - as long as they get initialized
> somehow, they can register themselves and their functions (function
> pointers work equally for static libraries).

The initialization is exactly the critical point. As we discussed in
the call (I am not much of a dynamic library expert), one can call a
special function when initializing a dynamic library, which then can
deal with all the registrations. Such a mechanism is missing for
static libraries.

A possible way to mitigate this, albeit a bit hackish, would be to
define an environment variable (e.g., MPI_TOOL_INIT) where the user
can provide a comma-separated list of initialization symbols to call.

What do you think?

Cheers,
Marc-Andre

>> On 19. Sep 2019, at 10:48, Marc-André Hermanns via mpiwg-tools
>> <mpiwg-tools at lists.mpi-forum.org
>> <mailto:mpiwg-tools at lists.mpi-forum.org>> wrote:
>>
>> Hi everyone,
>>
>> I put up my notes to out meeting on the wiki:
>>
>> https://github.com/mpiwg-tools/tools-issues/wiki/Notes-2019-09-19
>>
>> Feel free to add/change/remove as appropriate.
>>
>> Cheers,
>> Marc-Andre
>>
>> On 19.09.19 08:13, Marc-André Hermanns via mpiwg-tools wrote:
>>> Hi everyone,
>>>
>>> This is a reminder of our regularly scheduled call today September 19
>>> at 8 am Pacific / 11 am Eastern / 5pm CET.
>>>
>>> Webex Info:
>>>  https://github.com/mpiwg-tools/mpi-standard/wiki
>>>
>>> *NOTE*: The WebEx information moved to to a non-public wiki. You have
>>> to be logged into Github *and* have to have access to the private
>>> Tools WG repositories. If you need access contact Kathryn or me.
>>>
>>> Agenda:
>>> -------
>>> - Face-to-face meeting summary
>>> - Virtual Meeting on MPI_T UIDs summary
>>> - QMPI (identifying next steps)
>>>
>>> Cheers,
>>> Marc-Andre
>>>
>>>
>>> _______________________________________________
>>> mpiwg-tools mailing list
>>> mpiwg-tools at lists.mpi-forum.org
>>> <mailto:mpiwg-tools at lists.mpi-forum.org>
>>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-tools
>>>
>>
>> -- 
>> Dr. rer. nat. Marc-André Hermanns
>>
>> IT Center
>> Group: High Performance Computing
>> Division: Computational Science and Engineering
>> RWTH Aachen University
>> Seffenter Weg 23
>> 52074 Aachen
>> Phone: +49 241 80-24381
>> hermanns at itc.rwth-aachen.de <mailto:hermanns at itc.rwth-aachen.de>
>> www.itc.rwth-aachen.de <http://www.itc.rwth-aachen.de>
>>
>> _______________________________________________
>> mpiwg-tools mailing list
>> mpiwg-tools at lists.mpi-forum.org <mailto:mpiwg-tools at lists.mpi-forum.org>
>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-tools
> 

-- 
Dr. rer. nat. Marc-André Hermanns

IT Center
Group: High Performance Computing
Division: Computational Science and Engineering
RWTH Aachen University
Seffenter Weg 23
52074 Aachen
Phone: +49 241 80-24381
hermanns at itc.rwth-aachen.de
www.itc.rwth-aachen.de

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5336 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-tools/attachments/20190923/4be4ea00/attachment.p7s>