[mpiwg-tools] Problem with API From 2020-10-15 Conversation

[Joachim Protze]

Hi Wesley, Martin,

I still don't see the need for splitting the function, or what this
would give us.
I attached the slides from yesterday for illustration. Slide 3 also
sketches, how I would emulate the PnMPI_Xy_NewStack function.

Whoever implements the function (Supertool?) will need to know the
ordering of the tools and apply a mapping to the stacks. That's why I
propose a function like:

QMPI_Get_tool_ids (int count, int * array_of_tool_ids);

to query the tool IDs according to their ordering in the QMPI_TOOLS env
variable. The supertool can then build a map for the lookup of stacks:
{"stack1": 12, "stack2": 13}
A position map:
pos = {0:1, 45:2, 12:3, 3:4, 5:5, ...}
As well as a map for the end of the stacks:
stacklimit = {12: 6, 13: 9}


I also propose to have a predefined tool-ID (QMPI_MPI_ID) to allow for
querying the actual MPI function.


Best
Joachim



Am 16.10.20 um 16:17 schrieb Wesley Bland via mpiwg-tools:
> Hi all,
> 
> In yesterday’s call, we discussed separating the current QMPI_Get_function call (that takes the current tool_id and an enum value to get the function in the call stack) into two function calls. The first would determine the next tool in the stack and the second would get the desired function for that tool. So they would look something like this:
> 
> QMPI_Get_next_tool(int tool_id, int num_next_tools, int *next_tool_ids);
> QMPI_Get_tool_fn(int tool_id, enum QMPI_Functions_enum function_enum, void (*function_ptr)(void), QMPI_Context *context);
> 
> However, as I went to implement that today, I realized that the reason this is a single function today is that not all tools implement all functions. So if you have a tool stack that looks like this:
> 
> A - B - C
> 
> and tools A and C intercept MPI_SEND and tool B does not, when tool A asks for the next tool to implement MPI_SEND, it should get back the function for tool C. Unfortunately, if we don’t ask for a specific function, MPI doesn’t have a way to figure out whether it should return B or C.
> 
> So there’s two options here.
> 
> 1) Keep the two functions combined. This makes it clear in the current, linear case which tool comes next. However, it doesn’t move us toward the potential future of having a mechanism (internal or external) to make dynamic decisions about the next tool.
> 
> 2) Add an enum value to the QMPI_Get_next_tool function so it is only searching for tools that implement the particular function. This does complicate the search functionality a little bit more, but not significantly. The main drawback I see here is just the confusion of passing that enum value into both functions. If we’re not providing this decision functionality in QMPI v1, it’s weird that we have this placeholder for it now. However, that’s already true with the array of next tools that will always be 1, so maybe it’s not a big deal.
> 
> I would imagine that 2 is the best option here to support the PnMPI use case, but I’m happy to hear feedback.
> 
> Thanks,
> Wes
> 
> 
> _______________________________________________
> mpiwg-tools mailing list
> mpiwg-tools at lists.mpi-forum.org
> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-tools
> 


-- 
Dipl.-Inf. Joachim Protze

IT Center
Group: High Performance Computing
Division: Computational Science and Engineering
RWTH Aachen University
Seffenter Weg 23
D 52074  Aachen (Germany)
Tel: +49 241 80- 24765
Fax: +49 241 80-624765
protze at itc.rwth-aachen.de
www.itc.rwth-aachen.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: QMPI-stacks.pdf
Type: application/pdf
Size: 144684 bytes
Desc: not available
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-tools/attachments/20201016/63d72573/attachment-0001.pdf>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5327 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-tools/attachments/20201016/63d72573/attachment-0001.p7s>