[mpiwg-tools] PMPI for a complete MPI wrapper

[Marc-Andre Hermanns]

Dear Max,

> my question is concerning the PMPI specification. I hope the list is
> the correct place to ask.

In this specific case regarding the general API, it is. ;-) If you
want internal information about implementation detail, the respective
mailing lists of that specific implementation are the right point of
contact.

> I want to write a complete wrapper for MPI. That is every define,
> typedef and function will be wrapped and might be completely changed.

The question here is, _why_ do you want to write a complete wrapper
including all constants, types, etc.?

As Jean-Baptiste already mentioned, the MPI standard only defines PMPI
symbols for its functions only. However, with (almost) every function
being wrapped, you are in full control of the interaction with the MPI
implementation.

For example: In MPI_Init, you could create a new communicator
'my_new_world' and keep a reference to it internally. Then when
wrapping any function with a communicator argument, you can silently
swap MPI_COMM_WORLD against this new communicator.

int MPI_Barrier(MPI_Comm comm) {

  if (comm == MPI_COMM_WORLD) {
    PMPI_Barrier(new_world_comm);
  }

}

Agreed, this is a runtime change, as opposed to a compile-time change
that would apply if you changed a constant directly and this may imply
more overhead.

> Currently I prefixed everything with AMPI_ such that no name
> clashes exist. But the user would need to rename every occurrence
> of MPI_ with AMPI_
Requiring users to change their code is usually critical, as beyond
simple benchmarks, it can become cumbersome and error prone quite quickly.

Furthermore, the definition of the PMPI interface is actually designed
such that a user does not have to modify any code.

> I would now like to use the PMPI definition of MPI to define my
> wrappers as the MPI version which then use the PMPI definitions.
> Unfortunately I could not find tutorials for a complete wrapper.

I also do not have a tutorial at hand, but you should be able to learn
by example from any tool package that uses PMPI wrappers (e.g.,
Score-P [1]; check code in src/adapters/mpi/SCOREP_Mpi_*.c)

The recipe is pretty simple:

1) #include <mpi.h>
2) define (not declare) the function you want to wrap (see barrier above)
3) compile
4) link to MPI application

> As an example take MPI_COMM_WORLD. I made a grep on the openmpi
> installation on my linux machine for PMPI_COMM_WORLD but the result
> was empty. The definition of MPI_COMM_WORLD was 
> #define MPI_COMM_WORLD OMPI_PREDEFINED_GLOBAL( MPI_Comm,
> ompi_mpi_comm_world)
> without any chance to switch to PMPI_COMM_WORLD as a predefined macro.

An MPI implementation is free to define any constants and handles as
they see fit. Some implementations use pointers to structs, others use
integers to identify handles. IMHO, trying to redefine handles or
types portably across different implementations is a moot project.

> I also checked the newest source tarball of openmpi and I could not
> find anything for PMPI_COMM_WORLD there.

As said before, the PMPI interface _only_ covers functions, and even
allows some exceptions. For example, MPI_Wtime is not required to be a
function and poses a PMPI symbol, but instead be defined completely as
a macro. As a consequence, one might not be able to intercept
MPI_Wtime in this way.

> In the mpi 3.0 standard on page 555 in section 14.2.1 the requirements
> are just listed for functions. Was the definition of the PMPI_
> supplements for defines, types etc. never discussed?

The question is again: What is it that you are trying to accomplish
that is not possible with PMPI today?

The PMPI interface was (afaik) introduced to enable easy profiling of
applications, i.e., creating wrappers to easily insert measurement
code before and after the actual MPI function invocation.

For constants, I do not see any similar application. Any handles are
opaque objects in that you do not know how they may be defined.
Redefining them and handing them to the implementation seems
problematic to me.

> I would have expected, that I can just include a pmpi.h and then I
> would have all the PMPI_ symbols without the MPI symbols available.
> Do you know of any way I could make my idea work?

IMHO, the 'what' and 'how' you proposed is not possible. But maybe, if
you elaborate more on the 'why', there is a way to leverage the PMPI
interface for your needs.

Cheers,
Marc-Andre


[1] http://www.vi-hps.org/upload/packages/scorep/scorep-3.1.tar.gz

-- 
Marc-Andre Hermanns
Jülich Aachen Research Alliance,
High Performance Computing (JARA-HPC)
Jülich Supercomputing Centre (JSC)

Wilhelm-Johnen-Str.
52425 Jülich
Germany

Phone: +49 2461 61 2509 | +49 241 80 24381
Fax: +49 2461 80 6 99753
www.jara.org/jara-hpc
email: hermanns at jara.rwth-aachen.de

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4899 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-tools/attachments/20171004/44b78ff3/attachment.p7s>