[mpiwg-tools] PMPI for a complete MPI wrapper

Wed Oct 4 08:50:57 CDT 2017

Hi Max,

> thanks for the fast answer. With the pmpi.h I mean a file like mpi.h
> but only containing the PMPI_ Interface. As you suggested I might try
> to create a full wrapper myself. I took a look on the wi4mpi project
> and there approach seems to create there own interface aka. mpi.h and
> then wrap this to the intel MPI or OpenMPI implementation. Due to this
> approach, they know the data types and can generate the interface.

For the wi4mpi project, Jean-Baptiste and people from CEA may be the
right people to talk to.

The wi4mpi is a bit of a special project, as it is provides a software
'glue' to make MPI implementations interchangeable. It is a way to
overcome the missing ABI (i.e., a specification of types, etc.).

Usually, the users will have to recompile their application every time
they choose a different MPI (potentially also when using a different
version of the same MPI), as values and types in the mpi.h may have
changed. For large simulation codes, this can take a long time. When
you have a translating 'glue' like wi4mpi in between, you can swap MPI
implementations via LD_PRELOAD at the start time of the application.

I don't know enough about wi4mpi to really know what their goal is:
Have a mixed MPI run (e.g., couple two codes compiled against differnt
MPIs)? Use a library compiled for one MPI together with an application
compiled against another? Just make it easier for users to link
against the right MPI? All of the above?

@Marc? Any comments on what the design goal of wi4mpi is? Does it
support other MPI implementations beside Intel-MPI and Open-MPI?

(If this discussion drifts more towards 'wi4mpi' specifics, we should
take this off the list and keep everyone else in CC)

> In my library I wanted to use a light wight wrapper. That is I wanted
> to use the original data types. With this approach I currently have
> structures like:
> 
> struct AMPI_Comm {
> // my own data;
> MPI_Comm comm; // the original object
> };
> 
> I can then simply call the pmpi functions with the stored original
> object.
> If I have a wrapper such that there is a PMPI_Comm object available, I
> could do the following:
> struct MPI_Comm {
> // my own data;
> PMPI_Comm comm; // the original object
> };
> 
> If the wrapper should use the same types from a general mpi.h, then I
> do not know the types and would need to declare something like:
> 
> hidden_mpi.c
> #include <mpi.h>
> decltype(MPI_COMM_WORLD) PMPI_COMM_WORLD = MPI_COMM_WORLD;
> 
> and then I need to use PMPI_COMM_WORLD in my library and I can
> generate a hmpi.h wich contains lines like:
> #define MPI_COMM_WORLD PMPI_COMM_WORLD
> 
> Which could be included by the user. But in order use PMPI_* in my
> library, I need to specify the symbol in a header file for which I
> need the type. In order to get the type I need to include mpi.h which
> will define MPI_COMM_WORLD and I have a name clash.

mpi.h only _declares_ the prototype. It does not define anything
(apart from CPP macros, etc.).

If you provide your own types, then you will need to declare your own
prototypes, which I would generate (see below).

> So unfortunately I see no way in providing a wrapper without writing a
> complete MPI Interface, which I would like to avoid. I might be able
> to use the wi4mpi Project and use there interface as a base for my
> implementation, which would add a dependency to my project.

For just a 'classic' wrapper, you just need to provide the definition
(implementation) of the function you want to replace, adhering to the
declared (in mpi.h) function prototype.

> A third and very ugly option would be, that I define all my types as
> void* in the interface for the user. But this disables type checking
> and I still would need to wrap from void* to references of my types.
> 
> So I might just stay in my AMPI namespace and provide a macro for the
> user to either call regular mpi functions or my wrapper functions.

If you need 'classic' wrappers for your project, you might consider
generating that code with a generator like 'wrap' [2].

As I mentioned in my other mail, writing a 'classic' wrapper is
straight forward.

Cheers,
Marc-Andre

[2] https://github.com/LLNL/wrap

> On Wed, 2017-10-04 at 11:00 +0200, Jean-Baptiste BESNARD wrote:
>> Dear Max,
>>
>> I’m not sure I completely understand what you mean by a « pmpi.h »
>> however I may have some initial elements below.
>>
>> The PMPI interface is currently targeting MPI functions only and
>> indeed some of the values you’ll find in your executable will be
>> compile time constants.
>> In fact, most MPI types/Constants are implementation dependent,
>> there is no unified ABI.
>>
>> Nonetheless, you might be able to interpret them in your wrapper
>> library in order to have them « rerouted » to your target
>> implementation.
>> I mean, knowing the value of MPI_COMM_WORLD you could rewrite it to
>> be MPI_COMM_WORLD2. 
>> And for sure you wont’t find a PMPI_COMM_WORLD.
>>
>> I can help on writing a wrapper for the whole PMPI interface. See my
>> repo here: https://github.com/besnardjb/mpi-snippets
>> There is a simple python script generating VIM snippets for MPI from
>> JSON specs, it can easily be converted to a script generating the
>> whole MPI interface.
>>
>> Eventually, an approach close to what you want to do might
>> be https://github.com/cea-hpc/wi4mpi which operates this systematic
>> handler conversion between MPI flavors, but this clearly involves
>> some rewriting.
>>
>> Hope this helps.
>>
>> Regards,
>>
>> Jean-Baptiste.
>>
>>> Le 4 oct. 2017 à 10:35, Max Sagebaum
>>> <max.sagebaum at scicomp.uni-kl.de
>>> <mailto:max.sagebaum at scicomp.uni-kl.de>> a écrit :
>>> Hello @ all,
>>>
>>> my question is concerning the PMPI specification. I hope the list
>>> is the correct place to ask.
>>>
>>> I want to write a complete wrapper for MPI. That is every define,
>>> typedef and function will be wrapped and might be completely
>>> changed. Currently I prefixed everything with AMPI_ such that no
>>> name clashes exist. But the user would need to rename every
>>> occurrence of MPI_ with AMPI_
>>>
>>> I would now like to use the PMPI definition of MPI to define my
>>> wrappers as the MPI version which then use the PMPI definitions.
>>> Unfortunately I could not find tutorials for a complete wrapper.
>>>
>>> As an example take MPI_COMM_WORLD. I made a grep on the openmpi
>>> installation on my linux machine for PMPI_COMM_WORLD but the result
>>> was empty. The definition of MPI_COMM_WORLD was 
>>> #define MPI_COMM_WORLD OMPI_PREDEFINED_GLOBAL( MPI_Comm,
>>> ompi_mpi_comm_world)
>>> without any chance to switch to PMPI_COMM_WORLD as a predefined macro.
>>>
>>> I also checked the newest source tarball of openmpi and I could not
>>> find anything for PMPI_COMM_WORLD there.
>>>
>>> In the mpi 3.0 standard on page 555 in section 14.2.1 the
>>> requirements are just listed for functions. Was the definition of
>>> the PMPI_ supplements for defines, types etc. never discussed?
>>>
>>> I would have expected, that I can just include a pmpi.h and then I
>>> would have all the PMPI_ symbols without the MPI symbols available.
>>>
>>> Do you know of any way I could make my idea work?
>>>
>>> Cheers
>>>
>>> Max
>>>
>>> -- 
>>> Max Sagebaum
>>>
>>> Chair for Scientific Computing,
>>> TU Kaiserslautern,
>>> Bldg/Geb 34, Paul-Ehrlich-Strasse,
>>> 67663 Kaiserslautern, Germany
>>>
>>> Phone: +49 (0)631 205 5638
>>> Fax:   +49 (0)631 205 3056
>>> Email: max.sagebaum at scicomp.uni-kl.de <mailto:max.sagebaum at scicomp.uni-kl.de>
>>> URL:   www.scicomp.uni-kl.de <http://www.scicomp.uni-kl.de>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> mpiwg-tools mailing list
>>> mpiwg-tools at lists.mpi-forum.org
>>> <mailto:mpiwg-tools at lists.mpi-forum.org>
>>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-tools
>>
>> _______________________________________________
>> mpiwg-tools mailing list
>> mpiwg-tools at lists.mpi-forum.org <mailto:mpiwg-tools at lists.mpi-forum.org>
>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-tools
> 
> -- 
> 
> Max Sagebaum
> 
> Chair for Scientific Computing,
> TU Kaiserslautern,
> Bldg/Geb 34, Paul-Ehrlich-Strasse,
> 67663 Kaiserslautern, Germany
> 
> Phone: +49 (0)631 205 5638
> Fax:   +49 (0)631 205 3056
> Email: max.sagebaum at scicomp.uni-kl.de
> URL:   www.scicomp.uni-kl.de
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> mpiwg-tools mailing list
> mpiwg-tools at lists.mpi-forum.org
> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-tools
> 

-- 
Marc-Andre Hermanns
Jülich Aachen Research Alliance,
High Performance Computing (JARA-HPC)
Jülich Supercomputing Centre (JSC)

Wilhelm-Johnen-Str.
52425 Jülich
Germany

Phone: +49 2461 61 2509 | +49 241 80 24381
Fax: +49 2461 80 6 99753
www.jara.org/jara-hpc
email: hermanns at jara.rwth-aachen.de

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4899 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-tools/attachments/20171004/1bc652a8/attachment-0001.p7s>