[Mpi-forum] ABI

[Jed Brown]

"Jeff Squyres (jsquyres)" <jsquyres at cisco.com> writes:

> Integer vs. pointer is but one of MANY issues that prevent MPI
> implementations from being ABI compatible.  Let's not also forget:

Yes, there are a lot of issues, but I don't think it's as insurmountable
as it may appear.

> - Values of constants.  Simple integer values are usually easy to
> device/resolve (but don't forget that some integer values are
> specifically chosen on specific platforms). Sentinel values like
> MPI_STATUS_IGNORE are not.

Section 2.5.4 (Named Constants) has a very short list of items that are
actually required to be compile-time constants.  Current implementations
put the values of far more constants into the ABI than the standard
requires.

> - Size/content of MPI_Status.  MPI implementations hide (different)
> non-public fields in MPI_Status.

These are not so wildly different.  Note that the non-public fields can
be used for different things, they just have to add up to the same total
size.

typedef struct MPI_Status {
    int MPI_SOURCE;
    int MPI_TAG;
    int MPI_ERROR;
    MPI_Count count;
    int cancelled;
    int abi_slush_fund[2];    
} MPI_Status;

struct ompi_status_public_t {
    /* These fields are publicly defined in the MPI specification.
       User applications may freely read from these fields. */
    int MPI_SOURCE;
    int MPI_TAG;
    int MPI_ERROR;
    /* The following two fields are internal to the Open MPI
       implementation and should not be accessed by MPI applications.
       They are subject to change at any time.  These are not the
       droids you're looking for. */
    int _cancelled;
    size_t _ucount;
};


> - Launcher differences.  The mpirun/mpiexec (or whatever launcher) is
> inherently different, with different CLI options and configuration,
> between different MPI implementations.

Doesn't matter because the launcher sets the environment so that the
correct library is used.  You would use the MPICH mpiexec to run with
the MPICH libraries and the OMPI mpiexec to run with OMPI libraries.
You only ever interact with your library.

> - Library names.  libmpi.so?  libompi.so?  libmpich.so?  libmpi.a?  And so on.

libmpi.so, obviously.  ;-)

This should only expose the standard symbols.  /mpich/lib/libmpi.so and
/ompi/lib/libmpi.so can link to whatever private libraries they want;
those are not part of the ABI.

> - Dependent libraries.  What else do you need to link when linking the
> application?  You can (usually) hide this when the MPI is a shared
> library, but a) not always, and b) that doesn't help when MPI is a
> static library.

A standard ABI is not useful for static libraries because you have to
rebuild (it could save you recompiling all the sources, but you're back
in the build system instead of the runtime).

Proper shared library etiquette is to always hide dependent libraries.

> - Compiler ABIs.  MPI middleware cannot solve the fact that C++ and
> Fortran compilers cannot (and will not) agree on an ABI (for many good
> reasons, BTW).

We can't standardize between incompatible compilers, so assume
compatible compilers.

> - Compiler options.  Was the MPI library compiled with -O3?  -i8?  -32 or -64?  ...?

Same problem we have today.  If you compile with the same flags, we'd
like the ABI to match.

> The fact of the matter is that the MPI API was *specifically designed
> with only source compatibility in mind*.  We now have nearly 20 years
> of momentum in different MPI implementations with different
> engineering design choices.
>
> The term "ABI" is a catchall for many, many different issues.  People
> seem to think that simply switching libraries at run time is a silver
> bullet -- "if only I could just change my LD_LIBRARY_PATH and use a
> different MPI implementation, then the world would be better".  But
> let's also not forget that most users barely know how to use
> LD_LIBRARY_PATH (if at all).

/mpich/bin/mpiexec -n 2 ./myexecutable

/ompi/bin/mpiexec -n 2 ./myexecutable

The launcher sets LD_LIBRARY_PATH.  That is feasible (I think) and is
the change that would save distributions, packagers, and users countless
hours.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: <http://lists.mpi-forum.org/pipermail/mpi-forum/attachments/20131204/052bb23c/attachment-0001.pgp>