[MPI3 Fortran] Fortran extra_state argument to MPI attribute functions

[Jeff Squyres]

On May 31, 2009, at 4:41 PM, Bill Long wrote:

> I read though some sections of the MPI document to try to figure out
> what an "attribute" was - evidently some new component you add to an
> object.  I still don't know what an "attribute key" is - the name
> suggests something to do with encryption. I'll assume that's not
> important.
>

I agree that the language about attributes in MPI is a bit  
confusing.  :-(

Attributes are basically arbitrary values that you can cache on MPI  
objects.  You create an integer "keyval" (aka "attribute key") that is  
then universal across all objects of a given type.  Then you use that  
keyval to store/retrieve values on specific objects.  Something like  
this:

1. create communicator keyval ID 17
2. store value X on communicator A with keyval 17
3. store value Y on communicator B with keyval 17
4. retrieve value X from communicator A with keyval 17
5. retrieve value Y from communicator B with keyval 17

Hence, the keyval 17 is available to be used to set/get distinct  
attributes on all communicators.

There's a few more details, but that's the general gist of it.

There are attribute callbacks invoked when communicators are copied  
and deleted.  The rationale here is that attribute values may actually  
require deep copying or freeing.  For example:

6. Copy communicator B to [new] communicator C
7. Since keyval 17 on B has a value, invoke attribute 17's copy  
callback with (comm, keyval, value, extra_state) tuple on the new  
communicator C
8. Delete communicator A
9. Since keyval 17 on A has a value, invoke attribute 17's delete  
callback with (comm, keyval, value, extra_state) tuple on communicator A

> The document also says that "callback" routines may be
> associated with MPI objects (a communicator in this case).  I'm  
> guessing
> the extra_state is some identifier that is associated with an  
> attribute
> that is created somewhere else.
>

Yes.  When you create a keyval (e.g., step 1, above), you assign a  
single extra_state value. This extra_state value will be passed to all  
copy and delete callbacks on all objects.  So steps 7 and 9 will get  
the same extra_state value, even though 7 is copy to new communicator  
C, and 9 is a delete of communicator A.

> Assuming my impressions above are close to correct, then I think  
> this is
> yet another example of an underlying defect in the MPI Fortran
> interfaces - the basic denial that derived types exist in Fortran.  
> From
> Fortran's point of view, all of the routines along the lines of
> MPI_Comm_create_keyval should be removed.   Instead, a derived type  
> for
> a communicator, MPI_comm_t, should be defined in the MPI module.  If
> users want to add information to the communicator, they can define a  
> new
> type that extends MPI_comm_t by adding the desired components, and an
> additional interface (and routine, if needed) for the generic binding
> copy_fn that indicates how to copy these components, and add an
> interface to the final binding that performs the delete_fn operation.
> Specify the communicator dummy arguments as class(MPI_comm_t).
>

This is neither the intent of the MPI Forum nor a defect in the design.

Attributes are specifically intended to be mixed-n-matched in a  
ploymorphic way -- to allow arbitrary layering of potentially non- 
linearly-derived sets of unrelated data from unrelated portions of  
software.  For example, MPI may be *the* lowest network layer that an  
application knows about, but 5 different pieces of middleware are  
linked into the same process (fairly common in complex MPI  
applications).  All 5 middleware packages are "sharing" MPI, and may  
be peers or layers of each other.  In this case, it wouldn't  
necessarily make sense to derive new types around a communicator just  
so you can cache your data on it -- especially for peer middleware  
packages, who should derive from whom?  What type does the upper layer  
application use?  Please, let's not go into a situation that implies  
or necessitates polymophic inheritance.  :-)  It is cleaner (IMHO) to  
have a registration-based interface (like the MPI attribute system)  
and allow anyone to hang their data off MPI objects in a neutral manner.

That being said, I'm all in favor of someone writing a higher-level  
MPI class library on top of advanced Fortran features.  Such features  
would likely diverge from the 1:1 mapping of MPI language-neutral  
specifications to language-specific bindings, and that's fine (e.g.,  
Boost.MPI did this for C++ -- and it's great.  It's just not suitable  
for standardization in the official Forum standards documents, which  
is fine).  Fortran programmers would enjoy the "natural" Fortran  
abstractions and features that they're used to (e.g., perhaps  
attributes could be implemented with inheritance, as described above),  
and the MPI spec stays "simple" without getting embroiled even further  
than it is in specific language debates (hah!).

-- 
Jeff Squyres
Cisco Systems