[Mpi-forum] MPI-3: MPI_T_ERR_... and MPI_ERR_LASTCODE
Adam T. Moody
moody20 at llnl.gov
Mon Jul 23 18:45:20 CDT 2012
This reminds me of the discussions we had on datatypes, regarding which
datatypes are available and can be used before MPI_Init is called.
Since people weren't keen on the fact of duplicating all of the
datatypes and associated routines for MPI_T, in that case, we boiled
down the list of datatypes allowed in MPI_T to a very limited set of the
existing MPI datatypes, and then required that implementations support
these after MPI_T_INIT but before MPI_INIT.
I think it's best to resolve this cleanly, even if that means things are
undefined between now and 3.1. I can think of two "clean" ways to
resolve this:
1) remove all overlap of MPI_T_ and MPI_ error codes, i.e., MPI_T_
errors live in a separate space from MPI_ errors, 0 = MPI_T_SUCCESS <
MPI_T_* < MPI_T_LAST_ERRCODE, and introduce new corresponding
MPI_T_ERROR_CLASS/STRING functions that can be invoked after
MPI_T_INIT. Then all MPI_T functions return MPI_T error codes (and only
MPI_T error codes), which must be interpretted with the new MPI_T_ERROR*
functions. Personally, it feels a little strange to pass an MPI_T_
error code to standard MPI_ functions, since we've otherwise separated
the interfaces as much as possible.
2) Specify that MPI_T error codes can be passed to the existing
MPI_ERROR* functions, but require that implementations allow one to call
these functions before MPI_INIT (at least for MPI_T_ error codes).
By the way, does the standard say anything about calling MPI_ERROR*
functions before MPI_INIT? What if someone wanted to process an error
code returned from one of the functions you are allowed to call before
MPI_INIT, such as MPI_INITIALIZED?
-Adam
Bronis R. de Supinski wrote:
>These are good points. I withdraw my agreement to the changes
>until Martin or Rolf addresses them.
>
>On Mon, 23 Jul 2012, Mohror, Kathryn wrote:
>
>
>
>>I have some concerns over this.
>>
>>
>>
>>>The topic about user-callability of MPI_ERROR_CLASS and MPI_ERROR_STRING
>>>outside of the initialized MPI should be revisited in MPI-3.1.
>>>This is not evident for the tools' developers because they know
>>>the meaning of their MPI_T_ERR... classes and only classes are returned.
>>>
>>>
>>If we are going to treat MPI_T error codes the same way they are in the rest of the standard, does it make sense to say that "only classes are returned" by MPI_T calls? I think that MPI calls return codes and the codes are translated to classes by the MPI_ERROR_CLASS function.
>>
>>If MPI_ERROR_CLASS and MPI_ERROR_STRING are not available before MPI_Init, then this may be a hardship on tools. I imagine many tools doing a significant amount of set up before MPI_Init to avoid overhead. However, if there is no information about what the return codes mean, then tools may have difficulty.
>>
>>Possibly it makes more sense to treat MPI_T return values differently than MPI calls? Could the MPI_T return values be a set of defined codes instead?
>>
>>Does anyone have different insight on error codes vs classes?
>>
>>Kathryn
>>
>>
>>
>>>-----Original Message-----
>>>From: mpi-forum-bounces at lists.mpi-forum.org [mailto:mpi-forum-
>>>bounces at lists.mpi-forum.org] On Behalf Of Bronis R. de Supinski
>>>Sent: Monday, July 23, 2012 10:37 AM
>>>To: Main MPI Forum mailing list
>>>Subject: Re: [Mpi-forum] MPI-3: MPI_T_ERR_... and MPI_ERR_LASTCODE
>>>
>>>
>>>OK by me.
>>>
>>>On Mon, 23 Jul 2012, Dave Goodell wrote:
>>>
>>>
>>>
>>>>I agree, we should make these changes.
>>>>
>>>>-Dave
>>>>
>>>>On Jul 23, 2012, at 8:55 AM CDT, Schulz, Martin wrote:
>>>>
>>>>
>>>>
>>>>>Hi Rolf, all,
>>>>>
>>>>>I agree with your suggestions and I am in favor of making these changes.
>>>>>
>>>>>
>>>Bronis (as well as Kathryn and Dave as the remaining members in the chapter
>>>committee for tools) should concur, though.
>>>
>>>
>>>>>Anybody else have any concerns over these changes?
>>>>>
>>>>>As for the callability of the error functions, I also agree - let's target those in
>>>>>
>>>>>
>>>3.1. This change is not vital and bit too big for final edits.
>>>
>>>
>>>>>Thanks,
>>>>>
>>>>>Martin
>>>>>
>>>>>
>>>>>On Jul 23, 2012, at 4:00 AM, Rolf Rabenseifner wrote:
>>>>>
>>>>>
>>>>>
>>>>>>Martin et al.,
>>>>>>
>>>>>>Consistently with your proposal, I would recommend:
>>>>>>
>>>>>>1) Sect 14.3.9 "Return Codes for the MPI tool information interface"
>>>>>>last sentence reads
>>>>>>
>>>>>> All return codes with the prefix MPI_T_ must be unique values and
>>>>>> cannot overlap with any other return values returned by the MPI
>>>>>> implementation.
>>>>>>
>>>>>>but should read
>>>>>>
>>>>>>
>>>>>>
>>>>>>>All return codes with the prefix MPI_T_ must be unique values and
>>>>>>>cannot overlap with any other
>>>>>>>
>>>>>>>
>>>>>> error codes and error classes
>>>>>>
>>>>>>
>>>>>>>returned by the MPI
>>>>>>>implementation. Further, they shall be treated as MPI error classes as
>>>>>>>defined in Chapter 8.4 and follow the same rules and restrictions,
>>>>>>>
>>>>>>>
>>>>>> especially they must satisfy
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>>>>0 = MPI_SUCCESS < MPI_T_ERR_... \leq MPI_ERR_LASTCODE.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>2) A.1.1 the first three tables are headed by
>>>>>>
>>>>>> Error classes
>>>>>> Error classes (continued)
>>>>>> Return Codes for the MPI tool information interface
>>>>>>
>>>>>>and these lines should read
>>>>>>
>>>>>> Error classes
>>>>>> Error classes (continued)
>>>>>> Error classes (continued)
>>>>>>
>>>>>>3) The last line of A.1.1, 2nd table
>>>>>>
>>>>>> MPI_ERR_LASTCODE
>>>>>>
>>>>>>should move after the last MPI_T_ERR_... code in the 3rd table.
>>>>>>
>>>>>>Okay?
>>>>>>
>>>>>>The topic about user-callability of MPI_ERROR_CLASS and
>>>>>>
>>>>>>
>>>MPI_ERROR_STRING
>>>
>>>
>>>>>>outside of the initialized MPI should be revisited in MPI-3.1.
>>>>>>This is not evident for the tools' developers because they know
>>>>>>the meaning of their MPI_T_ERR... classes and only classes are returned.
>>>>>>
>>>>>>I would like to have a final "okay" before I execute
>>>>>>the changes in chap-appLang.
>>>>>>
>>>>>>Best regards
>>>>>>Rolf
>>>>>>
>>>>>>----- Original Message -----
>>>>>>
>>>>>>
>>>>>>>From: "Martin Schulz" <schulzm at llnl.gov>
>>>>>>>To: "Main MPI Forum mailing list" <mpi-forum at lists.mpi-forum.org>
>>>>>>>Sent: Monday, July 23, 2012 1:23:40 AM
>>>>>>>Subject: Re: [Mpi-forum] MPI-3: MPI_T_ERR_... and
>>>>>>>
>>>>>>>
>>>MPI_ERR_LASTCODE
>>>
>>>
>>>>>>>Hi Rolf, all,
>>>>>>>
>>>>>>>I thought we had discussed the error semantics of MPI_T in several
>>>>>>>meetings/readings and nobody objected, but I generally agree with your
>>>>>>>comments below. However, they are not really error classes (at least
>>>>>>>in the tools group we/I never thought about them this way), but just
>>>>>>>well defined return codes. Nevertheless, they fit the model and
>>>>>>>semantics of classes and hence should integrated into the same rules
>>>>>>>for consistency. I also agree with Bronis, though, that larger
>>>>>>>feedback on this would be good to avoid errors because of rushing it.
>>>>>>>
>>>>>>>To keep changes minimal, I would suggest that we only change the
>>>>>>>following sentence, which IMHO is sufficient (and would essentially
>>>>>>>follow option b):
>>>>>>>
>>>>>>>All return codes with the prefix MPI_T_ must be unique values and
>>>>>>>cannot overlap with any other return values returned by the MPI
>>>>>>>implementation. Further, they shall be treated as MPI error classes as
>>>>>>>defined in Chapter 8.4 and follow the same rules and restrictions.
>>>>>>>
>>>>>>>This would also make changes in the appendix unnecessary.
>>>>>>>
>>>>>>>As for making MPI_ERROR_CLASS and MPI_ERROR_STRING callable
>>>>>>>
>>>>>>>
>>>before
>>>
>>>
>>>>>>>Init (and then also after finalize), yes that would be very useful,
>>>>>>>but is a general issue not only related to MPI_T. If we decide this is
>>>>>>>too invasive at this point, I would like to see this at least in 3.1.
>>>>>>>
>>>>>>>Martin
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>On Jul 22, 2012, at 1:46 PM, Rolf Rabenseifner wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>About 1. question:
>>>>>>>>
>>>>>>>>I changed the headers of the three error code tables in the Annex
>>>>>>>>according to Tables 8.1 and 8.2 from "Return codes"
>>>>>>>>into "Error classes".
>>>>>>>>
>>>>>>>>svn r1505: I used also "Error classes" for the tools table
>>>>>>>>svn r1507: I went back to the official ticket 266 text "Return
>>>>>>>>codes"
>>>>>>>>
>>>>>>>>The Tools group should decide whether
>>>>>>>>a. they want to stay with special return codes that are not
>>>>>>>>part of the rule in Set. 8.4
>>>>>>>>
>>>>>>>>0 = MPI_SUCCESS < MPI_ERR_... <= MPI_ERR_LASTCODE
>>>>>>>>
>>>>>>>>In this case they should change "return code"
>>>>>>>>into "error codes are returned" plus noting that
>>>>>>>>the routine MPI_ERROR_STRING can be applied
>>>>>>>>with the open question, whether MPI_EEROR_STRING
>>>>>>>>can be applied before a call to MPI_INIT.
>>>>>>>>
>>>>>>>>Similar problem with failed MPI_INIT and analysing its
>>>>>>>>returned error code:
>>>>>>>>One needs at least MPI_ERROR_CLASS and MPI_ERROR_STRING
>>>>>>>>callable before MPI_INIT.
>>>>>>>>
>>>>>>>>b. or you may use the same terminology as in Sect. 8.4,
>>>>>>>>then new table 14.5 must show "Error classes".
>>>>>>>>
>>>>>>>>In this case you should integrate the MPI_T_ERR_...
>>>>>>>>into the MPI_ERR_LASTCODE rule.
>>>>>>>>
>>>>>>>>You need at least MPI_ERROR_CLASS and MPI_ERROR_STRING
>>>>>>>>callable before MPI_INIT.
>>>>>>>>
>>>>>>>>I would prefer solution b together combined with the
>>>>>>>>minimal solution for failed MPI_INIT, i.e.,
>>>>>>>>adding MPI_ERROR_CLASS and MPI_ERROR_STRING to
>>>>>>>>the list of routines callable outside of MPI's
>>>>>>>>initialization.
>>>>>>>>
>>>>>>>>
>>>>>>>>About 2. question: MPI_T_ERR_CANTINIT -_>
>>>>>>>>
>>>>>>>>
>>>MPI_T_ERR_CANNOTINIT
>>>
>>>
>>>>>>>>svn r1506: done by Bronis in chap-tools/mpit.tex
>>>>>>>>svn r1508: done by Rolf in chap-appLang/appLang-Const.tex
>>>>>>>>
>>>>>>>>Rolf
>>>>>>>>
>>>>>>>>----- Original Message -----
>>>>>>>>
>>>>>>>>
>>>>>>>>>From: "Bronis R. de Supinski" <bronis at llnl.gov>
>>>>>>>>>To: "Main MPI Forum mailing list" <mpi-forum at lists.mpi-forum.org>
>>>>>>>>>Cc: "Rolf Rabenseifner" <rabenseifner at hlrs.de>
>>>>>>>>>Sent: Sunday, July 22, 2012 10:03:44 PM
>>>>>>>>>Subject: Re: [Mpi-forum] MPI-3: MPI_T_ERR_... and
>>>>>>>>>
>>>>>>>>>
>>>MPI_ERR_LASTCODE
>>>
>>>
>>>>>>>>>Rolf:
>>>>>>>>>
>>>>>>>>>OK, I am changing MPI_T_ERR_CANTINIT to
>>>>>>>>>
>>>>>>>>>
>>>MPI_T_ERR_CANNOTINIT.
>>>
>>>
>>>>>>>>>As said before, we need more opinions on the first question.
>>>>>>>>>
>>>>>>>>>Bronis
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>On Sun, 22 Jul 2012, Bronis R. de Supinski wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>Rolf:
>>>>>>>>>>
>>>>>>>>>>Re:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>Dear Bronis, Martin, Dave, and Kathryn,
>>>>>>>>>>>(Profiling/Tools chapter responsibles)
>>>>>>>>>>>
>>>>>>>>>>>(you may forward this to the tools list - I'm not on that list)
>>>>>>>>>>>
>>>>>>>>>>>Two important questions:
>>>>>>>>>>>
>>>>>>>>>>>1. Alexander Supalov detected that we did it wrong:
>>>>>>>>>>>The MPI_T_ERR_... list must be part of the total error
>>>>>>>>>>>list and sorted in before MPI_ERR_LASTCODE.
>>>>>>>>>>>
>>>>>>>>>>>I will change this in chap-appLang/appLang-Const.tex
>>>>>>>>>>>
>>>>>>>>>>>Please look, if there must be some additional wording on this.
>>>>>>>>>>>For example, the last sentence of Section 14.3.9
>>>>>>>>>>>
>>>>>>>>>>>"All return codes with the prefix MPI_T_ must be unique values
>>>>>>>>>>>and
>>>>>>>>>>>cannot overlap with any other return values returned by the MPI
>>>>>>>>>>>implementation."
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>The above wording clearly states that the return values
>>>>>>>>>>should be consistent with MPI_ERR_* return values. So,
>>>>>>>>>>either the above wording needs to change (I do not see
>>>>>>>>>>why MPI_T_* return values need to be distinct from
>>>>>>>>>>MPI_ERR_* return values since the user should know that
>>>>>>>>>>they were using an MPI_T_* function) or we need to adopt
>>>>>>>>>>something like what you suggest.
>>>>>>>>>>
>>>>>>>>>>I think we need broader opinions to make the decision.
>>>>>>>>>>
>>>>>>>>>>I am concerned that wrapping up the document for MPI 3.0
>>>>>>>>>>has led to a fast and loose attitude about making broader
>>>>>>>>>>changes "in order to get it done." This attitude can
>>>>>>>>>>easily lead to suboptimal solutions.
>>>>>>>>>>
>>>>>>>>>>Bronis
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>may be modified into
>>>>>>>>>>>
>>>>>>>>>>>"All return codes with the prefix MPI_T_ must be unique values
>>>>>>>>>>>and
>>>>>>>>>>>cannot overlap with any other return values returned by the MPI
>>>>>>>>>>>implementation and satisfy
>>>>>>>>>>>
>>>>>>>>>>>0 = MPI_SUCCESS < MPI_T_ERR_... <= MPI_ERR_LASTCODE. "
>>>>>>>>>>>
>>>>>>>>>>>or
>>>>>>>>>>>0 = MPI_SUCCESS < MPI_ERR_... < MPI_T_ERR_... <=
>>>>>>>>>>>MPI_ERR_LASTCODE.
>>>>>>>>>>>
>>>>>>>>>>>or
>>>>>>>>>>>0 = MPI_SUCCESS < MPI_ERR_... <= MPI_ERR_LASTCODE <
>>>>>>>>>>>MPI_T_ERR_... <= MPI_T_ERR_LASTCODE.
>>>>>>>>>>>
>>>>>>>>>>>and the MPI_ERR_LASTCODE and/or MPI_T_ERR_LASTCODE may
>>>>>>>>>>>
>>>>>>>>>>>
>>>be also
>>>
>>>
>>>>>>>>>>>repeated
>>>>>>>>>>>in Table 14.5 as last entry, as done for the first value
>>>>>>>>>>>MPI_SUCCESS.
>>>>>>>>>>>
>>>>>>>>>>>This proposal is based on Section 8.4, the sentence
>>>>>>>>>>>
>>>>>>>>>>>"The error codes satisfy,
>>>>>>>>>>>0 = MPI_SUCCESS < MPI_ERR_... <= MPI_ERR_LASTCODE: "
>>>>>>>>>>>
>>>>>>>>>>>Please confirm, that Alexander is right,
>>>>>>>>>>>please include me in your discussion and
>>>>>>>>>>>please tell me your result
>>>>>>>>>>>that I can do the correct changes in chap-appLang
>>>>>>>>>>>and you in chap-tools.
>>>>>>>>>>>
>>>>>>>>>>>2. Qeustion is less important; it is about naming:
>>>>>>>>>>>Alexander also noticed that your names do not fit to the names
>>>>>>>>>>>used in the past:
>>>>>>>>>>>
>>>>>>>>>>>MPI_T_ERR_CANTINIT --> MPI_T_ERR_CANNOTINIT (we do not use
>>>>>>>>>>>abbrevations)
>>>>>>>>>>>
>>>>>>>>>>>or
>>>>>>>>>>>
>>>>>>>>>>>MPI_T_ERR_CANTINIT --> MPI_T_ERR_CANNOT_INIT (we do not
>>>>>>>>>>>
>>>>>>>>>>>
>>>use
>>>
>>>
>>>>>>>>>>>abbrevations)
>>>>>>>>>>>
>>>>>>>>>>>In most cases, we do not combine words without underscore.
>>>>>>>>>>>
>>>>>>>>>>>As far as I see, you can do the underscores.
>>>>>>>>>>>Be sure to stay at maximum with 30 characters.
>>>>>>>>>>>
>>>>>>>>>>>Current longest MPI_T constants:
>>>>>>>>>>>
>>>>>>>>>>>123456789012345678901234567890
>>>>>>>>>>>MPI_T_PVAR_CLASS_HIGHWATERMARK
>>>>>>>>>>>MPI_T_PVAR_CLASS_LOWWATERMARK
>>>>>>>>>>>MPI_T_VERBOSITY_MPIDEV_DETAIL
>>>>>>>>>>>
>>>>>>>>>>>Based on this, I would stay with you names based on your MPI_T
>>>>>>>>>>>rule:
>>>>>>>>>>>MPI_T_<structual-areas-with-underscores>_<final-name-without-
>>>>>>>>>>>
>>>>>>>>>>>
>>>underscores>
>>>
>>>
>>>>>>>>>>>i.e., I would only change
>>>>>>>>>>>
>>>>>>>>>>>MPI_T_ERR_CANTINIT --> MPI_T_ERR_CANNOTINIT
>>>>>>>>>>>
>>>>>>>>>>>Please tell me also your decision
>>>>>>>>>>>that I can do the correct changes in chap-appLang
>>>>>>>>>>>and you in chap-tools.
>>>>>>>>>>>
>>>>>>>>>>>Best regards
>>>>>>>>>>>Rolf
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>--
>>>>>>>>>>>Dr. Rolf Rabenseifner . . . . . . . . . .. email
>>>>>>>>>>>rabenseifner at hlrs.de
>>>>>>>>>>>High Performance Computing Center (HLRS) . phone
>>>>>>>>>>>++49(0)711/685-65530
>>>>>>>>>>>University of Stuttgart . . . . . . . . .. fax ++49(0)711 /
>>>>>>>>>>>685-65832
>>>>>>>>>>>Head of Dpmt Parallel Computing . . .
>>>>>>>>>>>www.hlrs.de/people/rabenseifner
>>>>>>>>>>>Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring
>>>>>>>>>>>30)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>_______________________________________________
>>>>>>>>>>mpi-forum mailing list
>>>>>>>>>>mpi-forum at lists.mpi-forum.org
>>>>>>>>>>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>--
>>>>>>>>Dr. Rolf Rabenseifner . . . . . . . . . .. email
>>>>>>>>rabenseifner at hlrs.de
>>>>>>>>High Performance Computing Center (HLRS) . phone
>>>>>>>>++49(0)711/685-65530
>>>>>>>>University of Stuttgart . . . . . . . . .. fax ++49(0)711 /
>>>>>>>>685-65832
>>>>>>>>Head of Dpmt Parallel Computing . . .
>>>>>>>>www.hlrs.de/people/rabenseifner
>>>>>>>>Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)
>>>>>>>>_______________________________________________
>>>>>>>>mpi-forum mailing list
>>>>>>>>mpi-forum at lists.mpi-forum.org
>>>>>>>>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>______________________________________________________________________
>>>__
>>>
>>>
>>>>>>>Martin Schulz, schulzm at llnl.gov, http://people.llnl.gov/schulzm
>>>>>>>CASC @ Lawrence Livermore National Laboratory, Livermore, USA
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>_______________________________________________
>>>>>>>mpi-forum mailing list
>>>>>>>mpi-forum at lists.mpi-forum.org
>>>>>>>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>>>>>>>
>>>>>>>
>>>>>>--
>>>>>>Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de
>>>>>>High Performance Computing Center (HLRS) . phone ++49(0)711/685-
>>>>>>
>>>>>>
>>>65530
>>>
>>>
>>>>>>University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832
>>>>>>Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner
>>>>>>Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>______________________________________________________________________
>>>__
>>>
>>>
>>>>>Martin Schulz, schulzm at llnl.gov, http://people.llnl.gov/schulzm
>>>>>CASC @ Lawrence Livermore National Laboratory, Livermore, USA
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>_______________________________________________
>>>>>mpi-forum mailing list
>>>>>mpi-forum at lists.mpi-forum.org
>>>>>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>>>>>
>>>>>
>>>>_______________________________________________
>>>>mpi-forum mailing list
>>>>mpi-forum at lists.mpi-forum.org
>>>>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>>>>
>>>>
>>>>
>>>_______________________________________________
>>>mpi-forum mailing list
>>>mpi-forum at lists.mpi-forum.org
>>>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>>>
>>>
>>_______________________________________________
>>mpi-forum mailing list
>>mpi-forum at lists.mpi-forum.org
>>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>>
>>
>>
>_______________________________________________
>mpi-forum mailing list
>mpi-forum at lists.mpi-forum.org
>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
>
>
More information about the mpi-forum
mailing list