[Mpi-forum] Question on Cartesian Topologies
Rolf Rabenseifner
rabenseifner at hlrs.de
Tue Mar 8 04:25:30 CST 2011
Summary of my proposal:
-----------------------
I propose to clarify:
MPI-2.2, page 248, line 33 reads
If ndims is zero then a zero-dimensional Cartesian
topology is created.
but should read
If ndims > 0 and the product of dims[i] is zero
then MPI_COMM_NULL is returned in all processes.
If ndims is zero then and in each process,
in comm_cart a communicator is retunred
that contains only the process itself
and that is accociated with a zero-dimensional
Cartesian topology (i.e., comm_cart would
be the same if all processes in the group
of comm would have called MPI_CART_CREATE with
ndims=0 and MPI_COMM_SELF as input communicator).
----------------------
Details and reasons:
I agree that we should clarify the standard.
To be compatible with existing implementations
(they may handle it as an error), I would propose:
Add in MPI-2.2, page 248, line 34,
before "If ndims is zero":
If ndims > 0 and the product of dims[i] is zero
then MPI_COMM_NULL is returned in all processes.
This solution is in analogy to the definition of
MPI_GRAPH_CREATE in the case of nnodes==0,
see MPI-2.2, page 250, lines 30-31.
(For MPI_DIST_GRAPH_CREATE(_ADJACENT), we decided
another solution: Here, "isolated processes" are
allowed, see MPI-2.2, page 253, lines 30-33, and
page page 255, lines 15-17.
Additionally, I propose to clarify the next sentence,
i.e., MPI-2.2, page 248, line 33:
"If ndims is zero then a zero-dimensional Cartesian
topology is created."
Relevant is only the text in MPI_CART_CREATE and
MPI_CART_SUB.
The idea behind this additions in the standardization of
MPI-2.2 was, that MPI_CART_SUB had not forbidden
that all remain_dims[i] are false, and threfore
implementations had to return zero-dimensional
single-process-communicators in all processes.
In MPI_CART_CREATE, the outcome is not such clear,
because the number of processes that return
such a zero-dimensional topology (i.e. a point),
- can be ONE (and ann others return MPI_COMM_NULL)
- or SIZE(comm).
The idea behind the new wording in MPI-2.2 was
clearly to have it identical to the outcome
of MPI_COMM_SPLIT, i.e., all process return such a
zero-dimensional topology.
Therefore, I propose to clarify:
MPI-2.2, page 248, line 33 reads
If ndims is zero then a zero-dimensional Cartesian
topology is created.
but should read
If ndims > 0 and the product of dims[i] is zero
then MPI_COMM_NULL is returned in all processes.
If ndims is zero then and in each process,
comm_cart contains only the process itself
and comm_cart is accociated with a zero-dimensional
Cartesian topology (i.e., the outcme is
identical to calling this routine by all processes
in the group of comm, but with MPI_COMM_SELF
as input communicator).
Related lines to "zero-dimensional" are
- MPI_CARTDIM_GET, MPI-2.2, page 259, lines 35-37
- MPI_CART_RANK, MPI-2.2, page 260, lines 45-46
- MPI_CART_COORDS, MPI-2.2, page 261, lines 19-20
- MPI_CART_SHIFT, MPI-2.2, page 266, lines 1-4
- MPI_CART_SUB, MPI-2.2, page 267, lines 1-4
Best regards
Rolf
----- Original Message -----
> From: "Bronis R. de Supinski" <bronis at llnl.gov>
> To: "Main MPI Forum mailing list" <mpi-forum at lists.mpi-forum.org>
> Sent: Monday, March 7, 2011 4:22:09 PM
> Subject: Re: [Mpi-forum] Question on Cartesian Topologies
> Michael:
>
> I do not agree with your inference. The values in the
> dims array are not ranks so you cannot extend rank
> restrictions to them
>
> I can envision algorithms for which providing 0 for a
> dimension indicates that the communicator should be empty.
> It would be inefficient for the application to check
> if any dimension is 0. As the MPICH-2 implementation
> demonstrates, this choice can be supported. I'll
> admit that your interpretation is reasonable although
> I think returning MPI_COMM_NULL is better. I also
> agree that an error should be returned if any of the
> dims is negative.
>
> Regardless, the issue should not be left to inference.
> We need to update the standard to clarify this issue.
>
> Bronis
>
>
> On Mon, 7 Mar 2011, Michael Raymond wrote:
>
> > While there's nothing specified in the definition of
> > MPI_Cart_create(), you might be able to infer this from the
> > definition
> > of MPI_Cart_rank(). It specifies that a coordinate < 0 or >= the
> > size
> > of that dimension is out of range. 0 is >= 0, so it must be out of
> > range.
> >
> > Tobias Hilbrich wrote:
> >> Hi all,
> >>
> >> LANL, LLNL and the University of Dresden are working on a follow up
> >> tool of the runtime correctness checking tools Marmot and Umpire,
> >> which is done in cooperation by developers of both tools.
> >>
> >> We have some issues with basic correctness checks for integers.
> >> Were for most of the handles used in MPI calls there are fairly
> >> clear rules on what is allowed and what not, there are sometimes
> >> close to no rules for integer arguments. Most of the time common
> >> sense helps to come up with a reasonable assumption, but for the
> >> definition of cartesian communicators we are somewhat stuck, maybe
> >> you can help. (Maybe we missed some lines in the standard or are
> >> not aware of some base rule)
> >>
> >> What is unclear to us is "what is a correct definition for a
> >> cartesian topology?", we found no clear definition for that. So
> >> basically we care about the "ndims" and "dims" values in calls like
> >> MPI_Cart_create. Is it allowed to specify a value of 0 for one of
> >> the entries in the "dims" array? The standard specifies no
> >> restrictions in the text following MPI_Cart_create, I am referring
> >> to MPI-2.2 here. E.g. is the following correct or not:
> >>
> >> int dims[3]
> >> dims[0] = 1;
> >> dims[1] = 1;
> >> dims[2] = 0; /*Is this ok/suspicious/erroneous ?*/
> >> MPI_Cart_create( ... , 3 /*ndims*/, dims , ...);
> >>
> >> Our little testing revealed that it causes OpenMPI, LAM-MPI,
> >> MPICH-1 to crash, whereas MPICH-2 returns MPI_COMM_NULL here.
> >>
> >> many thanks, Tobias
> >>
> >> P.S.: This is probably not the last os such questions, if this is a
> >> bad way to ask these we would be glad to get suggestions on how to
> >> raise them. A tool that validates whether an MPI application
> >> behaves according to the standard or not, hopefully provides some
> >> motivates to think about these troublesome questions ;)
> >>
> >> --
> >> Dipl.-Inf. Tobias Hilbrich
> >> Wissenschaftlicher Mitarbeiter
> >>
> >> Technische Universitaet Dresden
> >> Zentrum fuer Informationsdienste und Hochleistungsrechnen (ZIH)
> >> (Center for Information Services and High Performance Computing
> >> (ZIH))
> >> Interdisziplinäre Anwenderunterstützung und Koordination
> >> (Interdisciplinary Application Development and Coordination)
> >> 01062 Dresden
> >> Tel.: +49 (351) 463-32041
> >> Fax: +49 (351) 463-37773
> >> E-Mail: tobias.hilbrich at zih.tu-dresden.de
> >>
> >>
> >> _______________________________________________
> >> mpi-forum mailing list
> >> mpi-forum at lists.mpi-forum.org
> >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
> >
> > --
> > Michael A. Raymond
> > Message Passing Toolkit Team
> > Silicon Graphics Inc
> > (651) 683-3434
> >
> > _______________________________________________
> > mpi-forum mailing list
> > mpi-forum at lists.mpi-forum.org
> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
> >
> _______________________________________________
> mpi-forum mailing list
> mpi-forum at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum
--
Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de
High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530
University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832
Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner
Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)
More information about the mpi-forum
mailing list