[MPIWG Fortran] Data type of F08 subarray

Bill Long longb at cray.com
Tue May 13 14:56:52 CDT 2014 

On May 13, 2014, at 2:48 PM, Junchao Zhang <jczhang at mcs.anl.gov> wrote:

> 
> On Tue, May 13, 2014 at 2:37 PM, Bill Long <longb at cray.com> wrote:
> 
> On May 13, 2014, at 2:19 PM, Junchao Zhang <jczhang at mcs.anl.gov> wrote:
> 
> >
> > On Tue, May 13, 2014 at 2:00 PM, Bill Long <longb at cray.com> wrote:
> >
> > On May 13, 2014, at 12:44 PM, Junchao Zhang <jczhang at mcs.anl.gov> wrote:
> >
> > >
> > > On Tue, May 13, 2014 at 11:56 AM, Rolf Rabenseifner <rabenseifner at hlrs.de> wrote:
> > > > REAL s(100)
> > > > MPI_SEND(s (1:100:5) , 3, dtype, ...)
> > > > dtype can only be MPI_REAL. In other words, dtype is kind of
> > > > redundant here since the type map is actually specified by the
> > > > subarray.
> >
> > Right.  The descriptor for the first argument has a member whose value is a type code.  In principal the library could verify this is compatible with the data type handle supplied as the third argument, and issue an error if not.  Perhaps in a “debug” mode.
> >
> > >
> > > No, if dtype is a vector then it is applied to a virtual
> > > contiguous array that consists of s(1), s(6), s(11) …
> >
> > dtype is not  a vector, is it?  That argument is a scalar of type TYPE(MPI_DATATYPE).  At least that is what the interface says.
> >
> > Rolf meant dtype is an MPI datatype created by MPI_Type_vector.  For this case, I will have two datatypes, one from the MPI_Datatype argument, the other from the choice buffer itself. It is hard to implement that. Perhaps it is useless since it obscures the program.
> 
> OK. But one of the virtues of the new interface for users is that you do not have to create such data types anymore for array sections.   Even if someone did do this, you can detect that the incoming data type is user-created, and in that case ignore the type code in the descriptor.  If the program is valid at all, the element length, strides, and extents in the descriptor should be correct.
> 
> Yes, I can do that. The hard part is when the subarray is non-contiguous, and it is a non-blocking call. I need to allocate a scratch buffer and pack the subarray. Since it is non-blocking, I can not free the buffer.

Can you create, locally, a datatype that describes the layout of the array section, and then call MPI_Isend again with that data type?  That avoids the contiguous local buffer and the problem of when to free it.

Cheers,
Bill




> 
> 
> Cheers,
> Bill
> 
> 
> 
> >
> >
> >
> > Cheers,
> > Bill
> >
> >
> > >
> > > That is nasty. Then I will have two data types. I even can not assume any relationship between the two types. I have to allocate a scratch buffer for the virtual contiguous array in MPI_ISEND etc, do memory copying and then free the buffer in MPI_WAIT. I'm not sure one can implement it efficiently.
> > >
> > >
> > >
> > > ----- Original Message -----
> > > > From: "Junchao Zhang" <jczhang at mcs.anl.gov>
> > > > To: "MPI-WG Fortran working group" <mpiwg-fortran at lists.mpi-forum.org>
> > > > Sent: Tuesday, May 13, 2014 6:23:08 PM
> > > > Subject: Re: [MPIWG Fortran] Data type of F08 subarray
> > > >
> > > >
> > > >
> > > > Thanks, Rolf. And I feel there is a jump from contiguous subarray to
> > > > non-contiguous subarray.
> > > >
> > > >
> > > >
> > > > For contiguous subarray, such as
> > > >
> > > >
> > > >
> > > >
> > > > REAL s(100)
> > > >
> > > > MPI_SEND(s(2:50), 3, dtype, ...) s only gives the start address.
> > > > dtype can be anything, e.g., either a basic type or a derived type
> > > > created by MPI_Type_vector() etc.
> > > >
> > > >
> > > >
> > > > For non-contiguous subarray, such as
> > > >
> > > >
> > > >
> > > > REAL s(100)
> > > > MPI_SEND(s (1:100:5) , 3, dtype, ...)
> > > > dtype can only be MPI_REAL. In other words, dtype is kind of
> > > > redundant here since the type map is actually specified by the
> > > > subarray.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > --Junchao Zhang
> > > >
> > > >
> > > > On Tue, May 13, 2014 at 10:20 AM, Rolf Rabenseifner <
> > > > rabenseifner at hlrs.de > wrote:
> > > >
> > > >
> > > > Dear Junchao,
> > > >
> > > > MPI-3.0 p25:7-8 describes only communication with language type
> > > > of the buffer argument matches to the MPI datatype used
> > > > in the datatype argument.
> > > > Same p83:36-37.
> > > >
> > > > Therefore, the answer is no and the compiler cannot detect
> > > > a mismatch beteen language buffer specification and
> > > > MPI datatype specification.
> > > >
> > > > I hope my answer could help.
> > > >
> > > > Best regards
> > > > Rolf
> > > >
> > > >
> > > >
> > > >
> > > > ----- Original Message -----
> > > > > From: "Junchao Zhang" < jczhang at mcs.anl.gov >
> > > > > To: "MPI-WG Fortran working group" <
> > > > > mpiwg-fortran at lists.mpi-forum.org >
> > > > > Sent: Tuesday, May 13, 2014 5:08:30 PM
> > > > > Subject: [MPIWG Fortran] Data type of F08 subarray
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > p626 of MPI-3.0 gives such an example
> > > > >
> > > > >
> > > > > REAL s(100), r(100)
> > > > > CALL MPI_Isend(s(1:100:5), 3, MPI_REAL, ..., rq, ierror)
> > > > >
> > > > > All nonblocking MPI functions behave as if the user-specified
> > > > > elements of choice buffers are copied to a contiguous scratch
> > > > > buffer
> > > > > in the MPI runtime environment. All datatype descriptions (in the
> > > > > example above, “3, MPI_REAL”) read and store data from and to this
> > > > > virtual contiguous scratch buffer ...
> > > > >
> > > > > Here, data type of s(100) match with MPI_REAL, so everything is
> > > > > fine.
> > > > > But I want to know if MPI permits mismatched types, for example,
> > > > > can
> > > > > s(100) be an integer array? If the answer is no, then compilers can
> > > > > not detect this error ; if yes, then it is hard to implement that.
> > > > > To avoid memory copying to a scratch buffer, I want to use MPI
> > > > > datatypes. But if I have two types, one is given by the choice
> > > > > buffer itself, the other is given by the MPI_Datatype argument, how
> > > > > could I do that?
> > > > >
> > > > > Any thoughts?
> > > > >
> > > > > Thanks
> > > > >
> > > > > --Junchao Zhang
> > > > > _______________________________________________
> > > > > mpiwg-fortran mailing list
> > > > > mpiwg-fortran at lists.mpi-forum.org
> > > > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran
> > > >
> > > > --
> > > > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de
> > > > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530
> > > > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832
> > > > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner
> > > > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
> > > > _______________________________________________
> > > > mpiwg-fortran mailing list
> > > > mpiwg-fortran at lists.mpi-forum.org
> > > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran
> > > >
> > > > _______________________________________________
> > > > mpiwg-fortran mailing list
> > > > mpiwg-fortran at lists.mpi-forum.org
> > > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran
> > >
> > > --
> > > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de
> > > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530
> > > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832
> > > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner
> > > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)
> > > _______________________________________________
> > > mpiwg-fortran mailing list
> > > mpiwg-fortran at lists.mpi-forum.org
> > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran
> > >
> > > _______________________________________________
> > > mpiwg-fortran mailing list
> > > mpiwg-fortran at lists.mpi-forum.org
> > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran
> >
> > Bill Long                                                                       longb at cray.com
> > Fortran Technical Suport  &                                  voice:  651-605-9024
> > Bioinformatics Software Development                     fax:  651-605-9142
> > Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101
> >
> >
> > _______________________________________________
> > mpiwg-fortran mailing list
> > mpiwg-fortran at lists.mpi-forum.org
> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran
> >
> > _______________________________________________
> > mpiwg-fortran mailing list
> > mpiwg-fortran at lists.mpi-forum.org
> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran
> 
> Bill Long                                                                       longb at cray.com
> Fortran Technical Suport  &                                  voice:  651-605-9024
> Bioinformatics Software Development                     fax:  651-605-9142
> Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101
> 
> 
> _______________________________________________
> mpiwg-fortran mailing list
> mpiwg-fortran at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran
> 
> _______________________________________________
> mpiwg-fortran mailing list
> mpiwg-fortran at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran

Bill Long                                                                       longb at cray.com
Fortran Technical Suport  &                                  voice:  651-605-9024
Bioinformatics Software Development                     fax:  651-605-9142
Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101

More information about the mpiwg-fortran mailing list