[MPI3 Fortran] [Interop-tr] [Mpi-forum] Comment on Fortran WG5 ballot N1846

Rolf Rabenseifner rabenseifner at hlrs.de
Thu Apr 14 10:26:10 CDT 2011


John,

this is already handled by all hints about code movement and
register optimization in the old MPI-2.2 and in more
detail in the upcoming MPI-3.0.

To summarize, there are several options:
 - TARGET buf
 - Using a call to a dummy routine MPI_F_SYNC_REG(buf) 
   immediately after MPI_Wait
 - Storing buf as a module variable or in a common block
 - (VOALTILE buf, not recommended)

'ASYNCHRONOUS buf' is explicitly not an option, because
a Fortran compiler 
- that implements Fortran asynchronous input/output
  with blocking I/O routines
- is allowed to ignore the ASYNCHRONOUS attribute.

Only compilers with really asynchronous Fortran I/O
have to also internally imply the TARGET attribute when 
the ASYNCHRONOUS attribute is given in the application.

Is my analysis correct?


> You do not need to be a member of J3. Would you like to join
> interop-tr?
Most of the internals of the TR, I do not understand.
As I see, the list moderator can agree to deliver the few
mails from me also to the other members of the interop-tr.
It isn't a problem for me to put you, Bill, Nick, Craig, 
and Reinhold on the CC.

Best regards
Rolf


----- Original Message -----
> From: "John Reid" <John.Reid at stfc.ac.uk>
> To: "Rolf Rabenseifner" <rabenseifner at hlrs.de>
> Cc: "MPI-3 Fortran working group" <mpi3-fortran at lists.mpi-forum.org>, interop-tr at j3-fortran.org
> Sent: Thursday, April 14, 2011 4:33:56 PM
> Subject: Re: [Interop-tr] [Mpi-forum] Comment on Fortran WG5 ballot N1846
> Rolf Rabenseifner wrote:
> > [I expect that the mail is not delivered by
> > interop-tr at j3-fortran.org
> >  because I'm not a member of J3]
> 
> You do not need to be a member of J3. Would you like to join
> interop-tr?
> 
> > After all the discussion, it really looks best to
> > ignore the existence of the ASYNCHRONOUS attribute
> > when looking at the co-existence of Fortran nonblocking
> > and MPI.
> 
> Although the idea of needing no text in our TR is welcome, I do not
> agree with
> your conclusion. Consider the code
> 
> REAL :: buf(100,100)
> : ! Code that involves buf
> CALL MPI_Irecv(buf,...req,...)
> : ! Code that does not involve buf
> CALL MPI_Wait(req,...)
> : ! Code that involves buf
> 
> Since there is no mention of buf in the call of MPI_Wait, the compiler
> is free
> to perform code motion involving it. For example, it might load buf
> into a GPU
> during the calculations that do not involve buf, ready for those that
> do.
> 
> Asynchronous communication is so like asynchronous I/O that I think we
> need to
> have the same rules. If we can somehow write this into the normative
> text of the
> TR, the effect of adding ASYNCHRONOUS to the declaration of buf in the
> above
> example would be that the compiler will not be allowed to make the
> harmful
> change that I have just outlined.
> 
> Cheers,
> 
> John.
> 
> 
> >
> > Based on the discussion, I plan to add the following text
> > into the draft for MPI-3.0 and based on Option 1 in a
> > previous email:
> >
> > ------------------
> > [Text about an application with parts of an array
> > involved in a nonblocking MPI operation and other
> > parts involved in computation, and a compiler optimization
> > with temporarily copying variables into a local memory
> > and afterward back to the original memory]
> >
> > Note that this type of compiler optimization can
> > not be prevented when buf is declared with the
> > ASYNCHRONOUS Fortran attribute, and it should not
> > be prevented by declaring buf as VOLATILE because:
> >  - A Fortran asynchronous input/output operation on a part
> >    of an array variable implies that the access to the whole
> >    array (the so-called pending I/O storage sequence affector,
> >    see Section 9.6.2.5 in [Fortran 2008]) is restricted
> >    as if the whole array is involved in the pending I/O
> >    (see Section 9.6.4.1, paragraphs 5 and 6 in [Fortran 2008]).
> >    Therefore, the ASYNCHRONOUS cannot be used if parts of the
> >    array is involved in a asynchronous operation and other
> >    parts are used for computation.
> >  - In principle, the Fortran standard restricts the scope
> >    of the ASYNCHRONOUS attribute only to Fortran asynchronous
> >    input and output operations.
> >  - A Fortran compiler may implement the Fortran asynchronous
> >    input/output operations as blocking I/O. In this case,
> >    the compiler can fully ignore the ASYNCHRONOUS attribute.
> >  - The VOLATILE implies that all accesses to any storage unit (word)
> >    of buf must be directly done in the main memory exactly in
> >    the sequence defined by the application program. The attribute
> >    VOLATILE prevents every register or cache optimization.
> >  - Therefore, VOLATILE may cause a huge performance degradation.
> >
> > Instead of solving the problem, it is better to prevent the problem,
> > i.e., when overlapping communication and computation, the
> > communication
> > (or nonblocking IO) and the computation should be executed on
> > different
> > sets of variables. In this case, the temporary memory modifications
> > are done only on the variables used in the computation and cannot
> > have any side effect on the data used in the nonblocking MPI
> > operations.
> > ---------------
> >
> > The first three items in the list suggest that it is a bad idea
> > to enlarge the scope of ASYNCHRONOUS to nonblocking MPI.
> >
> > Best regards
> > Rolf
> >
> > ----- Original Message -----
> >> From: "N.M. Maclaren" <nmm1 at cam.ac.uk>
> >> To: "John Reid" <John.Reid at stfc.ac.uk>, interop-tr at j3-fortran.org
> >> Cc: interop-tr at j3-fortran.org, "Rolf Rabenseifner"
> >> <rabenseifner at hlrs.de>
> >> Sent: Thursday, April 14, 2011 11:33:32 AM
> >> Subject: Re: [Interop-tr] [Mpi-forum] Comment on Fortran WG5 ballot
> >> N1846
> >> On Apr 14 2011, John Reid wrote:
> >>> If the comment in the middle is changed to
> >>>
> >>>   ! asynchronous I/O on ptr variable
> >>>
> >>> the program would be non-conforming because A would be an affector
> >>> without the ASYNCHRONOUS attribute.
> >>>
> >>> Perhaps we need normative text that says (somehow) that
> >>> asynchronous
> >>> communication is regarded as if it were asynchronous i/o.
> >> In which case my reading of the standard is wrong. I have just
> >> looked
> >> again, and think that either I probably was or the wording is more
> >> confusing than it should be.
> >>
> >> I don't think that Fortran intends to forbid:
> >>
> >> PROGRAM Main
> >> REAL :: array(100)
> >> CALL Fred(array(:50),array(51:))
> >> END PROGRAM Main
> >>
> >> SUBROUTINE Fred (buffer, scratch)
> >> REAL, ASYNCHRONOUS :: buffer(:)
> >> REAL :: scratch(:)
> >> CALL MPI_Irecv(buffer,...)
> >> CALL DGEMM(scratch,...)
> >> CALL MPI_Wait(...)
> >> END SUBROUTINE Fred
> >>
> >> As you say, the same problem can be shown using Fortran
> >> asynchronous
> >> I/O.
> >>
> >> However, this issue is not limited to ASYNCHRONOUS and affects
> >> variables
> >> that have the TARGET attribute, too. I assert that it is
> >> conceptually
> >> identical to the one where TARGET is used, MPI_Irecv sets up a
> >> pointer
> >> and DGEMM uses that pointer. I.e. it's a question of exactly when
> >> association stops.
> >>
> >>
> >> Regards,
> >> Nick Maclaren.
> >
> 
> --
> Scanned by iCritical.

-- 
Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de
High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530
University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832
Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner
Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)



More information about the mpiwg-fortran mailing list