[MPI3 Fortran] ASYNC attribute / SYNC_POINT
Aleksandar Donev
donev1 at llnl.gov
Wed May 6 16:21:43 CDT 2009
We had a discussion about this option and concluded that it is not
necessary to mark the actual routines like WAIT, i.e., we concluded that
SYNC_POINT was not necessary. In fact, it is problematic because it
would require putting it on any routine that itself calls MPI_Wait.
Otherwise, what exactly is its point?
The compiler simply must not move accesses to ASYNC variables around
procedure calls it does not know enough about. This is already so for
TARGET, ASYNCHRONOUS, and VOLATILE.
Aleks
Rolf Rabenseifner wrote:
> I try to write my ASYNC/SYNC proposal in a similar way as
> Craigs proposal.
>
> ASYNC implies that a variable or a portion of it (e.g., a subarray) is
> potentially active in asynchronous operations outside of Fortran.
> Whether it is used asynchronously or which portion is used
> asynchronously, can be changed at each synchronization point.
> A synchronization point is either a routine call to a routine with the
> ASYNC variable as an actual argument, or a call to a routine that has an
> explicit interface with the SYNC_POINT option.
>
> If a variable is declare as ASYNC within a block, then within this block
> - motion of code involving this ASYNC variable would be suppressed
> across any synchronization point,
> - between two synchronization points, only those portions of the ASYNC
> variable can be written to the memory that are explicitly modified by
> the application program (i.e., caching of other portions, overwriting
> and restoring them for, e.g., loop fusion is prohibited),
> - if it is an actual argument in a call to a function or subroutine with
> a explicit interface then copyin/out of this argument is prohibited.
>
> If a dummy argument is declared as ASYNC in an explicit interface, then
> copyin/out of actual arguments is prohibited.
>
> Comments:
> - This is less restrictive than VOLATILE. For numerical operations,
> registers can be used as usual. Same data that is used in several
> iterations of a loop can be efficiently cached in registers independent
> of whether ASYNC is set or not.
>
> - All MPI WAIT, TEST and communication routines (because of usage of
> MPI_BOTTOM and absolute addresses in MPI derived datatypes) must be
> declared with SYNC_POINT option.
>
> Usage:
>
> ----------------
>
> real, ASYNC :: buf(100,100)
> type(MPI_Request) :: request
>
> ! initiate data transfer of boundary
> CALL MPI_IRecv(buf(:,1),...,request,...) ! no copyin/copyout will happen
>
> ! do work on interior of the buffer while transfer is in progress
> CALL UPDATE_INTERIOR(buf) ! dummy arg should have ASYNC attribute
>
> ! wait for the communication to finish
> CALL MPI_Wait(request,...) ! no code motion allowed with respect to buf
>
> ! finish work on buffer boundaries
> CALL UPDATE_BOUNDARIES(buf) ! should copyin/copyout be allowed here?
>
> ----------------
>
> interface
>
> SUBROUTINE UPDATE_INTERIOR(buf)
> TYPE(*), DIMENSION(..), ASYNC :: buf
> ! ASYNC is necessary to guarantee that
> ! only those portions of buf are written to
> ! the memory that are explicitly modified
> ! by UPDATE_INTERIOR
> END SUBROUTINE UPDATE_INTERIOR
>
> SUBROUTINE UPDATE_BOUNDARIES(buf)
> TYPE(*), DIMENSION(..) :: buf
> END SUBROUTINE UPDATE_BOUNDARIES
>
>
> SUBROUTINE MPI_Irecv(buf, count, datatype, source, tag, comm, request,
> err) SYNC_POINT BIND(C, name="MPI_Irecv")
> import MPI_Datatype, MPI_Comm, MPI_Request
> TYPE(*), DIMENSION(..), ASYNC :: buf
> integer, value, intent(in) :: count
> type(MPI_Datatype), intent(in) :: datatype
> integer, value, intent(in) :: source
> integer, value, intent(in) :: tag
> type(MPI_Comm), intent(in) :: comm
> type(MPI_Request), intent(out) :: request
> integer, intent(out) :: err
> END SUBROUTINE MPI_Irecv
>
> SUBROUTINE MPI_Isend(buf, count, datatype, dest, tag, comm, request,
> err) SYNC_POINT BIND(C, name="MPI_Isend")
> import MPI_Datatype, MPI_Comm, MPI_Request
> TYPE(*), DIMENSION(..), ASYNC :: buf
> integer, value, intent(in) :: count
> type(MPI_Datatype), intent(in) :: datatype
> integer, value, intent(in) :: dest
> integer, value, intent(in) :: tag
> type(MPI_Comm), intent(in) :: comm
> type(MPI_Request), intent(out) :: request
> integer, intent(out) :: err
> END SUBROUTINE MPI_Isend
>
> SUBROUTINE MPI_Wait(request, status, err) SYNC_POINT BIND(C,
> name="MPI_Wait")
> import MPI_Request, MPI_Status
> type(MPI_Request), intent(in) :: request ! inout?
> type(MPI_Status), intent(out) :: status
> integer, intent(out) :: err
> END SUBROUTINE MPI_Wait
>
> end interface
>
> --------------
>
> Any comments?
>
> Best regards
> Rolf
>
>
>
>
>
> On Tue, 5 May 2009 17:45:00 -0600
> Craig Rasmussen <crasmussen at newmexicoconsortium.org> wrote:
>> I'm at the WG5 Fortran standards meeting and I think we've made
>> significant advances regarding the MPI-3 Fortran API. We are
>> discussing adding a new variable attribute, ASYNC. I'd like to get
>> feedback from the MPI-3 Fortran working group on these possible changes.
>>
>> ASYNC implies that a variable is potentially active in asynchronous
>> operations outside of Fortran. The use of this new attribute should
>> give the compiler enough information to inhibit optimizations similar
>> to inhibitions involved in Fortran asynchronous I/O, specifically
>> copyin/copyout and code motion. The compiler will likely have to
>> inhibit code motion regarding any use of a variable with the ASYNC
>> attribute and procedures other than intrinsics. The affect is
>> similar to the use of the volatile attribute (but not regarding loads
>> and stores).
>>
>> Usage is outlined below:
>>
>> ----------------
>>
>> real, ASYNC_EXTERNAL :: buf(100,100)
>> type(MPI_Request) :: request
>>
>> ! initiate data transfer of boundary
>> CALL MPI_IRecv(buf(:,1),...,request,...) ! no copyin/copyout will happen
>>
>> ! do work on interior of the buffer while transfer is in progress
>> CALL UPDATE_INTERIOR(buf) ! dummy arg should have ASYNC attribute
>>
>> ! wait for the communication to finish
>> CALL MPI_Wait(request,...) ! no code motion allowed with respect to
>> buf
>>
>> ! finish work on buffer boundaries
>> CALL UPDATE_BOUNDARIES(buf) ! should copyin/copyout be allowed here?
>>
>> ----------------
>>
>> So how does this look? Anything we've left out? I'm also including
>> the interface definition.
>>
>> - craig
>>
>> ----------------
>>
>> interface
>>
>> SUBROUTINE MPI_Irecv(buf, count, datatype, source, tag, comm,
>> request, err) BIND(C, name="MPI_Irecv")
>> import MPI_Datatype, MPI_Comm, MPI_Request
>> TYPE(*), DIMENSION(..), ASYNC :: buf
>> integer, value, intent(in) :: count
>> type(MPI_Datatype), intent(in) :: datatype
>> integer, value, intent(in) :: source
>> integer, value, intent(in) :: tag
>> type(MPI_Comm), intent(in) :: comm
>> type(MPI_Request), intent(out) :: request
>> integer, intent(out) :: err
>> END SUBROUTINE MPI_Irecv
>>
>> SUBROUTINE MPI_Isend(buf, count, datatype, dest, tag, comm, request,
>> err) BIND(C, name="MPI_Isend")
>> import MPI_Datatype, MPI_Comm, MPI_Request
>> TYPE(*), DIMENSION(..), ASYNC :: buf
>> integer, value, intent(in) :: count
>> type(MPI_Datatype), intent(in) :: datatype
>> integer, value, intent(in) :: dest
>> integer, value, intent(in) :: tag
>> type(MPI_Comm), intent(in) :: comm
>> type(MPI_Request), intent(out) :: request
>> integer, intent(out) :: err
>> END SUBROUTINE MPI_Isend
>>
>> SUBROUTINE MPI_Wait(request, status, err) BIND(C, name="MPI_Wait")
>> import MPI_Request, MPI_Status
>> type(MPI_Request), intent(in) :: request ! inout?
>> type(MPI_Status), intent(out) :: status
>> integer, intent(out) :: err
>> END SUBROUTINE MPI_Wait
>>
>> end interface
>>
>>
>>
>>
>>
>> _______________________________________________
>> mpi3-fortran mailing list
>> mpi3-fortran at lists.mpi-forum.org
>> http:// lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran
>
> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner at hlrs.de
> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530
> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832
> Head of Dpmt Parallel Computing . . . www. hlrs.de/people/rabenseifner
> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)
> _______________________________________________
> mpi3-fortran mailing list
> mpi3-fortran at lists.mpi-forum.org
> http:// lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-fortran
>
--
Aleksandar Donev, Ph.D.
Lawrence Postdoctoral Fellow @ LLNL
High Performance Computational Materials Science and Chemistry
E-mail: donev1 at llnl.gov
Phone: (925) 424-6816 Fax: (925) 423-0785
Address: P.O.Box 808, L-367, Livermore, CA 94551-9900
Web: http://cims.nyu.edu/~donev
More information about the mpiwg-fortran
mailing list