[MPI3 Fortran] [Interop-tr] [Mpi-forum] Comment on Fortran WG5 ballot N1846

[John Reid]

Rolf Rabenseifner wrote:
> [I expect that the mail is not delivered by interop-tr at j3-fortran.org 
>  because I'm not a member of J3]

You do not need to be a member of J3. Would you like to join interop-tr?

> After all the discussion, it really looks best to
> ignore the existence of the ASYNCHRONOUS attribute
> when looking at the co-existence of Fortran nonblocking 
> and MPI.

Although the idea of needing no text in our TR is welcome, I do not agree with 
your conclusion. Consider the code

REAL :: buf(100,100)
    : ! Code that involves buf
CALL MPI_Irecv(buf,...req,...)
    :  ! Code that does not involve buf
CALL MPI_Wait(req,...)
    :  ! Code that involves buf

Since there is no mention of buf in the call of MPI_Wait, the compiler is free 
to perform code motion involving it. For example, it might load buf into a GPU 
during the calculations that do not involve buf, ready for those that do.

Asynchronous communication is so like asynchronous I/O that I think we need to 
have the same rules. If we can somehow write this into the normative text of the 
TR, the effect of adding ASYNCHRONOUS to the declaration of buf in the above 
example would be that the compiler will not be allowed to make the harmful 
change that I have just outlined.

Cheers,

John.


> 
> Based on the discussion, I plan to add the following text
> into the draft for MPI-3.0 and based on Option 1 in a 
> previous email:
> 
> ------------------
> [Text about an application with parts of an array
> involved in a nonblocking MPI operation and other 
> parts involved in computation, and a compiler optimization 
> with temporarily copying variables into a local memory 
> and afterward back to the original memory]
> 
> Note that this type of compiler optimization can 
> not be prevented when buf is declared with the 
> ASYNCHRONOUS Fortran attribute, and it should not 
> be prevented by declaring buf as VOLATILE because:
>  - A Fortran asynchronous input/output operation on a part 
>    of an array variable implies that the access to the whole 
>    array (the so-called pending I/O storage sequence affector,
>    see Section 9.6.2.5 in [Fortran 2008]) is restricted 
>    as if the whole array is involved in the pending I/O 
>    (see Section 9.6.4.1, paragraphs 5 and 6 in [Fortran 2008]). 
>    Therefore, the ASYNCHRONOUS cannot be used if parts of the 
>    array is involved in a asynchronous operation and other 
>    parts are used for computation.
>  - In principle, the Fortran standard restricts the scope 
>    of the ASYNCHRONOUS attribute only to Fortran asynchronous 
>    input and output operations.
>  - A Fortran compiler may implement the Fortran asynchronous 
>    input/output operations as blocking I/O. In this case, 
>    the compiler can fully ignore the ASYNCHRONOUS attribute.
>  - The VOLATILE implies that all accesses to any storage unit (word) 
>    of buf must be directly done in the main memory exactly in 
>    the sequence defined by the application program. The attribute 
>    VOLATILE prevents every register or cache optimization.
>  - Therefore, VOLATILE may cause a huge performance degradation.
> 
> Instead of solving the problem, it is better to prevent the problem, 
> i.e., when overlapping communication and computation, the communication
> (or nonblocking IO) and the computation should be executed on different
> sets of variables. In this case, the temporary memory modifications 
> are done only on the variables used in the computation and cannot 
> have any side effect on the data used in the nonblocking MPI operations.
> ---------------
> 
> The first three items in the list suggest that it is a bad idea
> to enlarge the scope of ASYNCHRONOUS to nonblocking MPI.
> 
> Best regards
> Rolf
> 
> ----- Original Message -----
>> From: "N.M. Maclaren" <nmm1 at cam.ac.uk>
>> To: "John Reid" <John.Reid at stfc.ac.uk>, interop-tr at j3-fortran.org
>> Cc: interop-tr at j3-fortran.org, "Rolf Rabenseifner" <rabenseifner at hlrs.de>
>> Sent: Thursday, April 14, 2011 11:33:32 AM
>> Subject: Re: [Interop-tr] [Mpi-forum] Comment on Fortran WG5 ballot N1846
>> On Apr 14 2011, John Reid wrote:
>>> If the comment in the middle is changed to
>>>
>>>   ! asynchronous I/O on ptr variable
>>>
>>> the program would be non-conforming because A would be an affector
>>> without the ASYNCHRONOUS attribute.
>>>
>>> Perhaps we need normative text that says (somehow) that asynchronous
>>> communication is regarded as if it were asynchronous i/o.
>> In which case my reading of the standard is wrong. I have just looked
>> again, and think that either I probably was or the wording is more
>> confusing than it should be.
>>
>> I don't think that Fortran intends to forbid:
>>
>> PROGRAM Main
>> REAL :: array(100)
>> CALL Fred(array(:50),array(51:))
>> END PROGRAM Main
>>
>> SUBROUTINE Fred (buffer, scratch)
>> REAL, ASYNCHRONOUS :: buffer(:)
>> REAL :: scratch(:)
>> CALL MPI_Irecv(buffer,...)
>> CALL DGEMM(scratch,...)
>> CALL MPI_Wait(...)
>> END SUBROUTINE Fred
>>
>> As you say, the same problem can be shown using Fortran asynchronous
>> I/O.
>>
>> However, this issue is not limited to ASYNCHRONOUS and affects
>> variables
>> that have the TARGET attribute, too. I assert that it is conceptually
>> identical to the one where TARGET is used, MPI_Irecv sets up a pointer
>> and DGEMM uses that pointer. I.e. it's a question of exactly when
>> association stops.
>>
>>
>> Regards,
>> Nick Maclaren.
> 

-- 
Scanned by iCritical.