[Mpi-forum] Comment on Fortran WG5 ballot N1846

[Graham, Richard L.]

Bill,
  The concern that was raised to me (I have not followed the standard, but
rely on other folks in the forum for this) is that there is potential for
incorrect behavior if a single data object is being used both for
computation and for asynchronous communication (disjoint portions of the
object are involved in each).  The specific concern, if I understood
things correctly - others, please correct me if I am wrong - that
compilers may respect the ASYNCHRONOUS only with respect to other fortran
libraries, such as fortran async I/O, and feel free to modify portions of
the buffer being used for communications, only later to restore the
parameters. If this is indeed what could happen, this would clearly be a
correctness problem for MPI.  Seems to me that if this is indeed an issue,
MPI will not be the only thing impacted, but also code using other
languages, async libraries, etc.
  It is possible that there is a misunderstanding here, but with the
concern about potential correctness issues, want to raise this earlier
rather than later.
  Is there something to be concerned about here ?

Thanks,
Rich

On 3/30/11 11:24 PM, "Bill Long" <longb at cray.com> wrote:

>Hi Rich and others,
>
>
>On 3/30/11 11:51 AM, Graham, Richard L. wrote:
>> Dear member of the WG5 and J3 Fortran standardization committee,
>
>As I understand the core of this thread, the problem is that some
>vendors don't actually implement anything for ASYNCHRONOUS because their
>libraries do not support actual asynchronous I/O operations, which makes
>the ASYNCHRONOUS semantics moot.   If all of the vendors supported the
>semantics as expected, there would be no need for any changes.
>
>>
>> We recommend the following change of the first two paragraphs in this
>> section of the Fortran 2008 standard:
>>
>> 5.3.4 ASYNCHRONOUS attribute
>>
>> 1 An entity with the ASYNCHRONOUS attribute is a variable that may be
>>     subject to asynchronous input/output or other asynchronous access
>>     by means other than Fortran.
>>
>
>The "or other asynchronous access be means other than Fortran" seems
>semantically identical to "an object may be referenced, defined, or
>become undefined, by means not specified by the program.".    The second
>phrase is, of course, the definition of the VOLATILE attribute.  The
>proposed change, then, seems to have the effect of making ASYNCHRONOUS
>the same as VOLATILE.  Some comments on that possibility:
>
>1) It is probably true that some vendors implement ASYNCHRONOUS that way
>already.
>
>2) The vendors that are more precise in their ASYNCHRONOUS
>implementation would not be in favor of this change.
>
>3) If this is the only solution, then using VOLATILE directly, and
>forgetting about ASYNCHRONOUS, would be a solution for the MPI
>non-blocking transfer routines.  The implementation of VOLATILE is
>likely more uniform and widespread.
>
>
>> This should be treated only as a wording proposal.  We provide the
>> above text only as a suggestion; J3/WG5 are certainly free to adjust
>> it as they feel necessary.  The key idea is that we need ASYNCHRONOUS
>> to also include potential memory modifications from agents outside the
>> scope of the Fortran standard.
>>
>
>Right. Sounds like VOLATILE to me.
>
>> The following snipit is a correct code that allows the MPI library to
>> internally use DMA, communications offload (e.g., to a NIC), or other
>> method to asynchronously communicate parts of the array while the user
>> application/main CPU simultaneously reads and writes to ***another***
>> part of the same array.  Specifically, this code snipit is is free of
>> race conditions.
>>
>> USE mpi_f08
>> REAL, ASYNCHRONOUS :: buf(100,100)
>> ! communicating a boundary of the array:
>> CALL MPI_Irecv(buf(1,1:100),...req,...)
>> DO j=1,100
>>     DO i=2,100
>>       buf(i,j)=.... ! work only on the inner area
>>     END DO
>> END DO
>> CALL MPI_Wait(req,...)
>>
>
>It is not clear to me how this program example would show any difference
>in performance or semantics if ASYNCHRONOUS were replaced by VOLATILE.
>Even with the current definitions of the attributes, it is extremely
>unlikely that a compiler applies  the asynchronous attribute on an
>element by element basis for the array buf.
>
>
>>
>> With the current status of Fortran 2008 + TR 29113 (Version N1845), a
>> compiler can ignore the ASYNCHRONOUS attribute.  For example, a
>> compiler may choose to not implement Fortran asynchronous I/O in an
>> asynchronous way, meaning that there is no need for the compiler to
>> make any restrictions on optimization.
>>
>> To be clear: with the current Fortran 2008, this optimization is still
>> allowed.  With our proposal, this optimization will be prohibited.
>>
>> We want to mention that the use of VOLATILE would solve the problem,
>> but at a high performance price because it would also prohibit any
>> optimization of the "work" part (e.g., automatic cache optimization,
>> instruction reordering, etc.).  Especially since Fortran is used with
>> MPI in high-performance, numerically intensive applications, we feel
>> that VOLATILE cannot be recommended as a solution.
>>
>
>So, it appears that you are saying that the compiler cannot make the
>element-by-element distinction for the VOLATILE attribute, but it could
>for ASYNCHRONOUS (in order for your example to optimize the way you
>hope).  Seems like a weak point in the argument.
>
>I agree that VOLATILE has the issues you state. But I don't see how the
>new definition of ASYNCHRONOUS avoids the same issues.
>
>The MPI standard can state what is required of compiler implementations
>for them to be compatible with MPI.  I think a more direct solution
>would be to require that the compiler vendor supports the ASYNCHRONOUS
>semantics if it is to be used with MPI.
>
>Cheers,
>Bill
>
>
>
>-- 
>Bill Long                                           longb at cray.com
>Fortran Technical Support    &                 voice: 651-605-9024
>Bioinformatics Software Development            fax:   651-605-9142
>Cray Inc./Cray Plaza, Suite 210/380 Jackson St./St. Paul, MN 55101
>
>