[MPI3 Fortran] Argument data sizes

[Jeff Squyres]

On Sep 18, 2008, at 12:06 PM, Bill Long wrote:

>> I don't follow Cray hardware -- do you actually have machines that  
>> have more than 32GB per core?  If my math is right, 4^32 * 8 = 32GB  
>> would be the actual size of a 4B element array of doubles.
>
> Yes, but you don't really need that.  It is not uncommon to use  
> OpenMP within a local SMP domain and use MPI between such domains.   
> Local SMP domains of more than 32GB are not hard to find.  Only 16GB  
> is needed for an array with 4-byte elements.  PC's have that much  
> memory these days.  If the intention is to make a standard relevant  
> going forward, the target should be clear.

Did you read the rest of my mail?

>> Remember three key facts:
>>
>> - MPI's model is effectively distributed memory
>> - MPI's counts are *elements*, not *bytes*
>> - MPI datatypes are composable
>>
>> Most MPI apps that I have seen with huge arrays like this are  
>> actually representing multiple dimensions -- so they usually make a  
>> datatype for one row and use that as a unit to compose larger  
>> datatypes (e.g., a plane, a 3D space, ...etc.).  So most apps send  
>> "1 array" (or "N rows" or ...), not 2B (or 4B or ...) individual  
>> elements.
>>
>> This is at least part of the discussion in the Forum about this  
>> issue (I mentioned this in a prior mail as well): since MPI  
>> datatypes are composable, applications can easily smash the 2B int  
>> count limitation anyway.  As Alexander S. noted, however, there are  
>> other issues as well.

-- 
Jeff Squyres
Cisco Systems