[MPI3 Fortran] Argument data sizes

[Bill Long]

Jeff Squyres wrote:
> On Sep 18, 2008, at 12:06 PM, Bill Long wrote:
>
>>> I don't follow Cray hardware -- do you actually have machines that 
>>> have more than 32GB per core?  If my math is right, 4^32 * 8 = 32GB 
>>> would be the actual size of a 4B element array of doubles.
>>
>> Yes, but you don't really need that.  It is not uncommon to use 
>> OpenMP within a local SMP domain and use MPI between such domains.  
>> Local SMP domains of more than 32GB are not hard to find.  Only 16GB 
>> is needed for an array with 4-byte elements.  PC's have that much 
>> memory these days.  If the intention is to make a standard relevant 
>> going forward, the target should be clear.
>
> Did you read the rest of my mail?

Yes.


>
>>> Remember three key facts:
>>>
>>> - MPI's model is effectively distributed memory
>>> - MPI's counts are *elements*, not *bytes*
>>> - MPI datatypes are composable
>>>
>>> Most MPI apps that I have seen with huge arrays like this are 
>>> actually representing multiple dimensions -- so they usually make a 
>>> datatype for one row and use that as a unit to compose larger 
>>> datatypes (e.g., a plane, a 3D space, ...etc.).  So most apps send 
>>> "1 array" (or "N rows" or ...), not 2B (or 4B or ...) individual 
>>> elements.

This might occur to a C programmer, in that C does not really have 
arrays.  However, such contortions would be very unnatural for a Fortran 
programmer, and would only be considered if there were some artificial 
constraint on buffer sizes.

>>>
>>> This is at least part of the discussion in the Forum about this 
>>> issue (I mentioned this in a prior mail as well): since MPI 
>>> datatypes are composable, applications can easily smash the 2B int 
>>> count limitation anyway.  As Alexander S. noted, however, there are 
>>> other issues as well.

I'm not sure about "easily".  Certainly very unnaturally.  MPI is hard 
to use in the first place. Asking programmers to play these sorts of 
games makes it even worse.

Cheers,
Bill



-- 
Bill Long                                   longb at cray.com
Fortran Technical Support    &              voice: 651-605-9024
Bioinformatics Software Development         fax:   651-605-9142
Cray Inc., 1340 Mendota Heights Rd., Mendota Heights, MN, 55120