From david.solt at [hidden] Thu Jul 1 10:09:26 2010 From: david.solt at [hidden] (Solt, David George) Date: Thu, 1 Jul 2010 15:09:26 +0000 Subject: [Mpi3-bwcompat] MPI-3 Backwards compat meeting tomorrow In-Reply-To: Message-ID: Shall we meet tomorrow? Fab? Do you want to go over the MPI_Count status? Anyone have anything else to discuss? From ftillier at [hidden] Thu Jul 1 10:33:03 2010 From: ftillier at [hidden] (Fab Tillier) Date: Thu, 1 Jul 2010 15:33:03 +0000 Subject: [Mpi3-bwcompat] MPI-3 Backwards compat meeting tomorrow In-Reply-To: Message-ID: <91B583BE4B6FCD408F53D1B8F537E09212913982@TK5EX14MBXW602.wingroup.windeploy.ntdev.microsoft.com> Solt, David George wrote on Thu, 1 Jul 2010 at 08:09:26 > Shall we meet tomorrow? Fab? Do you want to go over the MPI_Count > status? Anyone have anything else to discuss? I don't know if it's worth having a call - you can see the MPI_Count ticket here: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/224 I'm in the process of going through the standard document looking for examples that need updating, and will add those to the ticket. If anyone has any comments about the ticket, please enter them (or you can mail here, either way is fine by me.) -Fab From ftillier at [hidden] Tue Jul 6 22:59:49 2010 From: ftillier at [hidden] (Fab Tillier) Date: Wed, 7 Jul 2010 03:59:49 +0000 Subject: [Mpi3-bwcompat] MPI-3 Backwards compat meeting tomorrow In-Reply-To: <91B583BE4B6FCD408F53D1B8F537E09212913982@TK5EX14MBXW602.wingroup.windeploy.ntdev.microsoft.com> Message-ID: <91B583BE4B6FCD408F53D1B8F537E0921291516E@TK5EX14MBXW602.wingroup.windeploy.ntdev.microsoft.com> Fab Tillier wrote on Thu, 1 Jul 2010 at 08:33:03 > I'm in the process of going through the standard document looking for > examples that need updating, and will add those to the ticket. Ok, I need a little Fortran help. :) Example 4.1.14, page 112, the calls to MPI_Type_vector passes a blocklength of 1. How do I change the example to specify that the '1' is of type MPI_COUNT_KIND? Same for the calls to MPI_TYPE_HVECTOR (though I might as well update these to use the non-deprecated function while I'm changing things.) Thanks, -Fab From ftillier at [hidden] Wed Jul 7 01:23:07 2010 From: ftillier at [hidden] (Fab Tillier) Date: Wed, 7 Jul 2010 06:23:07 +0000 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... Message-ID: <91B583BE4B6FCD408F53D1B8F537E09212915252@TK5EX14MBXW602.wingroup.windeploy.ntdev.microsoft.com> I'm going through the spec looking at examples that need updating, and came across example 4.22, which shows an interesting pattern that we may need to consider. Page 125, line 23, the call to MPI_Pack returns the postion as an output. This parameter is defined as an int in the standard. We thus can't type the position variable as an MPI_Count, otherwise taking the address and treating it like an int could leave some uninitialized bits. Then, on line 27, the buffer is sent, and position is used as the count of bytes, and in this case should be of type MPI_Count. Granted, in C the type will be promoted to MPI_Count, but there are subtle issues. I'll assert the following: 1. These two types should match. 2. The 'position' parameter to MPI_Pack and MPI_Unpack shouldn't be integers. 3. This should be extended to apply to the 'size' parameter to MPI_Type_size and MPI_Pack_size, which shouldn't be integers. This also applies to MPI_Pack for outsize, and MPI_Unpack for insize. While we could treat the second issue separately, the fact that the output parameter from MPI_Pack can be used as input to MPI_Send would suggest the two are fairly closely coupled: it doesn't make sense to have the range of the position type to MPI_Pack exceed the range of the count type to MPI_Send. The reverse could apply to MPI_Recv followed by MPI_Pack, thus MPI_Count shouldn't be greater in range than int or you could receive more than an integer's worth of MPI_PACKED data, and be unable to unpack it. Using these two parings {MPI_Pack, MPI_Send} and {MPI_Recv, MPI_Unpack} supports my first assertion above. Would anyone object if we defined MPI_Size rather than MPI_Count, and used it for count, position, and size parameters? We could then tackle the issues with MPI_Pack/Unpack/Pack_size/Type_size at the same time as counts. We can't use MPI_Aint here since we want to be able to be backward compatible, so must provide a separate type that can be defined as a standard integer. Thoughts? -Fab P.S. anyone feel like owning fixing the examples in the standard? From jsquyres at [hidden] Wed Jul 7 07:19:12 2010 From: jsquyres at [hidden] (Jeff Squyres) Date: Wed, 7 Jul 2010 08:19:12 -0400 Subject: [Mpi3-bwcompat] MPI-3 Backwards compat meeting tomorrow In-Reply-To: <91B583BE4B6FCD408F53D1B8F537E0921291516E@TK5EX14MBXW602.wingroup.windeploy.ntdev.microsoft.com> Message-ID: <0DD20A5B-169F-4A1F-9396-330B3CE2F4E1@cisco.com> On Jul 6, 2010, at 11:59 PM, Fab Tillier wrote: > Ok, I need a little Fortran help. :) Don't we all? :-) > Example 4.1.14, page 112, the calls to MPI_Type_vector passes a blocklength of 1. How do I change the example to specify that the '1' is of type MPI_COUNT_KIND? Do you mean example 4.1.13? I don't see the use of MPI_Type_vector in 4.1.14... The easiest way to pass a "1" of MPI_COUNT_KIND through as a parameter is to declare a new variable of that type and then pass that. I know there's a way to cast an integer to that type in the call to MPI_TYPE_VECTOR itself (i.e., without making a new variable), but I don't remember the exact syntax. INTEGER (kind = MPI_COUNT_KIND) :: count count = 1 ... call MPI_TYPE_VECTOR(9, count, 2, MPI_REAL, oneslice, ierr) Make sense? -- Jeff Squyres jsquyres_at_[hidden] For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ From jsquyres at [hidden] Wed Jul 7 07:23:43 2010 From: jsquyres at [hidden] (Jeff Squyres) Date: Wed, 7 Jul 2010 08:23:43 -0400 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: <91B583BE4B6FCD408F53D1B8F537E09212915252@TK5EX14MBXW602.wingroup.windeploy.ntdev.microsoft.com> Message-ID: <8E8D9965-0451-4F3C-A659-70E9E1DAB184@cisco.com> On Jul 7, 2010, at 2:23 AM, Fab Tillier wrote: > Would anyone object if we defined MPI_Size rather than MPI_Count, and used it for count, position, and size parameters? We could then tackle the issues with MPI_Pack/Unpack/Pack_size/Type_size at the same time as counts. We can't use MPI_Aint here since we want to be able to be backward compatible, so must provide a separate type that can be defined as a standard integer. I agree that tackling them all at the same time is the Right Thing to do. I also agree with all your other points. I still am a little uncomfortable with the name "MPI_Size" because sizes and counts are theoretically different things. But then again, I can't think of a better name offhand... -- Jeff Squyres jsquyres_at_[hidden] For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ From thakur at [hidden] Wed Jul 7 08:16:59 2010 From: thakur at [hidden] (Rajeev Thakur) Date: Wed, 7 Jul 2010 08:16:59 -0500 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: <8E8D9965-0451-4F3C-A659-70E9E1DAB184@cisco.com> Message-ID: <2E70B4685F0C43578540FD363B37764D@thakurlaptop> Can we define both (to be the same) since we are familiar with count for messages (it is a count of datatypes) and size for others. Rajeev > -----Original Message----- > From: mpi3-bwcompat-bounces_at_[hidden] > [mailto:mpi3-bwcompat-bounces_at_[hidden]] On Behalf Of Jeff Squyres > Sent: Wednesday, July 07, 2010 7:24 AM > To: MPI-3 backwards compatability WG > Subject: Re: [Mpi3-bwcompat] MPI_Count mixing with int... > > On Jul 7, 2010, at 2:23 AM, Fab Tillier wrote: > > > Would anyone object if we defined MPI_Size rather than > MPI_Count, and used it for count, position, and size > parameters? We could then tackle the issues with > MPI_Pack/Unpack/Pack_size/Type_size at the same time as > counts. We can't use MPI_Aint here since we want to be able > to be backward compatible, so must provide a separate type > that can be defined as a standard integer. > > I agree that tackling them all at the same time is the Right > Thing to do. I also agree with all your other points. > > I still am a little uncomfortable with the name "MPI_Size" > because sizes and counts are theoretically different things. > But then again, I can't think of a better name offhand... > > -- > Jeff Squyres > jsquyres_at_[hidden] > For corporate legal information go to: > http://www.cisco.com/web/about/doing_business/legal/cri/ > > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat > From koziol at [hidden] Thu Jul 8 08:44:37 2010 From: koziol at [hidden] (Quincey Koziol) Date: Thu, 8 Jul 2010 08:44:37 -0500 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: <2E70B4685F0C43578540FD363B37764D@thakurlaptop> Message-ID: <4A260B86-6166-4827-854D-EFD79CAB653E@hdfgroup.org> On Jul 7, 2010, at 8:16 AM, Rajeev Thakur wrote: > Can we define both (to be the same) since we are familiar with count for > messages (it is a count of datatypes) and size for others. I agree - we shouldn't conflate counts and sizes, they are totally different things. Defining them both makes sense to me. Quincey > Rajeev > >> -----Original Message----- >> From: mpi3-bwcompat-bounces_at_[hidden] >> [mailto:mpi3-bwcompat-bounces_at_[hidden]] On Behalf Of Jeff Squyres >> Sent: Wednesday, July 07, 2010 7:24 AM >> To: MPI-3 backwards compatability WG >> Subject: Re: [Mpi3-bwcompat] MPI_Count mixing with int... >> >> On Jul 7, 2010, at 2:23 AM, Fab Tillier wrote: >> >>> Would anyone object if we defined MPI_Size rather than >> MPI_Count, and used it for count, position, and size >> parameters? We could then tackle the issues with >> MPI_Pack/Unpack/Pack_size/Type_size at the same time as >> counts. We can't use MPI_Aint here since we want to be able >> to be backward compatible, so must provide a separate type >> that can be defined as a standard integer. >> >> I agree that tackling them all at the same time is the Right >> Thing to do. I also agree with all your other points. >> >> I still am a little uncomfortable with the name "MPI_Size" >> because sizes and counts are theoretically different things. >> But then again, I can't think of a better name offhand... >> >> -- >> Jeff Squyres >> jsquyres_at_[hidden] >> For corporate legal information go to: >> http://www.cisco.com/web/about/doing_business/legal/cri/ >> >> >> _______________________________________________ >> Mpi3-bwcompat mailing list >> Mpi3-bwcompat_at_[hidden] >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat >> > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat From ftillier at [hidden] Thu Jul 8 11:26:40 2010 From: ftillier at [hidden] (Fab Tillier) Date: Thu, 8 Jul 2010 16:26:40 +0000 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: <4A260B86-6166-4827-854D-EFD79CAB653E@hdfgroup.org> Message-ID: <91B583BE4B6FCD408F53D1B8F537E0921292029E@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Quincey Koziol wrote on Thu, 8 Jul 2010 at 06:44:37 > > On Jul 7, 2010, at 8:16 AM, Rajeev Thakur wrote: > >> Can we define both (to be the same) since we are familiar with count >> for messages (it is a count of datatypes) and size for others. > > I agree - we shouldn't conflate counts and sizes, they are > totally different things. Defining them both makes sense to me. But they are the same for the MPI_PACKED datatype, so really shouldn't be different. As an aside, can count ever be larger than size_t? -Fab >>> -----Original Message----- From: mpi3-bwcompat-bounces_at_[hidden] >>> [mailto:mpi3-bwcompat-bounces_at_[hidden]] On Behalf Of Jeff Squyres >>> Sent: Wednesday, July 07, 2010 7:24 AM To: MPI-3 backwards >>> compatability WG Subject: Re: [Mpi3-bwcompat] MPI_Count mixing with >>> int... >>> >>> On Jul 7, 2010, at 2:23 AM, Fab Tillier wrote: >>> >>>> Would anyone object if we defined MPI_Size rather than >>> MPI_Count, and used it for count, position, and size >>> parameters? We could then tackle the issues with >>> MPI_Pack/Unpack/Pack_size/Type_size at the same time as >>> counts. We can't use MPI_Aint here since we want to be able >>> to be backward compatible, so must provide a separate type >>> that can be defined as a standard integer. >>> >>> I agree that tackling them all at the same time is the Right >>> Thing to do. I also agree with all your other points. >>> >>> I still am a little uncomfortable with the name "MPI_Size" >>> because sizes and counts are theoretically different things. >>> But then again, I can't think of a better name offhand... >>> >>> -- >>> Jeff Squyres >>> jsquyres_at_[hidden] >>> For corporate legal information go to: >>> http://www.cisco.com/web/about/doing_business/legal/cri/ >>> >>> >>> _______________________________________________ >>> Mpi3-bwcompat mailing list >>> Mpi3-bwcompat_at_[hidden] >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat >>> >> >> _______________________________________________ >> Mpi3-bwcompat mailing list >> Mpi3-bwcompat_at_[hidden] >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat > > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat From jsquyres at [hidden] Thu Jul 8 12:29:37 2010 From: jsquyres at [hidden] (Jeff Squyres) Date: Thu, 8 Jul 2010 13:29:37 -0400 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: <91B583BE4B6FCD408F53D1B8F537E0921292029E@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Message-ID: On Jul 8, 2010, at 12:26 PM, Fab Tillier wrote: > > I agree - we shouldn't conflate counts and sizes, they are > > totally different things. Defining them both makes sense to me. > > But they are the same for the MPI_PACKED datatype, so really shouldn't be different. ...and let's not forget that MPI_Aint's are used for displacements in MPI_GEt, MPI_PUT, and MPI_ACCUMULATE. :-\ Should we disentangle those, too? > As an aside, can count ever be larger than size_t? I don't think so, but why do you ask? -- Jeff Squyres jsquyres_at_[hidden] For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ From koziol at [hidden] Thu Jul 8 12:32:11 2010 From: koziol at [hidden] (Quincey Koziol) Date: Thu, 8 Jul 2010 12:32:11 -0500 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: <91B583BE4B6FCD408F53D1B8F537E0921292029E@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Message-ID: On Jul 8, 2010, at 11:26 AM, Fab Tillier wrote: > Quincey Koziol wrote on Thu, 8 Jul 2010 at 06:44:37 > >> >> On Jul 7, 2010, at 8:16 AM, Rajeev Thakur wrote: >> >>> Can we define both (to be the same) since we are familiar with count >>> for messages (it is a count of datatypes) and size for others. >> >> I agree - we shouldn't conflate counts and sizes, they are >> totally different things. Defining them both makes sense to me. > > But they are the same for the MPI_PACKED datatype, so really shouldn't be different. I'm not looking at the standard document right now, but that just strikes me as the standard being vague about the difference between the two. > As an aside, can count ever be larger than size_t? I would imagine so - on a 32-bit CPU/system with a 64-bit file system. Quincey > -Fab > >>>> -----Original Message----- From: mpi3-bwcompat-bounces_at_[hidden] >>>> [mailto:mpi3-bwcompat-bounces_at_[hidden]] On Behalf Of Jeff Squyres >>>> Sent: Wednesday, July 07, 2010 7:24 AM To: MPI-3 backwards >>>> compatability WG Subject: Re: [Mpi3-bwcompat] MPI_Count mixing with >>>> int... >>>> >>>> On Jul 7, 2010, at 2:23 AM, Fab Tillier wrote: >>>> >>>>> Would anyone object if we defined MPI_Size rather than >>>> MPI_Count, and used it for count, position, and size >>>> parameters? We could then tackle the issues with >>>> MPI_Pack/Unpack/Pack_size/Type_size at the same time as >>>> counts. We can't use MPI_Aint here since we want to be able >>>> to be backward compatible, so must provide a separate type >>>> that can be defined as a standard integer. >>>> >>>> I agree that tackling them all at the same time is the Right >>>> Thing to do. I also agree with all your other points. >>>> >>>> I still am a little uncomfortable with the name "MPI_Size" >>>> because sizes and counts are theoretically different things. >>>> But then again, I can't think of a better name offhand... >>>> >>>> -- >>>> Jeff Squyres >>>> jsquyres_at_[hidden] >>>> For corporate legal information go to: >>>> http://www.cisco.com/web/about/doing_business/legal/cri/ >>>> >>>> >>>> _______________________________________________ >>>> Mpi3-bwcompat mailing list >>>> Mpi3-bwcompat_at_[hidden] >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat >>>> >>> >>> _______________________________________________ >>> Mpi3-bwcompat mailing list >>> Mpi3-bwcompat_at_[hidden] >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat >> >> >> _______________________________________________ >> Mpi3-bwcompat mailing list >> Mpi3-bwcompat_at_[hidden] >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat From jsquyres at [hidden] Thu Jul 8 12:34:46 2010 From: jsquyres at [hidden] (Jeff Squyres) Date: Thu, 8 Jul 2010 13:34:46 -0400 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: Message-ID: On Jul 8, 2010, at 1:32 PM, Quincey Koziol wrote: > > As an aside, can count ever be larger than size_t? > > I would imagine so - on a 32-bit CPU/system with a 64-bit file system. Oh ya, that whole filesystem thing. :-) But still -- why do you ask? I am hoping you don't plan to mention "size_t" in the standard at all... -- Jeff Squyres jsquyres_at_[hidden] For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ From thakur at [hidden] Thu Jul 8 14:52:58 2010 From: thakur at [hidden] (Rajeev Thakur) Date: Thu, 8 Jul 2010 14:52:58 -0500 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: <91B583BE4B6FCD408F53D1B8F537E0921292029E@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Message-ID: <25C938D7FABF4DB097902C3C5F92C263@thakurlaptop> > As an aside, can count ever be larger than size_t? Probably not in the communication or I/O functions since count refers to the number of items in the memory buffer, but it could be for the datatype constructor functions since datatypes can represent file layout. Rajeev From ftillier at [hidden] Thu Jul 8 15:30:30 2010 From: ftillier at [hidden] (Fab Tillier) Date: Thu, 8 Jul 2010 20:30:30 +0000 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: Message-ID: <91B583BE4B6FCD408F53D1B8F537E09212923EE7@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Jeff Squyres wrote on Thu, 8 Jul 2010 at 10:34:46 > On Jul 8, 2010, at 1:32 PM, Quincey Koziol wrote: > >>> As an aside, can count ever be larger than size_t? >> >> I would imagine so - on a 32-bit CPU/system with a 64-bit > file system. > > Oh ya, that whole filesystem thing. :-) But even on a 32-bit system, you can't do more than a 32-bit file access at a time. Even if you created a datatype that represented more than 2^31 bytes, you wouldn't be able to use it in any of the functions because you wouldn't be able to transfer that much data at once. > But still -- why do you ask? I am hoping you don't plan to mention > "size_t" in the standard at all... I ask because MPI_Size (for MPI_Type_size, MPI_Pack_size, and the position parameters of MPI_Pack/Unpack) would likely be size_t - how could you use more than the addressable space? I also think that MPI_Count would reasonably be size_t also, since I pretty firmly believe that MPI_Count and MPI_Size should be the same type. -Fab From ftillier at [hidden] Thu Jul 8 15:31:56 2010 From: ftillier at [hidden] (Fab Tillier) Date: Thu, 8 Jul 2010 20:31:56 +0000 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: <25C938D7FABF4DB097902C3C5F92C263@thakurlaptop> Message-ID: <91B583BE4B6FCD408F53D1B8F537E09212923EF7@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Rajeev Thakur wrote on Thu, 8 Jul 2010 at 12:52:58 >> As an aside, can count ever be larger than size_t? > > Probably not in the communication or I/O functions since count refers to > the number of items in the memory buffer, but it could be for the > datatype constructor functions since datatypes can represent file layout. How would you use a datatype that is larger than your address space? You can't read or write it, you can't send or receive it, you can't pack/unpack it... Am I missing something and a datatype like this would be useful? -Fab From ftillier at [hidden] Thu Jul 8 15:35:27 2010 From: ftillier at [hidden] (Fab Tillier) Date: Thu, 8 Jul 2010 20:35:27 +0000 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: Message-ID: <91B583BE4B6FCD408F53D1B8F537E09212923F11@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Jeff Squyres wrote on Thu, 8 Jul 2010 at 10:29:37 > On Jul 8, 2010, at 12:26 PM, Fab Tillier wrote: > >>> I agree - we shouldn't conflate counts and sizes, they are >>> totally different things. Defining them both makes sense to me. >> >> But they are the same for the MPI_PACKED datatype, so really >> shouldn't be different. > > ...and let's not forget that MPI_Aint's are used for displacements in > MPI_GEt, MPI_PUT, and MPI_ACCUMULATE. :-\ Should we disentangle > those, too? No, because MPI_Aint is as close as the standard comes to size_t, which is the right type. I would anticipate MPI implementations defining MPI_Size/MPI_Count as either int for back compat, or MPI_Aint. Though I wouldn't be surprised if some platform has some weird MPI_Aint limitation, but in my cosy little world of Windows, MPI_Aint is really size_t. >> As an aside, can count ever be larger than size_t? > > I don't think so, but why do you ask? Because if count could be larger than size_t, but MPI_Size cannot, you have a mismatch that we need to make sure we understand. In this context, MPI_Size is the type used for functions that should have used MPI_Aint but didn't for historical reasons such as MPI_Pack/Unpack/Type_size/Pack_size. -Fab From david.solt at [hidden] Thu Jul 8 15:41:09 2010 From: david.solt at [hidden] (Solt, David George) Date: Thu, 8 Jul 2010 20:41:09 +0000 Subject: [Mpi3-bwcompat] Meeting schedule In-Reply-To: <91B583BE4B6FCD408F53D1B8F537E09212923EF7@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Message-ID: Our reoccurring backwards compatibility meeting expired. It was set for every other Friday at 10am EST. I can just fire it up again, but want to check in and see if we want to keep that time. If you would attend if the meeting was at a different time or if you attend regularly but would prefer a different time, please shoot me an e-mail. If I don't hear from many people I will fire it up again starting next week. Dave From ftillier at [hidden] Thu Jul 8 15:51:04 2010 From: ftillier at [hidden] (Fab Tillier) Date: Thu, 8 Jul 2010 20:51:04 +0000 Subject: [Mpi3-bwcompat] Meeting schedule In-Reply-To: Message-ID: <91B583BE4B6FCD408F53D1B8F537E09212923F90@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> I would much prefer a later meeting time, early mornings are not good for me given morning activities of getting the kids out the door and off to their various activities. Something like 9AM PST or later. :) -Fab Solt, David George wrote on Thu, 8 Jul 2010 at 13:41:09 > Our reoccurring backwards compatibility meeting expired. It was set for > every other Friday at 10am EST. I can just fire it up again, but want > to check in and see if we want to keep that time. If you would attend > if the meeting was at a different time or if you attend regularly but > would prefer a different time, please shoot me an e-mail. If I don't > hear from many people I will fire it up again starting next week. > > Dave > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat From thakur at [hidden] Thu Jul 8 20:24:16 2010 From: thakur at [hidden] (Rajeev Thakur) Date: Thu, 8 Jul 2010 20:24:16 -0500 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: <91B583BE4B6FCD408F53D1B8F537E09212923EF7@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Message-ID: <19E95A664AB44146ADCBEF43132DB027@thakurlaptop> You can define a filetype that is many Gbytes and read only a small part of it. The filetype is specified in MPI_File_set_view and is not passed in the read/write calls. The datatype passed in the read/write calls refers to the buffer passed to that call. Rajeev > -----Original Message----- > From: mpi3-bwcompat-bounces_at_[hidden] > [mailto:mpi3-bwcompat-bounces_at_[hidden]] On Behalf Of Fab Tillier > Sent: Thursday, July 08, 2010 3:32 PM > To: MPI-3 backwards compatability WG > Subject: Re: [Mpi3-bwcompat] MPI_Count mixing with int... > > Rajeev Thakur wrote on Thu, 8 Jul 2010 at 12:52:58 > > >> As an aside, can count ever be larger than size_t? > > > > Probably not in the communication or I/O functions since > count refers > > to the number of items in the memory buffer, but it could > be for the > > datatype constructor functions since datatypes can > represent file layout. > > How would you use a datatype that is larger than your address > space? You can't read or write it, you can't send or receive > it, you can't pack/unpack it... > > Am I missing something and a datatype like this would be useful? > > -Fab > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat > From ftillier at [hidden] Thu Jul 8 23:20:01 2010 From: ftillier at [hidden] (Fab Tillier) Date: Fri, 9 Jul 2010 04:20:01 +0000 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: <19E95A664AB44146ADCBEF43132DB027@thakurlaptop> Message-ID: <91B583BE4B6FCD408F53D1B8F537E09212924221@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Rajeev Thakur wrote on Thu, 8 Jul 2010 at 18:24:16 > You can define a filetype that is many Gbytes and read only a small part > of it. The filetype is specified in MPI_File_set_view and is not passed > in the read/write calls. The datatype passed in the read/write calls > refers to the buffer passed to that call. Perfect, thanks for the example! Does this imply that MPI_Type_get_extent, which returns an MPI_Aint, might overflow the extent parameter on a 32-bit platform on which a user defines a many Gbyte filetype? How should we address this (there are probably other APIs affected similarly)? -Fab >> -----Original Message----- >> From: Fab Tillier >> Sent: Thursday, July 08, 2010 3:32 PM >> >> Rajeev Thakur wrote on Thu, 8 Jul 2010 at 12:52:58 >> >>>> As an aside, can count ever be larger than size_t? >>> >>> Probably not in the communication or I/O functions since count refers >>> to the number of items in the memory buffer, but it could be for the >>> datatype constructor functions since datatypes can >> represent file layout. >> >> How would you use a datatype that is larger than your address >> space? You can't read or write it, you can't send or receive >> it, you can't pack/unpack it... >> >> Am I missing something and a datatype like this would be useful? >> >> -Fab From thakur at [hidden] Fri Jul 9 00:06:24 2010 From: thakur at [hidden] (Rajeev Thakur) Date: Fri, 9 Jul 2010 00:06:24 -0500 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: <91B583BE4B6FCD408F53D1B8F537E09212924221@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Message-ID: <26DC428476F84C46BF8C41EF5AE8576D@thakurlaptop> Yes, that is a long standing problem -- that datatypes use MPI_Aint (wherever they do), but they really need to use the larger of MPI_Aint or MPI_Offset. Rajeev > -----Original Message----- > From: mpi3-bwcompat-bounces_at_[hidden] > [mailto:mpi3-bwcompat-bounces_at_[hidden]] On Behalf Of Fab Tillier > Sent: Thursday, July 08, 2010 11:20 PM > To: MPI-3 backwards compatability WG > Subject: Re: [Mpi3-bwcompat] MPI_Count mixing with int... > > Rajeev Thakur wrote on Thu, 8 Jul 2010 at 18:24:16 > > > You can define a filetype that is many Gbytes and read only a small > > part of it. The filetype is specified in MPI_File_set_view > and is not > > passed in the read/write calls. The datatype passed in the > read/write > > calls refers to the buffer passed to that call. > > Perfect, thanks for the example! Does this imply that > MPI_Type_get_extent, which returns an MPI_Aint, might > overflow the extent parameter on a 32-bit platform on which a > user defines a many Gbyte filetype? How should we address > this (there are probably other APIs affected similarly)? > > -Fab > > >> -----Original Message----- > >> From: Fab Tillier > >> Sent: Thursday, July 08, 2010 3:32 PM > >> > >> Rajeev Thakur wrote on Thu, 8 Jul 2010 at 12:52:58 > >> > >>>> As an aside, can count ever be larger than size_t? > >>> > >>> Probably not in the communication or I/O functions since count > >>> refers to the number of items in the memory buffer, but > it could be > >>> for the datatype constructor functions since datatypes can > >> represent file layout. > >> > >> How would you use a datatype that is larger than your > address space? > >> You can't read or write it, you can't send or receive it, > you can't > >> pack/unpack it... > >> > >> Am I missing something and a datatype like this would be useful? > >> > >> -Fab > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat > From ftillier at [hidden] Fri Jul 9 00:26:27 2010 From: ftillier at [hidden] (Fab Tillier) Date: Fri, 9 Jul 2010 05:26:27 +0000 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: <26DC428476F84C46BF8C41EF5AE8576D@thakurlaptop> Message-ID: <91B583BE4B6FCD408F53D1B8F537E092129242EE@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Rajeev Thakur wrote on Thu, 8 Jul 2010 at 22:06:24 > Yes, that is a long standing problem -- that datatypes use MPI_Aint > (wherever they do), but they really need to use the larger of MPI_Aint > or MPI_Offset. So an 'optimal' implementation would be one where MPI_Aint, MPI_Offset, MPI_Count, and MPI_Size all map to the same underlying type. Ugh, this is getting messy quickly. I think I'll just deal with the MPI_Count and MPI_Size issues, ignoring the MPI_Aint/MPI_Offset issues. Some poor soul a decade from now will likely curse me for it, but I don't want to get into deprecating a bunch of functions (those with Aint) to normalize them too. So to summarize, my plan is as follows: 1. Define MPI_Size (instead of MPI_Count) 2. Use MPI_Size as the type for all size, count, and position variables. 3. Ignore MPI_Aint and MPI_Offset inconsistencies. If anyone has issues with this, please holler sooner than later. Thanks, -Fab > >> -----Original Message----- >> From: Fab Tillier >> Sent: Thursday, July 08, 2010 11:20 PM >> >> Rajeev Thakur wrote on Thu, 8 Jul 2010 at 18:24:16 >> >>> You can define a filetype that is many Gbytes and read only a small >>> part of it. The filetype is specified in MPI_File_set_view and is not >>> passed in the read/write calls. The datatype passed in the read/write >>> calls refers to the buffer passed to that call. >> >> Perfect, thanks for the example! Does this imply that >> MPI_Type_get_extent, which returns an MPI_Aint, might >> overflow the extent parameter on a 32-bit platform on which a >> user defines a many Gbyte filetype? How should we address >> this (there are probably other APIs affected similarly)? >> >> -Fab >> >>>> -----Original Message----- >>>> From: Fab Tillier >>>> Sent: Thursday, July 08, 2010 3:32 PM >>>> >>>> Rajeev Thakur wrote on Thu, 8 Jul 2010 at 12:52:58 >>>> >>>>>> As an aside, can count ever be larger than size_t? >>>>> >>>>> Probably not in the communication or I/O functions since count >>>>> refers to the number of items in the memory buffer, but it could be >>>>> for the datatype constructor functions since datatypes can >>>> represent file layout. >>>> >>>> How would you use a datatype that is larger than your address space? >>>> You can't read or write it, you can't send or receive it, you can't >>>> pack/unpack it... >>>> >>>> Am I missing something and a datatype like this would be useful? >>>> >>>> -Fab From koziol at [hidden] Fri Jul 9 14:05:52 2010 From: koziol at [hidden] (Quincey Koziol) Date: Fri, 9 Jul 2010 14:05:52 -0500 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: <91B583BE4B6FCD408F53D1B8F537E092129242EE@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Message-ID: <1D2848A8-2B83-4DD2-B4E1-C7623C2F88C8@hdfgroup.org> Hi Fab, On Jul 9, 2010, at 12:26 AM, Fab Tillier wrote: > Rajeev Thakur wrote on Thu, 8 Jul 2010 at 22:06:24 > >> Yes, that is a long standing problem -- that datatypes use MPI_Aint >> (wherever they do), but they really need to use the larger of MPI_Aint >> or MPI_Offset. > > So an 'optimal' implementation would be one where MPI_Aint, MPI_Offset, MPI_Count, and MPI_Size all map to the same underlying type. > > Ugh, this is getting messy quickly. I think I'll just deal with the MPI_Count and MPI_Size issues, ignoring the MPI_Aint/MPI_Offset issues. Some poor soul a decade from now will likely curse me for it, but I don't want to get into deprecating a bunch of functions (those with Aint) to normalize them too. > > So to summarize, my plan is as follows: > 1. Define MPI_Size (instead of MPI_Count) > 2. Use MPI_Size as the type for all size, count, and position variables. > 3. Ignore MPI_Aint and MPI_Offset inconsistencies. > > If anyone has issues with this, please holler sooner than later. I still think aliasing MPI_Size and MPI_Count will cause problems down the line, since the two concepts are different. I'd be much happier with separating them now rather than having to change things that are switched from int to MPI_Size now to MPI_Count later. Quincey > Thanks, > -Fab > >> >>> -----Original Message----- >>> From: Fab Tillier >>> Sent: Thursday, July 08, 2010 11:20 PM >>> >>> Rajeev Thakur wrote on Thu, 8 Jul 2010 at 18:24:16 >>> >>>> You can define a filetype that is many Gbytes and read only a small >>>> part of it. The filetype is specified in MPI_File_set_view and is not >>>> passed in the read/write calls. The datatype passed in the read/write >>>> calls refers to the buffer passed to that call. >>> >>> Perfect, thanks for the example! Does this imply that >>> MPI_Type_get_extent, which returns an MPI_Aint, might >>> overflow the extent parameter on a 32-bit platform on which a >>> user defines a many Gbyte filetype? How should we address >>> this (there are probably other APIs affected similarly)? >>> >>> -Fab >>> >>>>> -----Original Message----- >>>>> From: Fab Tillier >>>>> Sent: Thursday, July 08, 2010 3:32 PM >>>>> >>>>> Rajeev Thakur wrote on Thu, 8 Jul 2010 at 12:52:58 >>>>> >>>>>>> As an aside, can count ever be larger than size_t? >>>>>> >>>>>> Probably not in the communication or I/O functions since count >>>>>> refers to the number of items in the memory buffer, but it could be >>>>>> for the datatype constructor functions since datatypes can >>>>> represent file layout. >>>>> >>>>> How would you use a datatype that is larger than your address space? >>>>> You can't read or write it, you can't send or receive it, you can't >>>>> pack/unpack it... >>>>> >>>>> Am I missing something and a datatype like this would be useful? >>>>> >>>>> -Fab > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat From ftillier at [hidden] Fri Jul 9 14:22:11 2010 From: ftillier at [hidden] (Fab Tillier) Date: Fri, 9 Jul 2010 19:22:11 +0000 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: <1D2848A8-2B83-4DD2-B4E1-C7623C2F88C8@hdfgroup.org> Message-ID: <91B583BE4B6FCD408F53D1B8F537E092129244ED@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Quincey Koziol wrote on Fri, 9 Jul 2010 at 12:05:52 > I still think aliasing MPI_Size and MPI_Count will cause problems > down the line, since the two concepts are different. I'd be much > happier with separating them now rather than having to change things > that are switched from int to MPI_Size now to MPI_Count later. Having MPI_Size and MPI_Count different types causes MPI_Pack/MPI_Send and MPI_Recv/MPI_Unpack to have problems, since the count and sizes are passed between the two functions and should be interchangeable, though. Do we not care, and assume the compiler will flag assignments that might result in truncation? -Fab From jsquyres at [hidden] Wed Jul 14 15:23:45 2010 From: jsquyres at [hidden] (Jeff Squyres) Date: Wed, 14 Jul 2010 16:23:45 -0400 Subject: [Mpi3-bwcompat] Meeting schedule In-Reply-To: Message-ID: <16A842B5-FCD0-4322-ABF3-8FBBE4B7DCCD@cisco.com> Did you pick a new time based on Fab's input? I can setup webex's, if it's useful. On Jul 8, 2010, at 4:41 PM, Solt, David George wrote: > Our reoccurring backwards compatibility meeting expired. It was set for every other Friday at 10am EST. I can just fire it up again, but want to check in and see if we want to keep that time. If you would attend if the meeting was at a different time or if you attend regularly but would prefer a different time, please shoot me an e-mail. If I don't hear from many people I will fire it up again starting next week. > > Dave > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat > -- Jeff Squyres jsquyres_at_[hidden] For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ From david.solt at [hidden] Wed Jul 14 15:33:03 2010 From: david.solt at [hidden] (Solt, David George) Date: Wed, 14 Jul 2010 20:33:03 +0000 Subject: [Mpi3-bwcompat] Meeting schedule In-Reply-To: Message-ID: Thanks for the overwhelming input :). Anyone have a problem moving it to 10PST/1EST every other Friday? Current time is hard for our West Coast participant(s). Dave -----Original Message----- From: mpi3-bwcompat-bounces_at_[hidden] [mailto:mpi3-bwcompat-bounces_at_[hidden]] On Behalf Of Solt, David George Sent: Thursday, July 08, 2010 3:41 PM To: MPI-3 backwards compatability WG Subject: [Mpi3-bwcompat] Meeting schedule Our reoccurring backwards compatibility meeting expired. It was set for every other Friday at 10am EST. I can just fire it up again, but want to check in and see if we want to keep that time. If you would attend if the meeting was at a different time or if you attend regularly but would prefer a different time, please shoot me an e-mail. If I don't hear from many people I will fire it up again starting next week. Dave _______________________________________________ Mpi3-bwcompat mailing list Mpi3-bwcompat_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat From jsquyres at [hidden] Wed Jul 14 15:40:17 2010 From: jsquyres at [hidden] (Jeff Squyres) Date: Wed, 14 Jul 2010 16:40:17 -0400 Subject: [Mpi3-bwcompat] Meeting schedule In-Reply-To: Message-ID: Works for me. On Jul 14, 2010, at 4:33 PM, Solt, David George wrote: > Thanks for the overwhelming input :). Anyone have a problem moving it to 10PST/1EST every other Friday? Current time is hard for our West Coast participant(s). > Dave > > -----Original Message----- > From: mpi3-bwcompat-bounces_at_[hidden] [mailto:mpi3-bwcompat-bounces_at_[hidden]] On Behalf Of Solt, David George > Sent: Thursday, July 08, 2010 3:41 PM > To: MPI-3 backwards compatability WG > Subject: [Mpi3-bwcompat] Meeting schedule > > Our reoccurring backwards compatibility meeting expired. It was set for every other Friday at 10am EST. I can just fire it up again, but want to check in and see if we want to keep that time. If you would attend if the meeting was at a different time or if you attend regularly but would prefer a different time, please shoot me an e-mail. If I don't hear from many people I will fire it up again starting next week. > > Dave > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat > -- Jeff Squyres jsquyres_at_[hidden] For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ From ftillier at [hidden] Wed Jul 14 16:53:46 2010 From: ftillier at [hidden] (Fab Tillier) Date: Wed, 14 Jul 2010 21:53:46 +0000 Subject: [Mpi3-bwcompat] Meeting schedule In-Reply-To: Message-ID: <91B583BE4B6FCD408F53D1B8F537E0921292AE9B@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Works for me, thanks for accommodating me. -Fab Solt, David George wrote on Wed, 14 Jul 2010 at 13:33:03 > Thanks for the overwhelming input :). Anyone have a problem moving it > to 10PST/1EST every other Friday? Current time is hard for our West > Coast participant(s). > Dave > > -----Original Message----- > From: mpi3-bwcompat-bounces_at_[hidden] [mailto:mpi3-bwcompat- > bounces_at_[hidden]] On Behalf Of Solt, David George > Sent: Thursday, July 08, 2010 3:41 PM > To: MPI-3 backwards compatability WG > Subject: [Mpi3-bwcompat] Meeting schedule > > Our reoccurring backwards compatibility meeting expired. It was set for > every other Friday at 10am EST. I can just fire it up again, but want > to check in and see if we want to keep that time. If you would attend > if the meeting was at a different time or if you attend regularly but > would prefer a different time, please shoot me an e-mail. If I don't > hear from many people I will fire it up again starting next week. > > Dave > > _______________________________________________ Mpi3-bwcompat mailing > list Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat From koziol at [hidden] Thu Jul 15 02:42:06 2010 From: koziol at [hidden] (Quincey Koziol) Date: Thu, 15 Jul 2010 02:42:06 -0500 Subject: [Mpi3-bwcompat] Meeting schedule In-Reply-To: Message-ID: <2BA88F7F-88DD-4A30-8D32-6DB0739EC2BC@hdfgroup.org> On Jul 14, 2010, at 3:33 PM, Solt, David George wrote: > Thanks for the overwhelming input :). Anyone have a problem moving it to 10PST/1EST every other Friday? Current time is hard for our West Coast participant(s). Works for me. (Although y'all might have to listen to me eat ;-) Quincey > Dave > > -----Original Message----- > From: mpi3-bwcompat-bounces_at_[hidden] [mailto:mpi3-bwcompat-bounces_at_[hidden]] On Behalf Of Solt, David George > Sent: Thursday, July 08, 2010 3:41 PM > To: MPI-3 backwards compatability WG > Subject: [Mpi3-bwcompat] Meeting schedule > > Our reoccurring backwards compatibility meeting expired. It was set for every other Friday at 10am EST. I can just fire it up again, but want to check in and see if we want to keep that time. If you would attend if the meeting was at a different time or if you attend regularly but would prefer a different time, please shoot me an e-mail. If I don't hear from many people I will fire it up again starting next week. > > Dave > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat From koziol at [hidden] Thu Jul 15 02:50:14 2010 From: koziol at [hidden] (Quincey Koziol) Date: Thu, 15 Jul 2010 02:50:14 -0500 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: <91B583BE4B6FCD408F53D1B8F537E092129244ED@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Message-ID: <946A45B2-1324-4EF0-AF9A-15469BE4AFD6@hdfgroup.org> Hi Fab, Sorry for the delay in replying... On Jul 9, 2010, at 2:22 PM, Fab Tillier wrote: > Quincey Koziol wrote on Fri, 9 Jul 2010 at 12:05:52 > >> I still think aliasing MPI_Size and MPI_Count will cause problems >> down the line, since the two concepts are different. I'd be much >> happier with separating them now rather than having to change things >> that are switched from int to MPI_Size now to MPI_Count later. > > Having MPI_Size and MPI_Count different types causes MPI_Pack/MPI_Send and MPI_Recv/MPI_Unpack to have problems, since the count and sizes are passed between the two functions and should be interchangeable, though. Well, there probably is something wrong with the definition of MPI_Pack and MPI_Unpack then. OK, I've gone off and re-read the definition of MPI_Pack in the standard and it looks OK to me. What is the problem with using MPI_Count for the 'incount' parameter and MPI_Size for the 'outsize' parameter? (I think I would suggest changing 'inposition' from a 'int *' to a 'void *' while we're mucking about in here, also) Quincey > Do we not care, and assume the compiler will flag assignments that might result in truncation? > > -Fab > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat From ftillier at [hidden] Thu Jul 15 11:36:41 2010 From: ftillier at [hidden] (Fab Tillier) Date: Thu, 15 Jul 2010 16:36:41 +0000 Subject: [Mpi3-bwcompat] MPI_Count mixing with int... In-Reply-To: <946A45B2-1324-4EF0-AF9A-15469BE4AFD6@hdfgroup.org> Message-ID: <91B583BE4B6FCD408F53D1B8F537E0921292B367@TK5EX14MBXW601.wingroup.windeploy.ntdev.microsoft.com> Quincey Koziol wrote on Thu, 15 Jul 2010 at 00:50:14 > Hi Fab, > Sorry for the delay in replying... > > On Jul 9, 2010, at 2:22 PM, Fab Tillier wrote: > >> Quincey Koziol wrote on Fri, 9 Jul 2010 at 12:05:52 >> >>> I still think aliasing MPI_Size and MPI_Count will cause problems >>> down the line, since the two concepts are different. I'd be much >>> happier with separating them now rather than having to change things >>> that are switched from int to MPI_Size now to MPI_Count later. >> >> Having MPI_Size and MPI_Count different types causes > MPI_Pack/MPI_Send and MPI_Recv/MPI_Unpack to have problems, since the > count and sizes are passed between the two functions and should be > interchangeable, though. > > Well, there probably is something wrong with the definition of > MPI_Pack and MPI_Unpack then. OK, I've gone off and re-read the > definition of MPI_Pack in the standard and it looks OK to me. What is > the problem with using MPI_Count for the 'incount' parameter and > MPI_Size for the 'outsize' parameter? When you go to MPI_Send a packed buffer, you would pass the 'outsize' as the count parameter, and MPI_PACKED as the datatype. If MPI_Size has a larger range than MPI_Count, you could run into problems. The converse is true of MPI_Recv and MPI_Unpack - if the MPI_Count type is larger than the MPI_Size type, you may not be able to unpack (you could receive more than what you can express in the call to MPI_Unpack.) > (I think I would suggest > changing 'inposition' from a 'int *' to a 'void *' while we're mucking > about in here, also) The inposition is an offset from buf, it's an inout parameter thus being passed by reference so that MPI_Pack can update the position. -Fab From david.solt at [hidden] Fri Jul 23 17:03:41 2010 From: david.solt at [hidden] (Solt, David George) Date: Fri, 23 Jul 2010 22:03:41 +0000 Subject: [Mpi3-bwcompat] Meeting notes from 7/23 Message-ID: Also available on https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/BackCompatMeetings Discussion was around the continued effort around MPI_Count. The latest effort (#224) is a re-work of an early 2.2 proposal to introduce MPI_Count but allow implementations to define MPI_Count to be an int in order to retain backward compatibility. Fabian has been working through some of the details, and there are a few "boundary" cases where MPI_Count interfaces with other concepts. In particular MPI_Pack which currently looks like this: int MPI_Pack(void* inbuf, int incount, MPI_Datatype datatype, void* outbuf, int outsize, int *position, MPI_Comm comm) This API uses outsize and position, neither of which are well-defined as either an int or an MPI_Count. Several ideas were discussed around making one or other of outsize and position one of the following: MPI_Count, size_t, uint64, MPI_Aint, MPI_Size, MPI_Position. We wrestled with the exact meaning of "what is a position" and how does it relate to other uses of MPI_Aint, etc. We discussed whether there needs to be a distinction between the size of an "offset" in memory vs. within a message vs. within a file. (Notetaker's note: MPI_Offset is generally used for file positions, MPI_Aint for memory positions and MPI_Count is proposed to address positions within a message) MPI_Pack ties several of these concepts together because MPI_Pack creates something in memory but is also intended to be used as a message or could be used in a file operation. The "result" of MPI_Pack is currently an int (position) which represents a position in memory (therefore should have been MPI_Aint) but is used roughly as a message count in sends/recvs (thus MPI_Count). The number of bytes which can be packed can exceed the size of MPI_Count and thus could force us to make requirements about the relative sizes of MPI_Aint and MPI_Count. We did not want to do that. In the end we agreed that position, which is currently defined as "current position in buffer, in bytes" should be re-defined as a "current count of MPI_PACKED datatypes within the buffer" (something like that) and therefore can be an MPI_Count. This means that MPI_Pack is now restricted to working with buffers which are only capable of being "indexed" with MPI_Count values. The argument really represents the position within a buffer, but w e have re-framed it in this way to avoid introducing a new type and to ensure that position will be something we can pass as an MPI_Count to communication routines. Another option we considered here was using MPI_Offset for the position argument -- because that's pretty much what it is: an offset. But we can't do that for two reasons: 1) it would break backwards compatibility (because MPI_Offset != int in many current MPI implementations), and 2) you can't pass an MPI_Offset to the count argument of MPI_Send. We will add some text explaining that MPI_Pack is not intended to pack extremely large buffers and that extremely large messages should be pipe-lined with multiple MPI_Pack/MPI_Send operations for performance reasons. The outsize argument, on the other hand, cannot as easily be converted to an MPI_Count argument. We could say that the position is always in terms of how many counts of MPI_BYTE's would fit into the buffer, but that is a contorted use of MPI_Count. So, we proposed to introduce MPI_Size for use as the outsize argument to MPI_Pack/MPI_Unpack. We did not discuss it much, but I believe it will also get used for MPI_Type_size. Another option we considered here was using MPI_Offset for the position argument -- because that's pretty much what it is: an offset. But we can't do that for two reasons: 1) it would break backwards compatibility (because MPI_Offset != int in many current MPI implementations), and 2) you can't pass an MPI_Offset to the count argument of MPI_Send. MPI_Size will not need to be tied in any way to the size of MPI_Count. If on a system, MPI_Count is smaller than MPI_Size, it only means that we can pass in very large buffers to MPI_Pack, but not very large counts. If MPI_Count is larger than MPI _Size, it means that we may not be able to pack as large a message as we can send directly with communication routines. Agreed on a schedule for the next 2 weeks in which Fab will update the ticket based on today's discussion and the other attendees are encouraged to start dividing up the chapters to search for example code which should be changed to reflect the changes being proposed. The plan is to polish off ticket #224 and have an implementation for the October, 2010 meeting. From jsquyres at [hidden] Fri Jul 23 17:08:56 2010 From: jsquyres at [hidden] (Jeff Squyres) Date: Fri, 23 Jul 2010 18:08:56 -0400 Subject: [Mpi3-bwcompat] Meeting notes from 7/23 In-Reply-To: Message-ID: <0A6C7F1D-00CE-465B-A543-F00C46EC01DE@cisco.com> If you couldn't tell from the notes, this was quite a lively meeting. :-) On Jul 23, 2010, at 6:03 PM, Solt, David George wrote: > Also available on https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/BackCompatMeetings > > Discussion was around the continued effort around MPI_Count. The latest effort (#224) is a re-work of an early 2.2 proposal to introduce MPI_Count but allow implementations to define MPI_Count to be an int in order to retain backward compatibility. Fabian has been working through some of the details, and there are a few "boundary" cases where MPI_Count interfaces with other concepts. In particular MPI_Pack which currently looks like this: > > int MPI_Pack(void* inbuf, int incount, MPI_Datatype datatype, void* outbuf, int outsize, int *position, MPI_Comm comm) > > This API uses outsize and position, neither of which are well-defined as either an int or an MPI_Count. Several ideas were discussed around making one or other of outsize and position one of the following: MPI_Count, size_t, uint64, MPI_Aint, MPI_Size, MPI_Position. We wrestled with the exact meaning of "what is a position" and how does it relate to other uses of MPI_Aint, etc. We discussed whether there needs to be a distinction between the size of an "offset" in memory vs. within a message vs. within a file. (Notetaker's note: MPI_Offset is generally used for file positions, MPI_Aint for memory positions and MPI_Count is proposed to address positions within a message) > > MPI_Pack ties several of these concepts together because MPI_Pack creates something in memory but is also intended to be used as a message or could be used in a file operation. The "result" of MPI_Pack is currently an int (position) which represents a position in memory (therefore should have been MPI_Aint) but is used roughly as a message count in sends/recvs (thus MPI_Count). The number of bytes which can be packed can exceed the size of MPI_Count and thus could force us to make requirements about the relative sizes of MPI_Aint and MPI_Count. We did not want to do that. In the end we agreed that position, which is currently defined as "current position in buffer, in bytes" should be re-defined as a "current count of MPI_PACKED datatypes within the buffer" (something like that) and therefore can be an MPI_Count. This means that MPI_Pack is now restricted to working with buffers which are only capable of being "indexed" with MPI_Count values. The argument really represents t! > he position within a buffer, but we have re-framed it in this way to avoid introducing a new type and to ensure that position will be something we can pass as an MPI_Count to communication routines. Another option we considered here was using MPI_Offset for the position argument -- because that's pretty much what it is: an offset. But we can't do that for two reasons: 1) it would break backwards compatibility (because MPI_Offset != int in many current MPI implementations), and 2) you can't pass an MPI_Offset to the count argument of MPI_Send. We will add some text explaining that MPI_Pack is not intended to pack extremely large buffers and that extremely large messages should be pipe-lined with multiple MPI_Pack/MPI_Send operations for performance reasons. > > The outsize argument, on the other hand, cannot as easily be converted to an MPI_Count argument. We could say that the position is always in terms of how many counts of MPI_BYTE's would fit into the buffer, but that is a contorted use of MPI_Count. So, we proposed to introduce MPI_Size for use as the outsize argument to MPI_Pack/MPI_Unpack. We did not discuss it much, but I believe it will also get used for MPI_Type_size. Another option we considered here was using MPI_Offset for the position argument -- because that's pretty much what it is: an offset. But we can't do that for two reasons: 1) it would break backwards compatibility (because MPI_Offset != int in many current MPI implementations), and 2) you can't pass an MPI_Offset to the count argument of MPI_Send. MPI_Size will not need to be tied in any way to the size of MPI_Count. If on a system, MPI_Count is smaller than MPI_Size, it only means that we can pass in very large buffers to MPI_Pack, but not very large count! > s. If MPI_Count is larger than MPI_Size, it means that we may not be able to pack as large a message as we can send directly with communication routines. > > Agreed on a schedule for the next 2 weeks in which Fab will update the ticket based on today's discussion and the other attendees are encouraged to start dividing up the chapters to search for example code which should be changed to reflect the changes being proposed. > > The plan is to polish off ticket #224 and have an implementation for the October, 2010 meeting. > > _______________________________________________ > Mpi3-bwcompat mailing list > Mpi3-bwcompat_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-bwcompat > -- Jeff Squyres jsquyres_at_[hidden] For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/