From alexander.supalov at [hidden] Tue Jun 3 11:29:29 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Tue, 3 Jun 2008 17:29:29 +0100 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at a telecon next week? Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201662BB3@swsmsx413.ger.corp.intel.com> Hi everybody, It looks to me that we're on subsetting crossroads at the moment. There was a lot of good analysis so far, but I'm afraid we're slow on moving towards comparably sized synthesis. On one hand, we have a rather well defined and strictly aimed proposal from Dick that might fit into MPI-2.2 framework and is somewhat related to subsetting. We also have a wider proposal from Jeff, which also has interesting repercussions in the client/MPI data exchange area, potentially usable well outside of the subsetting per se. On the other hand we have a lot of things coming into the standard in MPI-3, and still no clear way to manage this complexity and cater for special purpose MPI configurations that may be necessary for MPI to expand to new areas. Moreover, the meeting in Chicago showed that not everybody agrees that introducing flexibility or going beyond the traditional HPC is a good idea at all. On this backdrop, I'd suggest we should meet on the phone before the Forum gathering in Menlo Park, and try to develop a workable proposition how subsetting can find its way into the standard. I happen to have time for such a meeting on the following days next week: Tuesday, May 10, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B Wednesday, May 11, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B Thursday, May 12, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B Please reply as usual to me whether you can make any of these slots. Best regards. Alexander PS. This time I took special care to have a subsetting slot at the F2F meeting. :) -- Dr Alexander Supalov Intel GmbH Hermuelheimer Strasse 8a 50321 Bruehl, Germany Phone: +49 2232 209034 Mobile: +49 173 511 8735 Fax: +49 2232 209029 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlgraham at [hidden] Tue Jun 3 12:48:14 2008 From: rlgraham at [hidden] (Richard Graham) Date: Tue, 03 Jun 2008 13:48:14 -0400 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at a telecon next week? In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201662BB3@swsmsx413.ger.corp.intel.com> Message-ID: Next week does not work for me at all. Rich On 6/3/08 12:29 PM, "Supalov, Alexander" wrote: > Hi everybody, > > It looks to me that we're on subsetting crossroads at the moment. There was a > lot of good analysis so far, but I'm afraid we're slow on moving towards > comparably sized synthesis. > > On one hand, we have a rather well defined and strictly aimed proposal from > Dick that might fit into MPI-2.2 framework and is somewhat related to > subsetting. We also have a wider proposal from Jeff, which also has > interesting repercussions in the client/MPI data exchange area, potentially > usable well outside of the subsetting per se. > > On the other hand we have a lot of things coming into the standard in MPI-3, > and still no clear way to manage this complexity and cater for special purpose > MPI configurations that may be necessary for MPI to expand to new areas. > Moreover, the meeting in Chicago showed that not everybody agrees that > introducing flexibility or going beyond the traditional HPC is a good idea at > all. > > On this backdrop, I'd suggest we should meet on the phone before the Forum > gathering in Menlo Park, and try to develop a workable proposition how > subsetting can find its way into the standard. > > I happen to have time for such a meeting on the following days next week: > > Tuesday, May 10, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B > Wednesday, May 11, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B > Thursday, May 12, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B > > Please reply as usual to me whether you can make any of these slots. > > Best regards. > > Alexander > > PS. This time I took special care to have a subsetting slot at the F2F > meeting. :) > > -- > Dr Alexander Supalov > Intel GmbH > Hermuelheimer Strasse 8a > 50321 Bruehl, Germany > Phone: +49 2232 209034 > Mobile: +49 173 511 8735 > Fax: +49 2232 209029 > > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > > > _______________________________________________ > mpi3-subsetting mailing list > mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting * -------------- next part -------------- An HTML attachment was scrubbed... URL: From Terry.Dontje at [hidden] Wed Jun 4 05:02:00 2008 From: Terry.Dontje at [hidden] (Terry Dontje) Date: Wed, 04 Jun 2008 06:02:00 -0400 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at a telecon next week? In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201662BB3@swsmsx413.ger.corp.intel.com> Message-ID: <48466818.6070408@sun.com> Supalov, Alexander wrote: > Hi everybody, > > It looks to me that we're on subsetting crossroads at the moment. > There was a lot of good analysis so far, but I'm afraid we're slow on > moving towards comparably sized synthesis. > > On one hand, we have a rather well defined and strictly aimed proposal > from Dick that might fit into MPI-2.2 framework and is somewhat > related to subsetting. We also have a wider proposal from Jeff, > which also has interesting repercussions in the client/MPI > data exchange area, potentially usable well outside of the subsetting > per se. > > On the other hand we have a lot of things coming into the standard in > MPI-3, and still no clear way to manage this complexity and cater for > special purpose MPI configurations that may be necessary for MPI to > expand to new areas. Moreover, the meeting in Chicago showed that not > everybody agrees that introducing flexibility or going beyond the > traditional HPC is a good idea at all. > > On this backdrop, I'd suggest we should meet on the phone before the > Forum gathering in Menlo Park, and try to develop a workable > proposition how subsetting can find its way into the standard. > > I happen to have time for such a meeting on the following days next week: > > Tuesday, May 10, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B No > Wednesday, May 11, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT > yes/no/plan B No > Thursday, May 12, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT > yes/no/plan B Yes. --td > > Please reply as usual to me whether you can make any of these slots. > > Best regards. > > Alexander > > PS. This time I took special care to have a subsetting slot at the F2F > meeting. :) > > -- > Dr Alexander Supalov > Intel GmbH > Hermuelheimer Strasse 8a > 50321 Bruehl, Germany > Phone: +49 2232 209034 > Mobile: +49/ /173 511 8735 > Fax: +49 2232 209029 > > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > > ------------------------------------------------------------------------ > > _______________________________________________ > mpi3-subsetting mailing list > mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > From alexander.supalov at [hidden] Fri Jun 6 11:54:56 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Fri, 6 Jun 2008 17:54:56 +0100 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201662BB3@swsmsx413.ger.corp.intel.com> Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2016897F5@swsmsx413.ger.corp.intel.com> Hi everybody, >From the few replies I've got (thanks!), we're not going to have a quorum next week. Let's shoot for the week after that: Monday, June 16, 2008, 9:00 am PDT/noon EDT/18:00 CEST yes/no/plan B Tuesday, June 17, 2008, 9:00 am PDT/noon EDT/18:00 CEST yes/no/plan B Thursday, June 19, 2008, 9:00 am PDT/noon EDT/18:00 CEST yes/no/plan B Best regards. Alexander ________________________________ From: mpi3-subsetting-bounces_at_[hidden] [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Tuesday, June 03, 2008 6:29 PM To: mpi3-subsetting_at_[hidden] Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? Hi everybody, It looks to me that we're on subsetting crossroads at the moment. There was a lot of good analysis so far, but I'm afraid we're slow on moving towards comparably sized synthesis. On one hand, we have a rather well defined and strictly aimed proposal from Dick that might fit into MPI-2.2 framework and is somewhat related to subsetting. We also have a wider proposal from Jeff, which also has interesting repercussions in the client/MPI data exchange area, potentially usable well outside of the subsetting per se. On the other hand we have a lot of things coming into the standard in MPI-3, and still no clear way to manage this complexity and cater for special purpose MPI configurations that may be necessary for MPI to expand to new areas. Moreover, the meeting in Chicago showed that not everybody agrees that introducing flexibility or going beyond the traditional HPC is a good idea at all. On this backdrop, I'd suggest we should meet on the phone before the Forum gathering in Menlo Park, and try to develop a workable proposition how subsetting can find its way into the standard. I happen to have time for such a meeting on the following days next week: Tuesday, May 10, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B Wednesday, May 11, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B Thursday, May 12, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B Please reply as usual to me whether you can make any of these slots. Best regards. Alexander PS. This time I took special care to have a subsetting slot at the F2F meeting. :) -- Dr Alexander Supalov Intel GmbH Hermuelheimer Strasse 8a 50321 Bruehl, Germany Phone: +49 2232 209034 Mobile: +49 173 511 8735 Fax: +49 2232 209029 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Mon Jun 16 07:42:50 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Mon, 16 Jun 2008 13:42:50 +0100 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201662BB3@swsmsx413.ger.corp.intel.com> Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2016F6F95@swsmsx413.ger.corp.intel.com> Hi everybody, Basing on the overwhelming silence, I propose that we meet on Thursday, June 19, 2008, 9:00 am PDT/noon EDT/18:00 CEST Outside Intel: +1-916-356-2663, Inside Intel: 8-356-2663, Bridge: 4, Passcode: 1459440 - Opens - Subsetting: way forward Best regards. Alexander ________________________________ From: Supalov, Alexander Sent: Friday, June 06, 2008 6:55 PM To: 'MPI 3.0 Sub-setting working group' Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? Hi everybody, >From the few replies I've got (thanks!), we're not going to have a quorum next week. Let's shoot for the week after that: Monday, June 16, 2008, 9:00 am PDT/noon EDT/18:00 CEST yes/no/plan B Tuesday, June 17, 2008, 9:00 am PDT/noon EDT/18:00 CEST yes/no/plan B Thursday, June 19, 2008, 9:00 am PDT/noon EDT/18:00 CEST yes/no/plan B Best regards. Alexander ________________________________ From: mpi3-subsetting-bounces_at_[hidden] [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Tuesday, June 03, 2008 6:29 PM To: mpi3-subsetting_at_[hidden] Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? Hi everybody, It looks to me that we're on subsetting crossroads at the moment. There was a lot of good analysis so far, but I'm afraid we're slow on moving towards comparably sized synthesis. On one hand, we have a rather well defined and strictly aimed proposal from Dick that might fit into MPI-2.2 framework and is somewhat related to subsetting. We also have a wider proposal from Jeff, which also has interesting repercussions in the client/MPI data exchange area, potentially usable well outside of the subsetting per se. On the other hand we have a lot of things coming into the standard in MPI-3, and still no clear way to manage this complexity and cater for special purpose MPI configurations that may be necessary for MPI to expand to new areas. Moreover, the meeting in Chicago showed that not everybody agrees that introducing flexibility or going beyond the traditional HPC is a good idea at all. On this backdrop, I'd suggest we should meet on the phone before the Forum gathering in Menlo Park, and try to develop a workable proposition how subsetting can find its way into the standard. I happen to have time for such a meeting on the following days next week: Tuesday, May 10, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B Wednesday, May 11, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B Thursday, May 12, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B Please reply as usual to me whether you can make any of these slots. Best regards. Alexander PS. This time I took special care to have a subsetting slot at the F2F meeting. :) -- Dr Alexander Supalov Intel GmbH Hermuelheimer Strasse 8a 50321 Bruehl, Germany Phone: +49 2232 209034 Mobile: +49 173 511 8735 Fax: +49 2232 209029 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsquyres at [hidden] Mon Jun 16 09:44:18 2008 From: jsquyres at [hidden] (Jeff Squyres) Date: Mon, 16 Jun 2008 10:44:18 -0400 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2016F6F95@swsmsx413.ger.corp.intel.com> Message-ID: Sorry; was away from e-mail last week. I have a conflict and am unable to attend at noon Easter 19 June. Please don't let that hold up the teleconference; I'm just letting you know that I unfortunately can't be there. On Jun 16, 2008, at 8:42 AM, Supalov, Alexander wrote: > Hi everybody, > > Basing on the overwhelming silence, I propose that we meet on > > Thursday, June 19, 2008, 9:00 am PDT/noon EDT/18:00 CEST > Outside Intel: +1-916-356-2663, Inside Intel: 8-356-2663, Bridge: 4, > Passcode: 1459440 > - Opens > - Subsetting: way forward > > Best regards. > > Alexander > > From: Supalov, Alexander > Sent: Friday, June 06, 2008 6:55 PM > To: 'MPI 3.0 Sub-setting working group' > Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the way > forward at atelecon next week? > > Hi everybody, > > From the few replies I've got (thanks!), we're not going to have a > quorum next week. Let's shoot for the week after that: > > Monday, June 16, 2008, 9:00 am PDT/noon EDT/18:00 CEST yes/no/ > plan B > Tuesday, June 17, 2008, 9:00 am PDT/noon EDT/18:00 CEST yes/no/ > plan B > Thursday, June 19, 2008, 9:00 am PDT/noon EDT/18:00 CEST yes/no/ > plan B > > Best regards. > > Alexander > > From: mpi3-subsetting-bounces_at_[hidden] [mailto:mpi3-subsetting-bounces_at_[hidden] > ] On Behalf Of Supalov, Alexander > Sent: Tuesday, June 03, 2008 6:29 PM > To: mpi3-subsetting_at_[hidden] > Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward > at atelecon next week? > > Hi everybody, > > It looks to me that we're on subsetting crossroads at the moment. > There was a lot of good analysis so far, but I'm afraid we're slow > on moving towards comparably sized synthesis. > > On one hand, we have a rather well defined and strictly aimed > proposal from Dick that might fit into MPI-2.2 framework and is > somewhat related to subsetting. We also have a wider proposal from > Jeff, which also has interesting repercussions in the client/MPI > data exchange area, potentially usable well outside of the > subsetting per se. > > On the other hand we have a lot of things coming into the standard > in MPI-3, and still no clear way to manage this complexity and cater > for special purpose MPI configurations that may be necessary for MPI > to expand to new areas. Moreover, the meeting in Chicago showed that > not everybody agrees that introducing flexibility or going beyond > the traditional HPC is a good idea at all. > > On this backdrop, I'd suggest we should meet on the phone before the > Forum gathering in Menlo Park, and try to develop a workable > proposition how subsetting can find its way into the standard. > > I happen to have time for such a meeting on the following days next > week: > > Tuesday, May 10, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/ > plan B > Wednesday, May 11, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/ > no/plan B > Thursday, May 12, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/ > plan B > > Please reply as usual to me whether you can make any of these slots. > > Best regards. > > Alexander > > PS. This time I took special care to have a subsetting slot at the > F2F meeting. :) > > -- > Dr Alexander Supalov > Intel GmbH > Hermuelheimer Strasse 8a > 50321 Bruehl, Germany > Phone: +49 2232 209034 > Mobile: +49 173 511 8735 > Fax: +49 2232 209029 > > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > _______________________________________________ > mpi3-subsetting mailing list > mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting -- Jeff Squyres Cisco Systems From alexander.supalov at [hidden] Thu Jun 19 12:10:08 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Thu, 19 Jun 2008 18:10:08 +0100 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201662BB3@swsmsx413.ger.corp.intel.com> Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201772D7F@swsmsx413.ger.corp.intel.com> Hi everybody, Thank you for your time today. Here's what we discussed: Present: Mark Schulz (LLNL), Reiner Keller (HLRS), Alexander Supalov (Intel) - Opens - Alexander is not going to be in Menlo Park, will work with Rich to find an adequate solution - Subsetting: way forward - Subsetting in its extreme form (all of the standard is composed of subsets that have user defined hierarchies, attach to opaque objects, etc.) does not seem to be terribly practicable. However, architectural influence, performance, and memory footprint aspects of subsetting warrant further research and discussion in this area. - During the discussion so far we've considered several proposals that apparently go in the subsetting direction, or at least overlap in some parts with the subsetting idea, in that they allow control of the essential aspects related to performance and memory footprint. The WG feels compelled to express their opinion about some of these ideas as follows: - Runtime - The subsetting WG supports Dick Treumann's MPI_Init time int based assertion proposal as expressed in a series of emails, and encourages creation of a full proposal, with the following notes: - The set of standard assertions should be settled and accepted by all implementors - The question of allowing or disallowing implementation specific assertions should be resolved - The distinction between assertions and hints should be formulated in a way understandable to a first time user - The matter of underlying libraries and tools in their relation to assertions should be clarified - It looks that tools that intercept the MPI_Init_* call family will be able to deal with assertions they cannot support by basically ignoring them - Layered underlying libraries are unlikely to be allowed to change assertions already in effect, or adding their own assertions on top of them. These libraries should either adapt themselves to the defined assertions, or bail out - The subsetting WG would like to see more details of Jeff Squyres' proposal on Info based data passing in and out of MPI library before it expresses its opinion, with the following notes: - The effect of setting certain internal thresholds, like eager threshold, should be considered as a collective action on all processes - Generally, the WG does not see a compelling need for assertions/hints/subsets to be attached to the communicator/window/file or other MPI entities, as this does not seem to be directly usable for performance increase or memory footprint reduction - Link time - The WG does not see the need to either specify or prohibit specific link time arrangements like various library names, versions, combinations, dynamic loading thereof, etc. The details should be left to the implementation, with the ABI and compatibility aspects in mind. - Compile time - The WG encourages Reiner Keller to produce a full proposal, minding the ABI and application/MPI compatibility aspects - Actions - Please provide feedback to these minutes by end of week - Alexander will change the proposal in the Wiki accordingly by the time of the MPI Forum meeting Best regards. Alexander ________________________________ From: Supalov, Alexander Sent: Monday, June 16, 2008 2:43 PM To: 'MPI 3.0 Sub-setting working group' Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? Hi everybody, Basing on the overwhelming silence, I propose that we meet on Thursday, June 19, 2008, 9:00 am PDT/noon EDT/18:00 CEST Outside Intel: +1-916-356-2663, Inside Intel: 8-356-2663, Bridge: 4, Passcode: 1459440 - Opens - Subsetting: way forward Best regards. Alexander ________________________________ From: Supalov, Alexander Sent: Friday, June 06, 2008 6:55 PM To: 'MPI 3.0 Sub-setting working group' Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? Hi everybody, >From the few replies I've got (thanks!), we're not going to have a quorum next week. Let's shoot for the week after that: Monday, June 16, 2008, 9:00 am PDT/noon EDT/18:00 CEST yes/no/plan B Tuesday, June 17, 2008, 9:00 am PDT/noon EDT/18:00 CEST yes/no/plan B Thursday, June 19, 2008, 9:00 am PDT/noon EDT/18:00 CEST yes/no/plan B Best regards. Alexander ________________________________ From: mpi3-subsetting-bounces_at_[hidden] [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Tuesday, June 03, 2008 6:29 PM To: mpi3-subsetting_at_[hidden] Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? Hi everybody, It looks to me that we're on subsetting crossroads at the moment. There was a lot of good analysis so far, but I'm afraid we're slow on moving towards comparably sized synthesis. On one hand, we have a rather well defined and strictly aimed proposal from Dick that might fit into MPI-2.2 framework and is somewhat related to subsetting. We also have a wider proposal from Jeff, which also has interesting repercussions in the client/MPI data exchange area, potentially usable well outside of the subsetting per se. On the other hand we have a lot of things coming into the standard in MPI-3, and still no clear way to manage this complexity and cater for special purpose MPI configurations that may be necessary for MPI to expand to new areas. Moreover, the meeting in Chicago showed that not everybody agrees that introducing flexibility or going beyond the traditional HPC is a good idea at all. On this backdrop, I'd suggest we should meet on the phone before the Forum gathering in Menlo Park, and try to develop a workable proposition how subsetting can find its way into the standard. I happen to have time for such a meeting on the following days next week: Tuesday, May 10, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B Wednesday, May 11, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B Thursday, May 12, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B Please reply as usual to me whether you can make any of these slots. Best regards. Alexander PS. This time I took special care to have a subsetting slot at the F2F meeting. :) -- Dr Alexander Supalov Intel GmbH Hermuelheimer Strasse 8a 50321 Bruehl, Germany Phone: +49 2232 209034 Mobile: +49 173 511 8735 Fax: +49 2232 209029 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From treumann at [hidden] Thu Jun 19 14:00:47 2008 From: treumann at [hidden] (Richard Treumann) Date: Thu, 19 Jun 2008 15:00:47 -0400 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201772D7F@swsmsx413.ger.corp.intel.com> Message-ID: Hello to the Subset WG I appreciate the interest in my proposal for Init time assertions. Unfortunately I will not be able to come to the next Forum meeting to discuss face to face. I did write up a fairly detailed draft a few weeks back and post it on the MPI 2.2 Wiki at: http://svn.mpi-forum.org/trac/mpi-forum-web/wiki/InitTimeAssertions Maybe that will help clarify what I have in mind. My proposal does not include "hints" so I did not discuss the definition of a hint. I did try to be firm about the meaning of an assertion. Dick Treumann Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Fri Jun 20 01:13:20 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Fri, 20 Jun 2008 07:13:20 +0100 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201662BB3@swsmsx413.ger.corp.intel.com> Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201772E20@swsmsx413.ger.corp.intel.com> Correction: Martin Schulz (LLNL) rather than Mark. Sorry. ________________________________ From: Supalov, Alexander Sent: Thursday, June 19, 2008 7:10 PM To: 'MPI 3.0 Sub-setting working group' Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? Hi everybody, Thank you for your time today. Here's what we discussed: Present: Mark Schulz (LLNL), Reiner Keller (HLRS), Alexander Supalov (Intel) - Opens - Alexander is not going to be in Menlo Park, will work with Rich to find an adequate solution - Subsetting: way forward - Subsetting in its extreme form (all of the standard is composed of subsets that have user defined hierarchies, attach to opaque objects, etc.) does not seem to be terribly practicable. However, architectural influence, performance, and memory footprint aspects of subsetting warrant further research and discussion in this area. - During the discussion so far we've considered several proposals that apparently go in the subsetting direction, or at least overlap in some parts with the subsetting idea, in that they allow control of the essential aspects related to performance and memory footprint. The WG feels compelled to express their opinion about some of these ideas as follows: - Runtime - The subsetting WG supports Dick Treumann's MPI_Init time int based assertion proposal as expressed in a series of emails, and encourages creation of a full proposal, with the following notes: - The set of standard assertions should be settled and accepted by all implementors - The question of allowing or disallowing implementation specific assertions should be resolved - The distinction between assertions and hints should be formulated in a way understandable to a first time user - The matter of underlying libraries and tools in their relation to assertions should be clarified - It looks that tools that intercept the MPI_Init_* call family will be able to deal with assertions they cannot support by basically ignoring them - Layered underlying libraries are unlikely to be allowed to change assertions already in effect, or adding their own assertions on top of them. These libraries should either adapt themselves to the defined assertions, or bail out - The subsetting WG would like to see more details of Jeff Squyres' proposal on Info based data passing in and out of MPI library before it expresses its opinion, with the following notes: - The effect of setting certain internal thresholds, like eager threshold, should be considered as a collective action on all processes - Generally, the WG does not see a compelling need for assertions/hints/subsets to be attached to the communicator/window/file or other MPI entities, as this does not seem to be directly usable for performance increase or memory footprint reduction - Link time - The WG does not see the need to either specify or prohibit specific link time arrangements like various library names, versions, combinations, dynamic loading thereof, etc. The details should be left to the implementation, with the ABI and compatibility aspects in mind. - Compile time - The WG encourages Reiner Keller to produce a full proposal, minding the ABI and application/MPI compatibility aspects - Actions - Please provide feedback to these minutes by end of week - Alexander will change the proposal in the Wiki accordingly by the time of the MPI Forum meeting Best regards. Alexander ________________________________ From: Supalov, Alexander Sent: Monday, June 16, 2008 2:43 PM To: 'MPI 3.0 Sub-setting working group' Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? Hi everybody, Basing on the overwhelming silence, I propose that we meet on Thursday, June 19, 2008, 9:00 am PDT/noon EDT/18:00 CEST Outside Intel: +1-916-356-2663, Inside Intel: 8-356-2663, Bridge: 4, Passcode: 1459440 - Opens - Subsetting: way forward Best regards. Alexander ________________________________ From: Supalov, Alexander Sent: Friday, June 06, 2008 6:55 PM To: 'MPI 3.0 Sub-setting working group' Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? Hi everybody, >From the few replies I've got (thanks!), we're not going to have a quorum next week. Let's shoot for the week after that: Monday, June 16, 2008, 9:00 am PDT/noon EDT/18:00 CEST yes/no/plan B Tuesday, June 17, 2008, 9:00 am PDT/noon EDT/18:00 CEST yes/no/plan B Thursday, June 19, 2008, 9:00 am PDT/noon EDT/18:00 CEST yes/no/plan B Best regards. Alexander ________________________________ From: mpi3-subsetting-bounces_at_[hidden] [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Tuesday, June 03, 2008 6:29 PM To: mpi3-subsetting_at_[hidden] Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? Hi everybody, It looks to me that we're on subsetting crossroads at the moment. There was a lot of good analysis so far, but I'm afraid we're slow on moving towards comparably sized synthesis. On one hand, we have a rather well defined and strictly aimed proposal from Dick that might fit into MPI-2.2 framework and is somewhat related to subsetting. We also have a wider proposal from Jeff, which also has interesting repercussions in the client/MPI data exchange area, potentially usable well outside of the subsetting per se. On the other hand we have a lot of things coming into the standard in MPI-3, and still no clear way to manage this complexity and cater for special purpose MPI configurations that may be necessary for MPI to expand to new areas. Moreover, the meeting in Chicago showed that not everybody agrees that introducing flexibility or going beyond the traditional HPC is a good idea at all. On this backdrop, I'd suggest we should meet on the phone before the Forum gathering in Menlo Park, and try to develop a workable proposition how subsetting can find its way into the standard. I happen to have time for such a meeting on the following days next week: Tuesday, May 10, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B Wednesday, May 11, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B Thursday, May 12, 2008, 8:00 am PDT/11:00 am EDT/17:00 CDT yes/no/plan B Please reply as usual to me whether you can make any of these slots. Best regards. Alexander PS. This time I took special care to have a subsetting slot at the F2F meeting. :) -- Dr Alexander Supalov Intel GmbH Hermuelheimer Strasse 8a 50321 Bruehl, Germany Phone: +49 2232 209034 Mobile: +49 173 511 8735 Fax: +49 2232 209029 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Fri Jun 20 01:29:23 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Fri, 20 Jun 2008 07:29:23 +0100 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? In-Reply-To: Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201772E36@swsmsx413.ger.corp.intel.com> Dear Dick, Thank you. I remember we exchanged a couple of emails about the possible extensions to the set of assertions, like one-sided and I/O, and in my recollection, almost reached an agreement that this can improve performance and possibly memory footprint, as well as be expressed thru assertions. Do you still feel favorable about this? Best regards. Alexander ________________________________ From: mpi3-subsetting-bounces_at_[hidden] [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Thursday, June 19, 2008 9:01 PM To: MPI 3.0 Sub-setting working group Subject: Re: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? Hello to the Subset WG I appreciate the interest in my proposal for Init time assertions. Unfortunately I will not be able to come to the next Forum meeting to discuss face to face. I did write up a fairly detailed draft a few weeks back and post it on the MPI 2.2 Wiki at: http://svn.mpi-forum.org/trac/mpi-forum-web/wiki/InitTimeAssertions Maybe that will help clarify what I have in mind. My proposal does not include "hints" so I did not discuss the definition of a hint. I did try to be firm about the meaning of an assertion. Dick Treumann Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Fri Jun 20 01:58:41 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Fri, 20 Jun 2008 07:58:41 +0100 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? In-Reply-To: Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201772E55@swsmsx413.ger.corp.intel.com> Dear Dick, A couple of suggestions re your proposal: - If ASSERTIONS is put at the end of the MPI_INIT_ASSERTED argument list, in C++ one can declare the last argument as having a zero default value, and skip it if necessary. This might help with deprecation of the earlier MPI_INIT_* calls. - In non-Cray parts of the world, an MPI_INT followed by MPI_FLOAT is likely to be a 4-byte int followed by a 4-byte float. This sometimes depends on the compiler settings in effect, too. - I don't think MPI_NO_THREAD_CONTENTION is really necessary. The original thread level settings, in particular, the use of anything but MPI_THREAD_MULTIPLE, seem to capture the semantics that you proposed. - I can't fully follow the motivation for MPI_NO_ANY_SOURCE deprioritization. AFAIK, a rendezvous exchange usually starts with a ready-to-send packet that contains the size of the message. In this case the receiving side will normally reply with a ready-to-receive regardless of the buffer space available, and flag MPI_ERR_TRUNCATED on message arrival if necessary. In this case, neither MPI_ANY_SOURCE not MPI_NO_ANY_SOURCE seem to get into way. Best regards. Alexander ________________________________ From: Supalov, Alexander Sent: Friday, June 20, 2008 8:29 AM To: 'MPI 3.0 Sub-setting working group' Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? Dear Dick, Thank you. I remember we exchanged a couple of emails about the possible extensions to the set of assertions, like one-sided and I/O, and in my recollection, almost reached an agreement that this can improve performance and possibly memory footprint, as well as be expressed thru assertions. Do you still feel favorable about this? Best regards. Alexander ________________________________ From: mpi3-subsetting-bounces_at_[hidden] [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Thursday, June 19, 2008 9:01 PM To: MPI 3.0 Sub-setting working group Subject: Re: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? Hello to the Subset WG I appreciate the interest in my proposal for Init time assertions. Unfortunately I will not be able to come to the next Forum meeting to discuss face to face. I did write up a fairly detailed draft a few weeks back and post it on the MPI 2.2 Wiki at: http://svn.mpi-forum.org/trac/mpi-forum-web/wiki/InitTimeAssertions Maybe that will help clarify what I have in mind. My proposal does not include "hints" so I did not discuss the definition of a hint. I did try to be firm about the meaning of an assertion. Dick Treumann Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From treumann at [hidden] Fri Jun 20 08:56:43 2008 From: treumann at [hidden] (Richard Treumann) Date: Fri, 20 Jun 2008 09:56:43 -0400 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201772E55@swsmsx413.ger.corp.intel.com> Message-ID: Hi Alexander Comments imbedded below. I have no objections to someone providing a rationale for assertions related to MPI-IO and MPI_1sided. If the rationale is sound I have no objection to putting them in the proposal. I feel the proposal should be evaluated by the following algorithm. If (this concept is one that seems plausible) { for each proposed assertion { if (rationale not solid) discard if (deal breaker downside) discard } if ((concept makes sense) & (set of worthwhile assertions is not empty)) make this part of MPI 2.2 I do not see much reason to get every assertion that eventually gains traction into MPI 2.2. MPI 3.0 is soon enough for any that do not make the MPI 2.2 cut. I do not want to see the concept fall because some particular assertion is controversial. I consider MPI_NO_EAGER_THROTTLE to be the single most valuable assertion for MPI 2.2 because it is needed to allow MPI to scale to the levels we are already seeing. Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi3-subsetting-bounces_at_[hidden] wrote on 06/20/2008 02:58:41 AM: > Dear Dick, > > A couple of suggestions re your proposal: > > - If ASSERTIONS is put at the end of the MPI_INIT_ASSERTED argument > list, in C++ one can declare the last argument as having a zero > default value, and skip it if necessary. This might help with > deprecation of the earlier MPI_INIT_* calls. I have no objection. It seems reasonable to let C++ default the assertions parameter to "none" > - In non-Cray parts of the world, an MPI_INT followed by MPI_FLOAT > is likely to be a 4-byte int followed by a 4-byte float. This > sometimes depends on the compiler settings in effect, too. My rationale is not specific to any particular architecture. Some MPI datatypes are made entirely from the same base type. Some are mixtures of types. If libmpi knows at the moment a datatype is committed that the send side and receive side will always use the same internal representions then it does not need to keep track of the fact that one instance of {MPI_INT,MPI_FLOAT} has two distinct parts. The send side can gather and ship 8 bytes and the receive side can scatter the 8 bytes. If one side might use 4 byte integers while the other side uses 8 byte integers then at least one side will need to know there is a conversion to be done for the MPI_INT part. If an MPI job does a spawn or join that links to a different architecture after the datatype has been committed, and the MPI_Type_commit has discarded the details, it is too late to get them back. On the other hand, if it is known there will never be a different architecture added to the job, the extra information can be safely discarded. > - I don't think MPI_NO_THREAD_CONTENTION is really necessary. The > original thread level settings, in particular, the use of anything > but MPI_THREAD_MULTIPLE, seem to capture the semantics that you proposed. This one is kind of tricky and I also am not sure what it would mean. If we find a clear value we can keep it and if not we can remove it. > - I can't fully follow the motivation for MPI_NO_ANY_SOURCE > deprioritization. AFAIK, a rendezvous exchange usually starts with a > ready-to-send packet that contains the size of the message. In this > case the receiving side will normally reply with a ready-to-receive > regardless of the buffer space available, and flag MPI_ERR_TRUNCATED > on message arrival if necessary. In this case, neither > MPI_ANY_SOURCE not MPI_NO_ANY_SOURCE seem to get into way. My point is that MPI_NO_ANY_SOURCE might allow this round trip protocol to be replaced by a 1/2 rendezvous protocol. If it is known that MPI_ANY_SOURCE will not be used then the receive side can send an "envelop and ready for data" packet to the send side. As long as the send side knows it will receive the "envelop and ready for data" packet when the receive is posted, it does not need to do the first 1/2 of the rendezvous. The message matching can be done at the send side. A send for which the receive was preposted has a good chance of finding the "envelop and ready for data" sitting in an early queue and the large send can avoid any rendezvous delay. Data begins to flow immediately vs waiting for a round trip of a full rendezvous. In many cases we cut the delay in half and best case we eliminate rendezvous delay completely. If the receive side is late in posting the receive we still save a packet traversal but do not save any time. If there may be an MPI_ANY_SOURCE then this does not work because the receive side that has an MPI_ANY_SOURCE cannot guess which sender to notify so the sender cannot count on getting a 1/2 rendezvous notification for a message that should match the MPI_ANY_SOURCE receive. The problem that made me lower the priority is that many MPIs use an eager protocol for small messages and a rendezvous protocol for large messages. If the send side and receive side have the same size buffer then both sides can reach the same conclusion: eager vs 1/2 rendezvous. If both decide on eager, the receive side will not send an "envelop and ready for data" packet and the send side will not look for one. If both sides decide on 1/2 rendezvous then the receive side will send an "envelop and ready for data" packet and the send side will look for and consume the notice. If the send side is for an 8 byte message and the receive uses a "big enough" receive buffer of 64KB then the two sides will probably not be able to reach the same conclusion about the protocol. The receive side will ship off an "envelop and ready for data" packet that the send side will not know what to do with. > > Best regards. > > Alexander > > From: Supalov, Alexander > Sent: Friday, June 20, 2008 8:29 AM > To: 'MPI 3.0 Sub-setting working group' > Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the way > forward at atelecon next week? > Dear Dick, > > Thank you. I remember we exchanged a couple of emails about the > possible extensions to the set of assertions, like one-sided and > I/O, and in my recollection, almost reached an agreement that this > can improve performance and possibly memory footprint, as well as be > expressed thru assertions. Do you still feel favorable about this? > > Best regards. > > Alexander > * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Fri Jun 20 09:03:00 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Fri, 20 Jun 2008 15:03:00 +0100 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? In-Reply-To: Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A20177321F@swsmsx413.ger.corp.intel.com> Agree, thanks. ________________________________ From: mpi3-subsetting-bounces_at_[hidden] [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Friday, June 20, 2008 3:57 PM To: MPI 3.0 Sub-setting working group Subject: Re: [Mpi3-subsetting] MPI subsetting: charting the way forward at atelecon next week? Hi Alexander Comments imbedded below. I have no objections to someone providing a rationale for assertions related to MPI-IO and MPI_1sided. If the rationale is sound I have no objection to putting them in the proposal. I feel the proposal should be evaluated by the following algorithm. If (this concept is one that seems plausible) { for each proposed assertion { if (rationale not solid) discard if (deal breaker downside) discard } if ((concept makes sense) & (set of worthwhile assertions is not empty)) make this part of MPI 2.2 I do not see much reason to get every assertion that eventually gains traction into MPI 2.2. MPI 3.0 is soon enough for any that do not make the MPI 2.2 cut. I do not want to see the concept fall because some particular assertion is controversial. I consider MPI_NO_EAGER_THROTTLE to be the single most valuable assertion for MPI 2.2 because it is needed to allow MPI to scale to the levels we are already seeing. Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi3-subsetting-bounces_at_[hidden] wrote on 06/20/2008 02:58:41 AM: > Dear Dick, > > A couple of suggestions re your proposal: > > - If ASSERTIONS is put at the end of the MPI_INIT_ASSERTED argument > list, in C++ one can declare the last argument as having a zero > default value, and skip it if necessary. This might help with > deprecation of the earlier MPI_INIT_* calls. I have no objection. It seems reasonable to let C++ default the assertions parameter to "none" > - In non-Cray parts of the world, an MPI_INT followed by MPI_FLOAT > is likely to be a 4-byte int followed by a 4-byte float. This > sometimes depends on the compiler settings in effect, too. My rationale is not specific to any particular architecture. Some MPI datatypes are made entirely from the same base type. Some are mixtures of types. If libmpi knows at the moment a datatype is committed that the send side and receive side will always use the same internal representions then it does not need to keep track of the fact that one instance of {MPI_INT,MPI_FLOAT} has two distinct parts. The send side can gather and ship 8 bytes and the receive side can scatter the 8 bytes. If one side might use 4 byte integers while the other side uses 8 byte integers then at least one side will need to know there is a conversion to be done for the MPI_INT part. If an MPI job does a spawn or join that links to a different architecture after the datatype has been committed, and the MPI_Type_commit has discarded the details, it is too late to get them back. On the other hand, if it is known there will never be a different architecture added to the job, the extra information can be safely discarded. > - I don't think MPI_NO_THREAD_CONTENTION is really necessary. The > original thread level settings, in particular, the use of anything > but MPI_THREAD_MULTIPLE, seem to capture the semantics that you proposed. This one is kind of tricky and I also am not sure what it would mean. If we find a clear value we can keep it and if not we can remove it. > - I can't fully follow the motivation for MPI_NO_ANY_SOURCE > deprioritization. AFAIK, a rendezvous exchange usually starts with a > ready-to-send packet that contains the size of the message. In this > case the receiving side will normally reply with a ready-to-receive > regardless of the buffer space available, and flag MPI_ERR_TRUNCATED > on message arrival if necessary. In this case, neither > MPI_ANY_SOURCE not MPI_NO_ANY_SOURCE seem to get into way. My point is that MPI_NO_ANY_SOURCE might allow this round trip protocol to be replaced by a 1/2 rendezvous protocol. If it is known that MPI_ANY_SOURCE will not be used then the receive side can send an "envelop and ready for data" packet to the send side. As long as the send side knows it will receive the "envelop and ready for data" packet when the receive is posted, it does not need to do the first 1/2 of the rendezvous. The message matching can be done at the send side. A send for which the receive was preposted has a good chance of finding the "envelop and ready for data" sitting in an early queue and the large send can avoid any rendezvous delay. Data begins to flow immediately vs waiting for a round trip of a full rendezvous. In many cases we cut the delay in half and best case we eliminate rendezvous delay completely. If the receive side is late in posting the receive we still save a packet traversal but do not save any time. If there may be an MPI_ANY_SOURCE then this does not work because the receive side that has an MPI_ANY_SOURCE cannot guess which sender to notify so the sender cannot count on getting a 1/2 rendezvous notification for a message that should match the MPI_ANY_SOURCE receive. The problem that made me lower the priority is that many MPIs use an eager protocol for small messages and a rendezvous protocol for large messages. If the send side and receive side have the same size buffer then both sides can reach the same conclusion: eager vs 1/2 rendezvous. If both decide on eager, the receive side will not send an "envelop and ready for data" packet and the send side will not look for one. If both sides decide on 1/2 rendezvous then the receive side will send an "envelop and ready for data" packet and the send side will look for and consume the notice. If the send side is for an 8 byte message and the receive uses a "big enough" receive buffer of 64KB then the two sides will probably not be able to reach the same conclusion about the protocol. The receive side will ship off an "envelop and ready for data" packet that the send side will not know what to do with. > > Best regards. > > Alexander > > From: Supalov, Alexander > Sent: Friday, June 20, 2008 8:29 AM > To: 'MPI 3.0 Sub-setting working group' > Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the way > forward at atelecon next week? > Dear Dick, > > Thank you. I remember we exchanged a couple of emails about the > possible extensions to the set of assertions, like one-sided and > I/O, and in my recollection, almost reached an agreement that this > can improve performance and possibly memory footprint, as well as be > expressed thru assertions. Do you still feel favorable about this? > > Best regards. > > Alexander > --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlgraham at [hidden] Fri Jun 20 11:52:47 2008 From: rlgraham at [hidden] (Richard Graham) Date: Fri, 20 Jun 2008 12:52:47 -0400 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? In-Reply-To: Message-ID: I think we need to be careful here when it comes to assertions, and think hard about how you want to handle these in a standard. In some of the implementations I am familiar with a no-eager-throttle key word would be useless ­ it is vey implementation specific. I suppose this is a big problem with trying to add implementation specific keywords to a standard. It is a given that this will also cause trouble when trying to come up with an ABI, unless one has a large set of defined constants, and are willing to have these be no-ops in certain implementations. Rich On 6/20/08 9:56 AM, "Richard Treumann" wrote: > Hi Alexander > > Comments imbedded below. > > I have no objections to someone providing a rationale for assertions related > to MPI-IO and MPI_1sided. If the rationale is sound I have no objection to > putting them in the proposal. > > I feel the proposal should be evaluated by the following algorithm. > > If (this concept is one that seems plausible) { > for each proposed assertion { > if (rationale not solid) > discard > if (deal breaker downside) > discard > } > if ((concept makes sense) & (set of worthwhile assertions is not empty)) > make this part of MPI 2.2 > > I do not see much reason to get every assertion that eventually gains traction > into MPI 2.2. MPI 3.0 is soon enough for any that do not make the MPI 2.2 > cut. I do not want to see the concept fall because some particular assertion > is controversial. > > I consider MPI_NO_EAGER_THROTTLE to be the single most valuable assertion for > MPI 2.2 because it is needed to allow MPI to scale to the levels we are > already seeing. > > > Dick Treumann - MPI Team/TCEM > IBM Systems & Technology Group > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > > mpi3-subsetting-bounces_at_[hidden] wrote on 06/20/2008 02:58:41 AM: > >> > Dear Dick, >> > >> > A couple of suggestions re your proposal: >> > >> > - If ASSERTIONS is put at the end of the MPI_INIT_ASSERTED argument >> > list, in C++ one can declare the last argument as having a zero >> > default value, and skip it if necessary. This might help with >> > deprecation of the earlier MPI_INIT_* calls. > > I have no objection. It seems reasonable to let C++ default the > assertions parameter to "none" > >> > - In non-Cray parts of the world, an MPI_INT followed by MPI_FLOAT >> > is likely to be a 4-byte int followed by a 4-byte float. This >> > sometimes depends on the compiler settings in effect, too. > > My rationale is not specific to any particular architecture. > Some MPI datatypes are made entirely > from the same base type. Some are mixtures of types. If libmpi knows > at the moment a datatype is committed that the send side and receive > side will always use the same internal representions then it does not > need to keep track of the fact that one instance of {MPI_INT,MPI_FLOAT} > has two distinct parts. The send side can gather and ship 8 bytes > and the receive side can scatter the 8 bytes. If one side might use 4 > byte integers while the other side uses 8 byte integers then at > least one side will need to know there is a conversion to be done for > the MPI_INT part. If an MPI job does a spawn or join that links to a > different architecture after the datatype has been committed, and > the MPI_Type_commit has discarded the details, it is too late to get > them back. On the other hand, if it is known there will never be a > different architecture added to the job, the extra information can be > safely discarded. > >> > - I don't think MPI_NO_THREAD_CONTENTION is really necessary. The >> > original thread level settings, in particular, the use of anything >> > but MPI_THREAD_MULTIPLE, seem to capture the semantics that you proposed. > > This one is kind of tricky and I also am not sure what it would mean. If > we find a clear value we can keep it and if not we can remove it. > >> > - I can't fully follow the motivation for MPI_NO_ANY_SOURCE >> > deprioritization. AFAIK, a rendezvous exchange usually starts with a >> > ready-to-send packet that contains the size of the message. In this >> > case the receiving side will normally reply with a ready-to-receive >> > regardless of the buffer space available, and flag MPI_ERR_TRUNCATED >> > on message arrival if necessary. In this case, neither >> > MPI_ANY_SOURCE not MPI_NO_ANY_SOURCE seem to get into way. > > My point is that MPI_NO_ANY_SOURCE might allow this round trip > protocol to be replaced by a 1/2 rendezvous protocol. If it is known > that MPI_ANY_SOURCE will not be used then the receive side can send > an "envelop and ready for data" packet to the send side. As long as > the send side knows it will receive the "envelop and ready for data" > packet when the receive is posted, it does not need to do the first 1/2 > of the rendezvous. The message matching can be done at the send side. > > A send for which the receive was preposted has a > good chance of finding the "envelop and ready for data" sitting in > an early queue and the large send can avoid any rendezvous delay. > Data begins to flow immediately vs waiting for a round trip of a > full rendezvous. In many cases we cut the delay in half and best > case we eliminate rendezvous delay completely. If the receive side > is late in posting the receive we still save a packet traversal but > do not save any time. > > If there may be an MPI_ANY_SOURCE then this does not work because the > receive side that has an MPI_ANY_SOURCE cannot guess which sender to > notify so the sender cannot count on getting a 1/2 rendezvous > notification for a message that should match the MPI_ANY_SOURCE > receive. > > The problem that made me lower the priority is that many MPIs use an > eager protocol for small messages and a rendezvous protocol for large > messages. If the send side and receive side have the same size buffer > then both sides can reach the same conclusion: eager vs 1/2 rendezvous. > If both decide on eager, the receive side will not send an > "envelop and ready for data" packet and the send side will not look > for one. If both sides decide on 1/2 rendezvous then the receive side > will send an "envelop and ready for data" packet and the send side will > look for and consume the notice. If the send side is for an 8 byte > message and the receive uses a "big enough" receive buffer of 64KB > then the two sides will probably not be able to reach the same > conclusion about the protocol. The receive side will ship off an > "envelop and ready for data" packet that the send side will not > know what to do with. > > >> > >> > Best regards. >> > >> > Alexander >> > >> > From: Supalov, Alexander >> > Sent: Friday, June 20, 2008 8:29 AM >> > To: 'MPI 3.0 Sub-setting working group' >> > Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the way >> > forward at atelecon next week? > >> > Dear Dick, >> > >> > Thank you. I remember we exchanged a couple of emails about the >> > possible extensions to the set of assertions, like one-sided and >> > I/O, and in my recollection, almost reached an agreement that this >> > can improve performance and possibly memory footprint, as well as be >> > expressed thru assertions. Do you still feel favorable about this? >> > >> > Best regards. >> > >> > Alexander >> > > > > > _______________________________________________ > mpi3-subsetting mailing list > mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Fri Jun 20 11:58:03 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Fri, 20 Jun 2008 17:58:03 +0100 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? In-Reply-To: Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201773315@swsmsx413.ger.corp.intel.com> Hi, Ignoring an assertion should be perfectly legal. Best regards. Alexander ________________________________ From: mpi3-subsetting-bounces_at_[hidden] [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Richard Graham Sent: Friday, June 20, 2008 6:53 PM To: MPI 3.0 Sub-setting working group Subject: Re: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? I think we need to be careful here when it comes to assertions, and think hard about how you want to handle these in a standard. In some of the implementations I am familiar with a no-eager-throttle key word would be useless - it is vey implementation specific. I suppose this is a big problem with trying to add implementation specific keywords to a standard. It is a given that this will also cause trouble when trying to come up with an ABI, unless one has a large set of defined constants, and are willing to have these be no-ops in certain implementations. Rich On 6/20/08 9:56 AM, "Richard Treumann" wrote: Hi Alexander Comments imbedded below. I have no objections to someone providing a rationale for assertions related to MPI-IO and MPI_1sided. If the rationale is sound I have no objection to putting them in the proposal. I feel the proposal should be evaluated by the following algorithm. If (this concept is one that seems plausible) { for each proposed assertion { if (rationale not solid) discard if (deal breaker downside) discard } if ((concept makes sense) & (set of worthwhile assertions is not empty)) make this part of MPI 2.2 I do not see much reason to get every assertion that eventually gains traction into MPI 2.2. MPI 3.0 is soon enough for any that do not make the MPI 2.2 cut. I do not want to see the concept fall because some particular assertion is controversial. I consider MPI_NO_EAGER_THROTTLE to be the single most valuable assertion for MPI 2.2 because it is needed to allow MPI to scale to the levels we are already seeing. Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi3-subsetting-bounces_at_[hidden] wrote on 06/20/2008 02:58:41 AM: > Dear Dick, > > A couple of suggestions re your proposal: > > - If ASSERTIONS is put at the end of the MPI_INIT_ASSERTED argument > list, in C++ one can declare the last argument as having a zero > default value, and skip it if necessary. This might help with > deprecation of the earlier MPI_INIT_* calls. I have no objection. It seems reasonable to let C++ default the assertions parameter to "none" > - In non-Cray parts of the world, an MPI_INT followed by MPI_FLOAT > is likely to be a 4-byte int followed by a 4-byte float. This > sometimes depends on the compiler settings in effect, too. My rationale is not specific to any particular architecture. Some MPI datatypes are made entirely from the same base type. Some are mixtures of types. If libmpi knows at the moment a datatype is committed that the send side and receive side will always use the same internal representions then it does not need to keep track of the fact that one instance of {MPI_INT,MPI_FLOAT} has two distinct parts. The send side can gather and ship 8 bytes and the receive side can scatter the 8 bytes. If one side might use 4 byte integers while the other side uses 8 byte integers then at least one side will need to know there is a conversion to be done for the MPI_INT part. If an MPI job does a spawn or join that links to a different architecture after the datatype has been committed, and the MPI_Type_commit has discarded the details, it is too late to get them back. On the other hand, if it is known there will never be a different architecture added to the job, the extra information can be safely discarded. > - I don't think MPI_NO_THREAD_CONTENTION is really necessary. The > original thread level settings, in particular, the use of anything > but MPI_THREAD_MULTIPLE, seem to capture the semantics that you proposed. This one is kind of tricky and I also am not sure what it would mean. If we find a clear value we can keep it and if not we can remove it. > - I can't fully follow the motivation for MPI_NO_ANY_SOURCE > deprioritization. AFAIK, a rendezvous exchange usually starts with a > ready-to-send packet that contains the size of the message. In this > case the receiving side will normally reply with a ready-to-receive > regardless of the buffer space available, and flag MPI_ERR_TRUNCATED > on message arrival if necessary. In this case, neither > MPI_ANY_SOURCE not MPI_NO_ANY_SOURCE seem to get into way. My point is that MPI_NO_ANY_SOURCE might allow this round trip protocol to be replaced by a 1/2 rendezvous protocol. If it is known that MPI_ANY_SOURCE will not be used then the receive side can send an "envelop and ready for data" packet to the send side. As long as the send side knows it will receive the "envelop and ready for data" packet when the receive is posted, it does not need to do the first 1/2 of the rendezvous. The message matching can be done at the send side. A send for which the receive was preposted has a good chance of finding the "envelop and ready for data" sitting in an early queue and the large send can avoid any rendezvous delay. Data begins to flow immediately vs waiting for a round trip of a full rendezvous. In many cases we cut the delay in half and best case we eliminate rendezvous delay completely. If the receive side is late in posting the receive we still save a packet traversal but do not save any time. If there may be an MPI_ANY_SOURCE then this does not work because the receive side that has an MPI_ANY_SOURCE cannot guess which sender to notify so the sender cannot count on getting a 1/2 rendezvous notification for a message that should match the MPI_ANY_SOURCE receive. The problem that made me lower the priority is that many MPIs use an eager protocol for small messages and a rendezvous protocol for large messages. If the send side and receive side have the same size buffer then both sides can reach the same conclusion: eager vs 1/2 rendezvous. If both decide on eager, the receive side will not send an "envelop and ready for data" packet and the send side will not look for one. If both sides decide on 1/2 rendezvous then the receive side will send an "envelop and ready for data" packet and the send side will look for and consume the notice. If the send side is for an 8 byte message and the receive uses a "big enough" receive buffer of 64KB then the two sides will probably not be able to reach the same conclusion about the protocol. The receive side will ship off an "envelop and ready for data" packet that the send side will not know what to do with. > > Best regards. > > Alexander > > From: Supalov, Alexander > Sent: Friday, June 20, 2008 8:29 AM > To: 'MPI 3.0 Sub-setting working group' > Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the way > forward at atelecon next week? > Dear Dick, > > Thank you. I remember we exchanged a couple of emails about the > possible extensions to the set of assertions, like one-sided and > I/O, and in my recollection, almost reached an agreement that this > can improve performance and possibly memory footprint, as well as be > expressed thru assertions. Do you still feel favorable about this? > > Best regards. > > Alexander > ________________________________ _______________________________________________ mpi3-subsetting mailing list mpi3-subsetting_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From bronis at [hidden] Fri Jun 20 12:05:03 2008 From: bronis at [hidden] (Bronis R. de Supinski) Date: Fri, 20 Jun 2008 10:05:03 -0700 (PDT) Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201773315@swsmsx413.ger.corp.intel.com> Message-ID: Yes, but the best approach would be a query/subscribe interface, possibly with some set of standard ketwords that provide portability. On Fri, 20 Jun 2008, Supalov, Alexander wrote: > Hi, > > Ignoring an assertion should be perfectly legal. > > Best regards. > > Alexander > > ________________________________ > > From: mpi3-subsetting-bounces_at_[hidden] > [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of > Richard Graham > Sent: Friday, June 20, 2008 6:53 PM > To: MPI 3.0 Sub-setting working group > Subject: Re: [Mpi3-subsetting] MPI subsetting: charting the way forward > atatelecon next week? > > > I think we need to be careful here when it comes to assertions, and > think hard about how > you want to handle these in a standard. In some of the implementations > I am familiar with > a no-eager-throttle key word would be useless - it is vey > implementation specific. I suppose > this is a big problem with trying to add implementation specific > keywords to a standard. > It is a given that this will also cause trouble when trying to come up > with an ABI, unless > one has a large set of defined constants, and are willing to have these > be no-ops in > certain implementations. > > Rich > > > On 6/20/08 9:56 AM, "Richard Treumann" wrote: > > > > Hi Alexander > > Comments imbedded below. > > I have no objections to someone providing a rationale for > assertions related to MPI-IO and MPI_1sided. If the rationale is sound > I have no objection to putting them in the proposal. > > I feel the proposal should be evaluated by the following > algorithm. > > If (this concept is one that seems plausible) { > for each proposed assertion { > if (rationale not solid) > discard > if (deal breaker downside) > discard > } > if ((concept makes sense) & (set of worthwhile assertions is not > empty)) > make this part of MPI 2.2 > > I do not see much reason to get every assertion that eventually > gains traction into MPI 2.2. MPI 3.0 is soon enough for any that do not > make the MPI 2.2 cut. I do not want to see the concept fall because some > particular assertion is controversial. > > I consider MPI_NO_EAGER_THROTTLE to be the single most valuable > assertion for MPI 2.2 because it is needed to allow MPI to scale to the > levels we are already seeing. > > > Dick Treumann - MPI Team/TCEM > IBM Systems & Technology Group > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > > mpi3-subsetting-bounces_at_[hidden] wrote on 06/20/2008 > 02:58:41 AM: > > > Dear Dick, > > > > A couple of suggestions re your proposal: > > > > - If ASSERTIONS is put at the end of the MPI_INIT_ASSERTED > argument > > list, in C++ one can declare the last argument as having a > zero > > default value, and skip it if necessary. This might help with > > deprecation of the earlier MPI_INIT_* calls. > > I have no objection. It seems reasonable to let C++ default the > assertions parameter to "none" > > > - In non-Cray parts of the world, an MPI_INT followed by > MPI_FLOAT > > is likely to be a 4-byte int followed by a 4-byte float. This > > sometimes depends on the compiler settings in effect, too. > > My rationale is not specific to any particular architecture. > Some MPI datatypes are made entirely > from the same base type. Some are mixtures of types. If libmpi > knows > at the moment a datatype is committed that the send side and > receive > side will always use the same internal representions then it > does not > need to keep track of the fact that one instance of > {MPI_INT,MPI_FLOAT} > has two distinct parts. The send side can gather and ship 8 > bytes > and the receive side can scatter the 8 bytes. If one side might > use 4 > byte integers while the other side uses 8 byte integers then at > least one side will need to know there is a conversion to be > done for > the MPI_INT part. If an MPI job does a spawn or join that links > to a > different architecture after the datatype has been committed, > and > the MPI_Type_commit has discarded the details, it is too late to > get > them back. On the other hand, if it is known there will never > be a > different architecture added to the job, the extra information > can be > safely discarded. > > > - I don't think MPI_NO_THREAD_CONTENTION is really necessary. > The > > original thread level settings, in particular, the use of > anything > > but MPI_THREAD_MULTIPLE, seem to capture the semantics that > you proposed. > > This one is kind of tricky and I also am not sure what it would > mean. If > we find a clear value we can keep it and if not we can remove > it. > > > - I can't fully follow the motivation for MPI_NO_ANY_SOURCE > > deprioritization. AFAIK, a rendezvous exchange usually starts > with a > > ready-to-send packet that contains the size of the message. In > this > > case the receiving side will normally reply with a > ready-to-receive > > regardless of the buffer space available, and flag > MPI_ERR_TRUNCATED > > on message arrival if necessary. In this case, neither > > MPI_ANY_SOURCE not MPI_NO_ANY_SOURCE seem to get into way. > > My point is that MPI_NO_ANY_SOURCE might allow this round trip > protocol to be replaced by a 1/2 rendezvous protocol. If it is > known > that MPI_ANY_SOURCE will not be used then the receive side can > send > an "envelop and ready for data" packet to the send side. As long > as > the send side knows it will receive the "envelop and ready for > data" > packet when the receive is posted, it does not need to do the > first 1/2 > of the rendezvous. The message matching can be done at the send > side. > > A send for which the receive was preposted has a > good chance of finding the "envelop and ready for data" sitting > in > an early queue and the large send can avoid any rendezvous > delay. > Data begins to flow immediately vs waiting for a round trip of a > > full rendezvous. In many cases we cut the delay in half and best > > case we eliminate rendezvous delay completely. If the receive > side > is late in posting the receive we still save a packet traversal > but > do not save any time. > > If there may be an MPI_ANY_SOURCE then this does not work > because the > receive side that has an MPI_ANY_SOURCE cannot guess which > sender to > notify so the sender cannot count on getting a 1/2 rendezvous > notification for a message that should match the MPI_ANY_SOURCE > receive. > > The problem that made me lower the priority is that many MPIs > use an > eager protocol for small messages and a rendezvous protocol for > large > messages. If the send side and receive side have the same size > buffer > then both sides can reach the same conclusion: eager vs 1/2 > rendezvous. > If both decide on eager, the receive side will not send an > "envelop and ready for data" packet and the send side will not > look > for one. If both sides decide on 1/2 rendezvous then the receive > side > will send an "envelop and ready for data" packet and the send > side will > look for and consume the notice. If the send side is for an 8 > byte > message and the receive uses a "big enough" receive buffer of > 64KB > then the two sides will probably not be able to reach the same > conclusion about the protocol. The receive side will ship off an > "envelop and ready for data" packet that the send side will not > know what to do with. > > > > > > Best regards. > > > > Alexander > > > > From: Supalov, Alexander > > Sent: Friday, June 20, 2008 8:29 AM > > To: 'MPI 3.0 Sub-setting working group' > > Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the > way > > forward at atelecon next week? > > > Dear Dick, > > > > Thank you. I remember we exchanged a couple of emails about > the > > possible extensions to the set of assertions, like one-sided > and > > I/O, and in my recollection, almost reached an agreement that > this > > can improve performance and possibly memory footprint, as well > as be > > expressed thru assertions. Do you still feel favorable about > this? > > > > Best regards. > > > > Alexander > > > > > > ________________________________ > > _______________________________________________ > mpi3-subsetting mailing list > mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > > > > > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > From schulzm at [hidden] Fri Jun 20 12:10:47 2008 From: schulzm at [hidden] (Martin Schulz) Date: Fri, 20 Jun 2008 10:10:47 -0700 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201773315@swsmsx413.ger.cor p.intel.com> Message-ID: <200806201710.m5KHAnUA007595@mail-1.llnl.gov> At 09:58 AM 6/20/2008, Supalov, Alexander wrote: >Content-class: urn:content-classes:message >Content-Type: multipart/alternative; > boundary="----_=_NextPart_001_01C8D2F6.CE1CA5E4" > >Hi, > >Ignoring an assertion should be perfectly legal. I fully agree, ignoring should always be OK, which ensures the portability of any application using assertions. However, I do see Rich's point - how useful are assertions if we have hundreds of them and each just works on a particular MPI implementation (or even version)? Also, if these constants are really implementation specific, does it make sense to have them in the MPI standard? Each vender will want their own set (and rightfully so) and the burden is then on the programmer to know all of the different options and understand the subtle differences (and we have to document them all in the standard). Perhaps we should just define broad groups of assertions and define those in the standard. The user can then query for all available assertions in that group for a particular implementation. This would have to coupled with an ability to uniquely identify certain MPI implementations at runtime. Also, this does not solve the problem for the end-user of how to select the correct assertion. Martin Martin > >Best regards. > >Alexander > > >---------- >From: mpi3-subsetting-bounces_at_[hidden] >[mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of >Richard Graham >Sent: Friday, June 20, 2008 6:53 PM >To: MPI 3.0 Sub-setting working group >Subject: Re: [Mpi3-subsetting] MPI subsetting: charting the way >forward atatelecon next week? > >I think we need to be careful here when it comes to assertions, and >think hard about how > you want to handle these in a standard. In some of the > implementations I am familiar with > a no-eager-throttle key word would be useless – it is vey > implementation specific. I suppose > this is a big problem with trying to add implementation specific > keywords to a standard. > It is a given that this will also cause trouble when trying to > come up with an ABI, unless > one has a large set of defined constants, and are willing to have > these be no-ops in > certain implementations. > >Rich > > >On 6/20/08 9:56 AM, "Richard Treumann" wrote: > >Hi Alexander > >Comments imbedded below. > >I have no objections to someone providing a rationale for assertions >related to MPI-IO and MPI_1sided. If the rationale is sound I have >no objection to putting them in the proposal. > >I feel the proposal should be evaluated by the following algorithm. > >If (this concept is one that seems plausible) { > for each proposed assertion { > if (rationale not solid) > discard > if (deal breaker downside) > discard > } >if ((concept makes sense) & (set of worthwhile assertions is not empty)) > make this part of MPI 2.2 > >I do not see much reason to get every assertion that eventually >gains traction into MPI 2.2. MPI 3.0 is soon enough for any that do >not make the MPI 2.2 cut. I do not want to see the concept fall >because some particular assertion is controversial. > >I consider MPI_NO_EAGER_THROTTLE to be the single most valuable >assertion for MPI 2.2 because it is needed to allow MPI to scale to >the levels we are already seeing. > > >Dick Treumann - MPI Team/TCEM >IBM Systems & Technology Group >Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 >Tele (845) 433-7846 Fax (845) 433-8363 > > >mpi3-subsetting-bounces_at_[hidden] wrote on 06/20/2008 02:58:41 AM: > > > Dear Dick, > > > > A couple of suggestions re your proposal: > > > > - If ASSERTIONS is put at the end of the MPI_INIT_ASSERTED argument > > list, in C++ one can declare the last argument as having a zero > > default value, and skip it if necessary. This might help with > > deprecation of the earlier MPI_INIT_* calls. > >I have no objection. It seems reasonable to let C++ default the >assertions parameter to "none" > > > - In non-Cray parts of the world, an MPI_INT followed by MPI_FLOAT > > is likely to be a 4-byte int followed by a 4-byte float. This > > sometimes depends on the compiler settings in effect, too. > >My rationale is not specific to any particular architecture. >Some MPI datatypes are made entirely >from the same base type. Some are mixtures of types. If libmpi knows >at the moment a datatype is committed that the send side and receive >side will always use the same internal representions then it does not >need to keep track of the fact that one instance of {MPI_INT,MPI_FLOAT} >has two distinct parts. The send side can gather and ship 8 bytes >and the receive side can scatter the 8 bytes. If one side might use 4 >byte integers while the other side uses 8 byte integers then at >least one side will need to know there is a conversion to be done for >the MPI_INT part. If an MPI job does a spawn or join that links to a >different architecture after the datatype has been committed, and >the MPI_Type_commit has discarded the details, it is too late to get >them back. On the other hand, if it is known there will never be a >different architecture added to the job, the extra information can be >safely discarded. > > > - I don't think MPI_NO_THREAD_CONTENTION is really necessary. The > > original thread level settings, in particular, the use of anything > > but MPI_THREAD_MULTIPLE, seem to capture the semantics that you proposed. > >This one is kind of tricky and I also am not sure what it would mean. If >we find a clear value we can keep it and if not we can remove it. > > > - I can't fully follow the motivation for MPI_NO_ANY_SOURCE > > deprioritization. AFAIK, a rendezvous exchange usually starts with a > > ready-to-send packet that contains the size of the message. In this > > case the receiving side will normally reply with a ready-to-receive > > regardless of the buffer space available, and flag MPI_ERR_TRUNCATED > > on message arrival if necessary. In this case, neither > > MPI_ANY_SOURCE not MPI_NO_ANY_SOURCE seem to get into way. > >My point is that MPI_NO_ANY_SOURCE might allow this round trip >protocol to be replaced by a 1/2 rendezvous protocol. If it is known >that MPI_ANY_SOURCE will not be used then the receive side can send >an "envelop and ready for data" packet to the send side. As long as >the send side knows it will receive the "envelop and ready for data" >packet when the receive is posted, it does not need to do the first 1/2 >of the rendezvous. The message matching can be done at the send side. > >A send for which the receive was preposted has a >good chance of finding the "envelop and ready for data" sitting in >an early queue and the large send can avoid any rendezvous delay. >Data begins to flow immediately vs waiting for a round trip of a >full rendezvous. In many cases we cut the delay in half and best >case we eliminate rendezvous delay completely. If the receive side >is late in posting the receive we still save a packet traversal but >do not save any time. > >If there may be an MPI_ANY_SOURCE then this does not work because the >receive side that has an MPI_ANY_SOURCE cannot guess which sender to >notify so the sender cannot count on getting a 1/2 rendezvous >notification for a message that should match the MPI_ANY_SOURCE >receive. > >The problem that made me lower the priority is that many MPIs use an >eager protocol for small messages and a rendezvous protocol for large >messages. If the send side and receive side have the same size buffer >then both sides can reach the same conclusion: eager vs 1/2 rendezvous. >If both decide on eager, the receive side will not send an >"envelop and ready for data" packet and the send side will not look >for one. If both sides decide on 1/2 rendezvous then the receive side >will send an "envelop and ready for data" packet and the send side will >look for and consume the notice. If the send side is for an 8 byte >message and the receive uses a "big enough" receive buffer of 64KB >then the two sides will probably not be able to reach the same >conclusion about the protocol. The receive side will ship off an >"envelop and ready for data" packet that the send side will not >know what to do with. > > > > > > Best regards. > > > > Alexander > > > > From: Supalov, Alexander > > Sent: Friday, June 20, 2008 8:29 AM > > To: 'MPI 3.0 Sub-setting working group' > > Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the way > > forward at atelecon next week? > > > Dear Dick, > > > > Thank you. I remember we exchanged a couple of emails about the > > possible extensions to the set of assertions, like one-sided and > > I/O, and in my recollection, almost reached an agreement that this > > can improve performance and possibly memory footprint, as well as be > > expressed thru assertions. Do you still feel favorable about this? > > > > Best regards. > > > > Alexander > > > > > >_______________________________________________ >mpi3-subsetting mailing list >mpi3-subsetting_at_[hidden] >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > > > >--------------------------------------------------------------------- >Intel GmbH >Dornacher Strasse 1 >85622 Feldkirchen/Muenchen Germany >Sitz der Gesellschaft: Feldkirchen bei Muenchen >Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer >Registergericht: Muenchen HRB 47456 Ust.-IdNr. >VAT Registration No.: DE129385895 >Citibank Frankfurt (BLZ 502 109 00) 600119052 > >This e-mail and any attachments may contain confidential material for >the sole use of the intended recipient(s). Any review or distribution >by others is strictly prohibited. If you are not the intended >recipient, please contact the sender and delete all copies. > >_______________________________________________ >mpi3-subsetting mailing list >mpi3-subsetting_at_[hidden]http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting _______________________________________________________________________ Martin Schulz, schulzm_at_[hidden], http://people.llnl.gov/schulz6 CASC @ Lawrence Livermore National Laboratory, Livermore, USA * -------------- next part -------------- An HTML attachment was scrubbed... URL: From treumann at [hidden] Fri Jun 20 12:12:24 2008 From: treumann at [hidden] (Richard Treumann) Date: Fri, 20 Jun 2008 13:12:24 -0400 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201773315@swsmsx413.ger.corp.intel.com> Message-ID: Of course - An assertion is a statement that the application does not require some MPI standard feature or semantic guarantee. It is not a directive to the MPI implementation. The implementation is free to provide that feature or guarantee even if the application says it is not needed. The story is a bit different for a helper library because the library is logically a more or less opaque part of the application. If the caller of MPI_INIT_ASSERTED says the application does not require something and then uses a library that does require that feature or guarantee - the library must raise an error or adjust its behavior to live within the assertion. Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi3-subsetting-bounces_at_[hidden] wrote on 06/20/2008 12:58:03 PM: > Hi, > > Ignoring an assertion should be perfectly legal. > > Best regards. > * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Fri Jun 20 12:12:49 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Fri, 20 Jun 2008 18:12:49 +0100 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? In-Reply-To: <200806201710.m5KHAnUA007595@mail-1.llnl.gov> Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201773328@swsmsx413.ger.corp.intel.com> Thanks. If you look into Dick's proposal, you'll find just a handful of assertions. A 32-bit int is not large enough for hundreds of assertions anyway. ________________________________ From: mpi3-subsetting-bounces_at_[hidden] [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Martin Schulz Sent: Friday, June 20, 2008 7:11 PM To: MPI 3.0 Sub-setting working group Subject: Re: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? At 09:58 AM 6/20/2008, Supalov, Alexander wrote: Content-class: urn:content-classes:message Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01C8D2F6.CE1CA5E4" Hi, Ignoring an assertion should be perfectly legal. I fully agree, ignoring should always be OK, which ensures the portability of any application using assertions. However, I do see Rich's point - how useful are assertions if we have hundreds of them and each just works on a particular MPI implementation (or even version)? Also, if these constants are really implementation specific, does it make sense to have them in the MPI standard? Each vender will want their own set (and rightfully so) and the burden is then on the programmer to know all of the different options and understand the subtle differences (and we have to document them all in the standard). Perhaps we should just define broad groups of assertions and define those in the standard. The user can then query for all available assertions in that group for a particular implementation. This would have to coupled with an ability to uniquely identify certain MPI implementations at runtime. Also, this does not solve the problem for the end-user of how to select the correct assertion. Martin Martin Best regards. Alexander ________________________________ From: mpi3-subsetting-bounces_at_[hidden] [ mailto:mpi3-subsetting-bounces_at_[hidden] ] On Behalf Of Richard Graham Sent: Friday, June 20, 2008 6:53 PM To: MPI 3.0 Sub-setting working group Subject: Re: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? I think we need to be careful here when it comes to assertions, and think hard about how you want to handle these in a standard. In some of the implementations I am familiar with a no-eager-throttle key word would be useless - it is vey implementation specific. I suppose this is a big problem with trying to add implementation specific keywords to a standard. It is a given that this will also cause trouble when trying to come up with an ABI, unless one has a large set of defined constants, and are willing to have these be no-ops in certain implementations. Rich On 6/20/08 9:56 AM, "Richard Treumann" wrote: Hi Alexander Comments imbedded below. I have no objections to someone providing a rationale for assertions related to MPI-IO and MPI_1sided. If the rationale is sound I have no objection to putting them in the proposal. I feel the proposal should be evaluated by the following algorithm. If (this concept is one that seems plausible) { for each proposed assertion { if (rationale not solid) discard if (deal breaker downside) discard } if ((concept makes sense) & (set of worthwhile assertions is not empty)) make this part of MPI 2.2 I do not see much reason to get every assertion that eventually gains traction into MPI 2.2. MPI 3.0 is soon enough for any that do not make the MPI 2.2 cut. I do not want to see the concept fall because some particular assertion is controversial. I consider MPI_NO_EAGER_THROTTLE to be the single most valuable assertion for MPI 2.2 because it is needed to allow MPI to scale to the levels we are already seeing. Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi3-subsetting-bounces_at_[hidden] wrote on 06/20/2008 02:58:41 AM: > Dear Dick, > > A couple of suggestions re your proposal: > > - If ASSERTIONS is put at the end of the MPI_INIT_ASSERTED argument > list, in C++ one can declare the last argument as having a zero > default value, and skip it if necessary. This might help with > deprecation of the earlier MPI_INIT_* calls. I have no objection. It seems reasonable to let C++ default the assertions parameter to "none" > - In non-Cray parts of the world, an MPI_INT followed by MPI_FLOAT > is likely to be a 4-byte int followed by a 4-byte float. This > sometimes depends on the compiler settings in effect, too. My rationale is not specific to any particular architecture. Some MPI datatypes are made entirely from the same base type. Some are mixtures of types. If libmpi knows at the moment a datatype is committed that the send side and receive side will always use the same internal representions then it does not need to keep track of the fact that one instance of {MPI_INT,MPI_FLOAT} has two distinct parts. The send side can gather and ship 8 bytes and the receive side can scatter the 8 bytes. If one side might use 4 byte integers while the other side uses 8 byte integers then at least one side will need to know there is a conversion to be done for the MPI_INT part. If an MPI job does a spawn or join that links to a different architecture after the datatype has been committed, and the MPI_Type_commit has discarded the details, it is too late to get them back. On the other hand, if it is known there will never be a different architecture added to the job, the extra information can be safely discarded. > - I don't think MPI_NO_THREAD_CONTENTION is really necessary. The > original thread level settings, in particular, the use of anything > but MPI_THREAD_MULTIPLE, seem to capture the semantics that you proposed. This one is kind of tricky and I also am not sure what it would mean. If we find a clear value we can keep it and if not we can remove it. > - I can't fully follow the motivation for MPI_NO_ANY_SOURCE > deprioritization. AFAIK, a rendezvous exchange usually starts with a > ready-to-send packet that contains the size of the message. In this > case the receiving side will normally reply with a ready-to-receive > regardless of the buffer space available, and flag MPI_ERR_TRUNCATED > on message arrival if necessary. In this case, neither > MPI_ANY_SOURCE not MPI_NO_ANY_SOURCE seem to get into way. My point is that MPI_NO_ANY_SOURCE might allow this round trip protocol to be replaced by a 1/2 rendezvous protocol. If it is known that MPI_ANY_SOURCE will not be used then the receive side can send an "envelop and ready for data" packet to the send side. As long as the send side knows it will receive the "envelop and ready for data" packet when the receive is posted, it does not need to do the first 1/2 of the rendezvous. The message matching can be done at the send side. A send for which the receive was preposted has a good chance of finding the "envelop and ready for data" sitting in an early queue and the large send can avoid any rendezvous delay. Data begins to flow immediately vs waiting for a round trip of a full rendezvous. In many cases we cut the delay in half and best case we eliminate rendezvous delay completely. If the receive side is late in posting the receive we still save a packet traversal but do not save any time. If there may be an MPI_ANY_SOURCE then this does not work because the receive side that has an MPI_ANY_SOURCE cannot guess which sender to notify so the sender cannot count on getting a 1/2 rendezvous notification for a message that should match the MPI_ANY_SOURCE receive. The problem that made me lower the priority is that many MPIs use an eager protocol for small messages and a rendezvous protocol for large messages. If the send side and receive side have the same size buffer then both sides can reach the same conclusion: eager vs 1/2 rendezvous. If both decide on eager, the receive side will not send an "envelop and ready for data" packet and the send side will not look for one. If both sides decide on 1/2 rendezvous then the receive side will send an "envelop and ready for data" packet and the send side will look for and consume the notice. If the send side is for an 8 byte message and the receive uses a "big enough" receive buffer of 64KB then the two sides will probably not be able to reach the same conclusion about the protocol. The receive side will ship off an "envelop and ready for data" packet that the send side will not know what to do with. > > Best regards. > > Alexander > > From: Supalov, Alexander > Sent: Friday, June 20, 2008 8:29 AM > To: 'MPI 3.0 Sub-setting working group' > Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the way > forward at atelecon next week? > Dear Dick, > > Thank you. I remember we exchanged a couple of emails about the > possible extensions to the set of assertions, like one-sided and > I/O, and in my recollection, almost reached an agreement that this > can improve performance and possibly memory footprint, as well as be > expressed thru assertions. Do you still feel favorable about this? > > Best regards. > > Alexander > _______________________________________________ mpi3-subsetting mailing list mpi3-subsetting_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. _______________________________________________ mpi3-subsetting mailing list mpi3-subsetting_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting _______________________________________________________________________ Martin Schulz, schulzm_at_[hidden], http://people.llnl.gov/schulz6 CASC @ Lawrence Livermore National Laboratory, Livermore, USA --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Fri Jun 20 12:14:30 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Fri, 20 Jun 2008 18:14:30 +0100 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? In-Reply-To: Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A20177332E@swsmsx413.ger.corp.intel.com> Why make it difficult when an int in the mpi_init call seems to be sufficient? -----Original Message----- From: mpi3-subsetting-bounces_at_[hidden] [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Bronis R. de Supinski Sent: Friday, June 20, 2008 7:05 PM To: MPI 3.0 Sub-setting working group Subject: Re: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? Yes, but the best approach would be a query/subscribe interface, possibly with some set of standard ketwords that provide portability. On Fri, 20 Jun 2008, Supalov, Alexander wrote: > Hi, > > Ignoring an assertion should be perfectly legal. > > Best regards. > > Alexander > > ________________________________ > > From: mpi3-subsetting-bounces_at_[hidden] > [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of > Richard Graham > Sent: Friday, June 20, 2008 6:53 PM > To: MPI 3.0 Sub-setting working group > Subject: Re: [Mpi3-subsetting] MPI subsetting: charting the way forward > atatelecon next week? > > > I think we need to be careful here when it comes to assertions, and > think hard about how > you want to handle these in a standard. In some of the implementations > I am familiar with > a no-eager-throttle key word would be useless - it is vey > implementation specific. I suppose > this is a big problem with trying to add implementation specific > keywords to a standard. > It is a given that this will also cause trouble when trying to come up > with an ABI, unless > one has a large set of defined constants, and are willing to have these > be no-ops in > certain implementations. > > Rich > > > On 6/20/08 9:56 AM, "Richard Treumann" wrote: > > > > Hi Alexander > > Comments imbedded below. > > I have no objections to someone providing a rationale for > assertions related to MPI-IO and MPI_1sided. If the rationale is sound > I have no objection to putting them in the proposal. > > I feel the proposal should be evaluated by the following > algorithm. > > If (this concept is one that seems plausible) { > for each proposed assertion { > if (rationale not solid) > discard > if (deal breaker downside) > discard > } > if ((concept makes sense) & (set of worthwhile assertions is not > empty)) > make this part of MPI 2.2 > > I do not see much reason to get every assertion that eventually > gains traction into MPI 2.2. MPI 3.0 is soon enough for any that do not > make the MPI 2.2 cut. I do not want to see the concept fall because some > particular assertion is controversial. > > I consider MPI_NO_EAGER_THROTTLE to be the single most valuable > assertion for MPI 2.2 because it is needed to allow MPI to scale to the > levels we are already seeing. > > > Dick Treumann - MPI Team/TCEM > IBM Systems & Technology Group > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > > mpi3-subsetting-bounces_at_[hidden] wrote on 06/20/2008 > 02:58:41 AM: > > > Dear Dick, > > > > A couple of suggestions re your proposal: > > > > - If ASSERTIONS is put at the end of the MPI_INIT_ASSERTED > argument > > list, in C++ one can declare the last argument as having a > zero > > default value, and skip it if necessary. This might help with > > deprecation of the earlier MPI_INIT_* calls. > > I have no objection. It seems reasonable to let C++ default the > assertions parameter to "none" > > > - In non-Cray parts of the world, an MPI_INT followed by > MPI_FLOAT > > is likely to be a 4-byte int followed by a 4-byte float. This > > sometimes depends on the compiler settings in effect, too. > > My rationale is not specific to any particular architecture. > Some MPI datatypes are made entirely > from the same base type. Some are mixtures of types. If libmpi > knows > at the moment a datatype is committed that the send side and > receive > side will always use the same internal representions then it > does not > need to keep track of the fact that one instance of > {MPI_INT,MPI_FLOAT} > has two distinct parts. The send side can gather and ship 8 > bytes > and the receive side can scatter the 8 bytes. If one side might > use 4 > byte integers while the other side uses 8 byte integers then at > least one side will need to know there is a conversion to be > done for > the MPI_INT part. If an MPI job does a spawn or join that links > to a > different architecture after the datatype has been committed, > and > the MPI_Type_commit has discarded the details, it is too late to > get > them back. On the other hand, if it is known there will never > be a > different architecture added to the job, the extra information > can be > safely discarded. > > > - I don't think MPI_NO_THREAD_CONTENTION is really necessary. > The > > original thread level settings, in particular, the use of > anything > > but MPI_THREAD_MULTIPLE, seem to capture the semantics that > you proposed. > > This one is kind of tricky and I also am not sure what it would > mean. If > we find a clear value we can keep it and if not we can remove > it. > > > - I can't fully follow the motivation for MPI_NO_ANY_SOURCE > > deprioritization. AFAIK, a rendezvous exchange usually starts > with a > > ready-to-send packet that contains the size of the message. In > this > > case the receiving side will normally reply with a > ready-to-receive > > regardless of the buffer space available, and flag > MPI_ERR_TRUNCATED > > on message arrival if necessary. In this case, neither > > MPI_ANY_SOURCE not MPI_NO_ANY_SOURCE seem to get into way. > > My point is that MPI_NO_ANY_SOURCE might allow this round trip > protocol to be replaced by a 1/2 rendezvous protocol. If it is > known > that MPI_ANY_SOURCE will not be used then the receive side can > send > an "envelop and ready for data" packet to the send side. As long > as > the send side knows it will receive the "envelop and ready for > data" > packet when the receive is posted, it does not need to do the > first 1/2 > of the rendezvous. The message matching can be done at the send > side. > > A send for which the receive was preposted has a > good chance of finding the "envelop and ready for data" sitting > in > an early queue and the large send can avoid any rendezvous > delay. > Data begins to flow immediately vs waiting for a round trip of a > > full rendezvous. In many cases we cut the delay in half and best > > case we eliminate rendezvous delay completely. If the receive > side > is late in posting the receive we still save a packet traversal > but > do not save any time. > > If there may be an MPI_ANY_SOURCE then this does not work > because the > receive side that has an MPI_ANY_SOURCE cannot guess which > sender to > notify so the sender cannot count on getting a 1/2 rendezvous > notification for a message that should match the MPI_ANY_SOURCE > receive. > > The problem that made me lower the priority is that many MPIs > use an > eager protocol for small messages and a rendezvous protocol for > large > messages. If the send side and receive side have the same size > buffer > then both sides can reach the same conclusion: eager vs 1/2 > rendezvous. > If both decide on eager, the receive side will not send an > "envelop and ready for data" packet and the send side will not > look > for one. If both sides decide on 1/2 rendezvous then the receive > side > will send an "envelop and ready for data" packet and the send > side will > look for and consume the notice. If the send side is for an 8 > byte > message and the receive uses a "big enough" receive buffer of > 64KB > then the two sides will probably not be able to reach the same > conclusion about the protocol. The receive side will ship off an > "envelop and ready for data" packet that the send side will not > know what to do with. > > > > > > Best regards. > > > > Alexander > > > > From: Supalov, Alexander > > Sent: Friday, June 20, 2008 8:29 AM > > To: 'MPI 3.0 Sub-setting working group' > > Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the > way > > forward at atelecon next week? > > > Dear Dick, > > > > Thank you. I remember we exchanged a couple of emails about > the > > possible extensions to the set of assertions, like one-sided > and > > I/O, and in my recollection, almost reached an agreement that > this > > can improve performance and possibly memory footprint, as well > as be > > expressed thru assertions. Do you still feel favorable about > this? > > > > Best regards. > > > > Alexander > > > > > > ________________________________ > > _______________________________________________ > mpi3-subsetting mailing list > mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > > > > > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > _______________________________________________ mpi3-subsetting mailing list mpi3-subsetting_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. From rlgraham at [hidden] Fri Jun 20 12:26:13 2008 From: rlgraham at [hidden] (Richard Graham) Date: Fri, 20 Jun 2008 13:26:13 -0400 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forwardatatelecon next week? In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A20177332E@swsmsx413.ger.corp.intel.com> Message-ID: An mpi_int is potentially asking for trouble. If this allow for 32 different parameters, this is far too limited. If it allows for 2^32 values, I hope we can¹t come up with that many restrictions. Back to the assertion issue. In the context of an ABI (if this becomes a reality), it does not make much sense to standardize on things that are not standard, but stick to items that are defined within the standard, such as ³I will not use wild card receives² (I am not advocating that one at all, just using it as an example), ³I will only use basic MPI types², ... For things that are not standard, such as ³no-eager-throttle², it would make sense to me to have a standard way for implementations to expose which ones the support, where their meaning is defined, and how a user would invoke these ­ at mpi_init, via mpi_int_?, or some other way. Rich On 6/20/08 1:14 PM, "Supalov, Alexander" wrote: > Why make it difficult when an int in the mpi_init call seems to be > sufficient? > > -----Original Message----- > From: mpi3-subsetting-bounces_at_[hidden] > [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Bronis > R. de Supinski > Sent: Friday, June 20, 2008 7:05 PM > To: MPI 3.0 Sub-setting working group > Subject: Re: [Mpi3-subsetting] MPI subsetting: charting the way forward > atatelecon next week? > > > Yes, but the best approach would be a query/subscribe > interface, possibly with some set of standard ketwords > that provide portability. > > On Fri, 20 Jun 2008, Supalov, Alexander wrote: > >> > Hi, >> > >> > Ignoring an assertion should be perfectly legal. >> > >> > Best regards. >> > >> > Alexander >> > >> > ________________________________ >> > >> > From: mpi3-subsetting-bounces_at_[hidden] >> > [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of >> > Richard Graham >> > Sent: Friday, June 20, 2008 6:53 PM >> > To: MPI 3.0 Sub-setting working group >> > Subject: Re: [Mpi3-subsetting] MPI subsetting: charting the way > forward >> > atatelecon next week? >> > >> > >> > I think we need to be careful here when it comes to assertions, and >> > think hard about how >> > you want to handle these in a standard. In some of the > implementations >> > I am familiar with >> > a no-eager-throttle key word would be useless - it is vey >> > implementation specific. I suppose >> > this is a big problem with trying to add implementation specific >> > keywords to a standard. >> > It is a given that this will also cause trouble when trying to come > up >> > with an ABI, unless >> > one has a large set of defined constants, and are willing to have > these >> > be no-ops in >> > certain implementations. >> > >> > Rich >> > >> > >> > On 6/20/08 9:56 AM, "Richard Treumann" wrote: >> > >> > >> > >> > Hi Alexander >> > >> > Comments imbedded below. >> > >> > I have no objections to someone providing a rationale for >> > assertions related to MPI-IO and MPI_1sided. If the rationale is > sound >> > I have no objection to putting them in the proposal. >> > >> > I feel the proposal should be evaluated by the following >> > algorithm. >> > >> > If (this concept is one that seems plausible) { >> > for each proposed assertion { >> > if (rationale not solid) >> > discard >> > if (deal breaker downside) >> > discard >> > } >> > if ((concept makes sense) & (set of worthwhile assertions is not >> > empty)) >> > make this part of MPI 2.2 >> > >> > I do not see much reason to get every assertion that eventually >> > gains traction into MPI 2.2. MPI 3.0 is soon enough for any that do > not >> > make the MPI 2.2 cut. I do not want to see the concept fall because > some >> > particular assertion is controversial. >> > >> > I consider MPI_NO_EAGER_THROTTLE to be the single most valuable >> > assertion for MPI 2.2 because it is needed to allow MPI to scale to > the >> > levels we are already seeing. >> > >> > >> > Dick Treumann - MPI Team/TCEM >> > IBM Systems & Technology Group >> > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 >> > Tele (845) 433-7846 Fax (845) 433-8363 >> > >> > >> > mpi3-subsetting-bounces_at_[hidden] wrote on 06/20/2008 >> > 02:58:41 AM: >> > >>> > > Dear Dick, >>> > > >>> > > A couple of suggestions re your proposal: >>> > > >>> > > - If ASSERTIONS is put at the end of the MPI_INIT_ASSERTED >> > argument >>> > > list, in C++ one can declare the last argument as having a >> > zero >>> > > default value, and skip it if necessary. This might help with >>> > > deprecation of the earlier MPI_INIT_* calls. >> > >> > I have no objection. It seems reasonable to let C++ default the >> > assertions parameter to "none" >> > >>> > > - In non-Cray parts of the world, an MPI_INT followed by >> > MPI_FLOAT >>> > > is likely to be a 4-byte int followed by a 4-byte float. This >>> > > sometimes depends on the compiler settings in effect, too. >> > >> > My rationale is not specific to any particular architecture. >> > Some MPI datatypes are made entirely >> > from the same base type. Some are mixtures of types. If libmpi >> > knows >> > at the moment a datatype is committed that the send side and >> > receive >> > side will always use the same internal representions then it >> > does not >> > need to keep track of the fact that one instance of >> > {MPI_INT,MPI_FLOAT} >> > has two distinct parts. The send side can gather and ship 8 >> > bytes >> > and the receive side can scatter the 8 bytes. If one side might >> > use 4 >> > byte integers while the other side uses 8 byte integers then at >> > least one side will need to know there is a conversion to be >> > done for >> > the MPI_INT part. If an MPI job does a spawn or join that links >> > to a >> > different architecture after the datatype has been committed, >> > and >> > the MPI_Type_commit has discarded the details, it is too late to >> > get >> > them back. On the other hand, if it is known there will never >> > be a >> > different architecture added to the job, the extra information >> > can be >> > safely discarded. >> > >>> > > - I don't think MPI_NO_THREAD_CONTENTION is really necessary. >> > The >>> > > original thread level settings, in particular, the use of >> > anything >>> > > but MPI_THREAD_MULTIPLE, seem to capture the semantics that >> > you proposed. >> > >> > This one is kind of tricky and I also am not sure what it would >> > mean. If >> > we find a clear value we can keep it and if not we can remove >> > it. >> > >>> > > - I can't fully follow the motivation for MPI_NO_ANY_SOURCE >>> > > deprioritization. AFAIK, a rendezvous exchange usually starts >> > with a >>> > > ready-to-send packet that contains the size of the message. In >> > this >>> > > case the receiving side will normally reply with a >> > ready-to-receive >>> > > regardless of the buffer space available, and flag >> > MPI_ERR_TRUNCATED >>> > > on message arrival if necessary. In this case, neither >>> > > MPI_ANY_SOURCE not MPI_NO_ANY_SOURCE seem to get into way. >> > >> > My point is that MPI_NO_ANY_SOURCE might allow this round trip >> > protocol to be replaced by a 1/2 rendezvous protocol. If it is >> > known >> > that MPI_ANY_SOURCE will not be used then the receive side can >> > send >> > an "envelop and ready for data" packet to the send side. As long >> > as >> > the send side knows it will receive the "envelop and ready for >> > data" >> > packet when the receive is posted, it does not need to do the >> > first 1/2 >> > of the rendezvous. The message matching can be done at the send >> > side. >> > >> > A send for which the receive was preposted has a >> > good chance of finding the "envelop and ready for data" sitting >> > in >> > an early queue and the large send can avoid any rendezvous >> > delay. >> > Data begins to flow immediately vs waiting for a round trip of a >> > >> > full rendezvous. In many cases we cut the delay in half and best >> > >> > case we eliminate rendezvous delay completely. If the receive >> > side >> > is late in posting the receive we still save a packet traversal >> > but >> > do not save any time. >> > >> > If there may be an MPI_ANY_SOURCE then this does not work >> > because the >> > receive side that has an MPI_ANY_SOURCE cannot guess which >> > sender to >> > notify so the sender cannot count on getting a 1/2 rendezvous >> > notification for a message that should match the MPI_ANY_SOURCE >> > receive. >> > >> > The problem that made me lower the priority is that many MPIs >> > use an >> > eager protocol for small messages and a rendezvous protocol for >> > large >> > messages. If the send side and receive side have the same size >> > buffer >> > then both sides can reach the same conclusion: eager vs 1/2 >> > rendezvous. >> > If both decide on eager, the receive side will not send an >> > "envelop and ready for data" packet and the send side will not >> > look >> > for one. If both sides decide on 1/2 rendezvous then the receive >> > side >> > will send an "envelop and ready for data" packet and the send >> > side will >> > look for and consume the notice. If the send side is for an 8 >> > byte >> > message and the receive uses a "big enough" receive buffer of >> > 64KB >> > then the two sides will probably not be able to reach the same >> > conclusion about the protocol. The receive side will ship off an >> > "envelop and ready for data" packet that the send side will not >> > know what to do with. >> > >> > >>> > > >>> > > Best regards. >>> > > >>> > > Alexander >>> > > >>> > > From: Supalov, Alexander >>> > > Sent: Friday, June 20, 2008 8:29 AM >>> > > To: 'MPI 3.0 Sub-setting working group' >>> > > Subject: RE: [Mpi3-subsetting] MPI subsetting: charting the >> > way >>> > > forward at atelecon next week? >> > >>> > > Dear Dick, >>> > > >>> > > Thank you. I remember we exchanged a couple of emails about >> > the >>> > > possible extensions to the set of assertions, like one-sided >> > and >>> > > I/O, and in my recollection, almost reached an agreement that >> > this >>> > > can improve performance and possibly memory footprint, as well >> > as be >>> > > expressed thru assertions. Do you still feel favorable about >> > this? >>> > > >>> > > Best regards. >>> > > >>> > > Alexander >>> > > >> > >> > >> > >> > ________________________________ >> > >> > _______________________________________________ >> > mpi3-subsetting mailing list >> > mpi3-subsetting_at_[hidden] >> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting >> > >> > >> > >> > >> > --------------------------------------------------------------------- >> > Intel GmbH >> > Dornacher Strasse 1 >> > 85622 Feldkirchen/Muenchen Germany >> > Sitz der Gesellschaft: Feldkirchen bei Muenchen >> > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer >> > Registergericht: Muenchen HRB 47456 Ust.-IdNr. >> > VAT Registration No.: DE129385895 >> > Citibank Frankfurt (BLZ 502 109 00) 600119052 >> > >> > This e-mail and any attachments may contain confidential material for >> > the sole use of the intended recipient(s). Any review or distribution >> > by others is strictly prohibited. If you are not the intended >> > recipient, please contact the sender and delete all copies. >> > > _______________________________________________ > mpi3-subsetting mailing list > mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > > > _______________________________________________ > mpi3-subsetting mailing list > mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > * -------------- next part -------------- An HTML attachment was scrubbed... URL: From schulzm at [hidden] Fri Jun 20 12:42:34 2008 From: schulzm at [hidden] (Martin Schulz) Date: Fri, 20 Jun 2008 10:42:34 -0700 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? In-Reply-To: Message-ID: <200806201742.m5KHgaZc029822@mail-1.llnl.gov> At 10:12 AM 6/20/2008, Richard Treumann wrote: >Of course - > >An assertion is a statement that the application does not require >some MPI standard feature or semantic guarantee. It is not a >directive to the MPI implementation. The implementation is free to >provide that feature or guarantee even if the application says it is >not needed. > >The story is a bit different for a helper library because the >library is logically a more or less opaque part of the application. >If the caller of MPI_INIT_ASSERTED says the application does not >require something and then uses a library that does require that >feature or guarantee - the library must raise an error or adjust its >behavior to live within the assertion. While I understand the intention, am a bit worried about the implications. Imagine a long running application that loads a new physics module after hours of computation and this new module requires different assertions. If the only option it has is to fail, a lot of progress in the main application will get lost. The only alternative is for the application to never state any assertion since it does not know which modules it will load (often depends on the input deck) - and then the whole exercise will be pointless anyway. Also, tools often need very specific MPI features and can't live with many assertions (just taking from Rich's examples in the other emails: there is often a need to use wildcards or complex datatypes even if the application doesn't use it). Not being able to change this would basically eliminate the ability to use certain tools on certain applications (I realize that tools that run from the beginning can intercept MPI_Init and turn assertions off, but tools that attach can't). Hence, I think we need some way to change assertions at runtime (at very specific points only and as global operations) despite the extra complexity it will introduce. Martin >Dick Treumann - MPI Team/TCEM >IBM Systems & Technology Group >Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 >Tele (845) 433-7846 Fax (845) 433-8363 > > >mpi3-subsetting-bounces_at_[hidden] wrote on 06/20/2008 12:58:03 PM: > > > Hi, > > > > Ignoring an assertion should be perfectly legal. > > > > Best regards. > > >_______________________________________________ >mpi3-subsetting mailing list >mpi3-subsetting_at_[hidden] >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting _______________________________________________________________________ Martin Schulz, schulzm_at_[hidden], http://people.llnl.gov/schulz6 CASC @ Lawrence Livermore National Laboratory, Livermore, USA * -------------- next part -------------- An HTML attachment was scrubbed... URL: From treumann at [hidden] Fri Jun 20 12:45:40 2008 From: treumann at [hidden] (Richard Treumann) Date: Fri, 20 Jun 2008 13:45:40 -0400 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? In-Reply-To: Message-ID: Since an assertion is a statement about the application, made by the application, I see no need for a query system that preceeds making the assertion. If the application author is confident enough about the way his application works to make one of the few predefined assertions, it can never do any harm to make the assertion. If he wants to experiment, he can use the query function within the application to find out if the assertion he made is being exploited by the MPI implementation. If the query function says the assertion is not being exploited then it is harmless. If the query function says it is being exploited then the application author can try his application with and without the assertion to see if the payoff is noticable. If he is the cautious type he can put in a query and issue a warning for any MPI implementation that exploits the assertion. The warning would say "Additional testing may be required". When he changes or upgrades his MPI and the new MPI implementation does exploit the assertion he will be warned. I think if there are more than a very few assertions available, the third party libraries will simply take the defensive actions of disallowing all assertions. If there are a very few will defined assertions, third party libraries can decide which ones to allow. If there are a small number of predefined assertion there is no reason the standard could not say which bit each one uses. If the standard said MPI_NO_EAGER_THROTTLE was to be 0x01 there would be no ABI issue. Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi3-subsetting-bounces_at_[hidden] wrote on 06/20/2008 01:05:03 PM: > > Yes, but the best approach would be a query/subscribe > interface, possibly with some set of standard ketwords > that provide portability. > * -------------- next part -------------- An HTML attachment was scrubbed... URL: From treumann at [hidden] Fri Jun 20 12:55:22 2008 From: treumann at [hidden] (Richard Treumann) Date: Fri, 20 Jun 2008 13:55:22 -0400 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? In-Reply-To: <200806201710.m5KHAnUA007595@mail-1.llnl.gov> Message-ID: Hi Martin Assertions are not MPI implementation specific. There are features of the MPI standard and semantic guarantees of the MPI standard that a few applications depend on and many do not. Some of these standard MPI features are costly in memory footprint or damage performance. A program author will be able to use a predefined assertion to declare that this application does not require a specific feature. The MPI implementation is then free to use optimizations that are not allowable if the feature or guarantee must be maintained just in case it is needed. It may not be simple to explain the meaning of each assertion but the meaning is 100% rooted in the MPI standard and is not different from one implementation to the next. Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 > ............. Also, if these constants > are really implementation specific, does it make sense to have > them in the MPI standard? Each vender will want their own set > (and rightfully so) and the burden is then on the programmer to > know all of the different options and understand the subtle > differences (and we have to document them all in the standard). > > Martin > * -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlgraham at [hidden] Fri Jun 20 13:07:52 2008 From: rlgraham at [hidden] (Richard Graham) Date: Fri, 20 Jun 2008 14:07:52 -0400 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? In-Reply-To: Message-ID: We may be talking across past each other - MPI_NO_EAGER_THROTTLE seems to me to be a statement about the MPI implementation, not the application, and thus does not fit the test of being an assertion on the application. I guess you could say that the application is asking for the library to respect this request, but this assumes this means something to all implementations. If we are going to put things into the standard that mean something in some implementations, and not in others, we have to have a way for every implementation to add assertions that are meaningful to them, and can¹t just pick a set that fits a sub-set of implementations, and as such need mechanisms for adding new parameters, and querying which ones are supported. Rich On 6/20/08 1:45 PM, "Richard Treumann" wrote: > Since an assertion is a statement about the application, made by the > application, I see no need for a query system that preceeds making the > assertion. If the application author is confident enough about the way his > application works to make one of the few predefined assertions, it can never > do any harm to make the assertion. > > If he wants to experiment, he can use the query function within the > application to find out if the assertion he made is being exploited by the MPI > implementation. If the query function says the assertion is not being > exploited then it is harmless. If the query function says it is being > exploited then the application author can try his application with and without > the assertion to see if the payoff is noticable. If he is the cautious type he > can put in a query and issue a warning for any MPI implementation that > exploits the assertion. The warning would say "Additional testing may be > required". When he changes or upgrades his MPI and the new MPI implementation > does exploit the assertion he will be warned. > > I think if there are more than a very few assertions available, the third > party libraries will simply take the defensive actions of disallowing all > assertions. If there are a very few will defined assertions, third party > libraries can decide which ones to allow. > > If there are a small number of predefined assertion there is no reason the > standard could not say which bit each one uses. If the standard said > MPI_NO_EAGER_THROTTLE was to be 0x01 there would be no ABI issue. > > > Dick Treumann - MPI Team/TCEM > IBM Systems & Technology Group > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > > mpi3-subsetting-bounces_at_[hidden] wrote on 06/20/2008 01:05:03 PM: > >> > >> > Yes, but the best approach would be a query/subscribe >> > interface, possibly with some set of standard ketwords >> > that provide portability. >> > > > > _______________________________________________ > mpi3-subsetting mailing list > mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting * -------------- next part -------------- An HTML attachment was scrubbed... URL: From treumann at [hidden] Fri Jun 20 13:11:02 2008 From: treumann at [hidden] (Richard Treumann) Date: Fri, 20 Jun 2008 14:11:02 -0400 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? In-Reply-To: <200806201742.m5KHgaZc029822@mail-1.llnl.gov> Message-ID: If an application has unpredictable behavior then it should not make assertions. The author of an application that loads arbitrary helpers at run time (like the "new physics module") should know that he cannot safely make assertions that cover those helpers. No application or helper can possibly "require an assertion". Zero assertions is valid for any correct MPI program. I think almost all the value of assertions is lost if the MPI implementation must be prepared at all times to have any assertion revoked. Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi3-subsetting-bounces_at_[hidden] wrote on 06/20/2008 01:42:34 PM: > At 10:12 AM 6/20/2008, Richard Treumann wrote: > Of course - > > An assertion is a statement that the application does not require > some MPI standard feature or semantic guarantee. It is not a > directive to the MPI implementation. The implementation is free to > provide that feature or guarantee even if the application says it is > not needed. > > The story is a bit different for a helper library because the > library is logically a more or less opaque part of the application. > If the caller of MPI_INIT_ASSERTED says the application does not > require something and then uses a library that does require that > feature or guarantee - the library must raise an error or adjust its > behavior to live within the assertion. > > While I understand the intention, am a bit worried about the > implications. Imagine a long running application that loads > a new physics module after hours of computation and this new > module requires different assertions. If the only option it > has is to fail, a lot of progress in the main application will > get lost. The only alternative is for the application to never > state any assertion since it does not know which modules it > will load (often depends on the input deck) - and then the > whole exercise will be pointless anyway. > > Also, tools often need very specific MPI features and can't live > with many assertions (just taking from Rich's examples in the > other emails: there is often a need to use wildcards or complex > datatypes even if the application doesn't use it). Not being > able to change this would basically eliminate the ability to > use certain tools on certain applications (I realize that tools > that run from the beginning can intercept MPI_Init and turn > assertions off, but tools that attach can't). > > Hence, I think we need some way to change assertions at runtime > (at very specific points only and as global operations) despite > the extra complexity it will introduce. > > Martin > > > > > > Dick Treumann - MPI Team/TCEM > IBM Systems & Technology Group > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > > mpi3-subsetting-bounces_at_[hidden] wrote on 06/20/2008 12:58:03 PM: > > > Hi, > > > > Ignoring an assertion should be perfectly legal. > > > > Best regards. > > > _______________________________________________ > mpi3-subsetting mailing list > mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > _______________________________________________________________________ > Martin Schulz, schulzm_at_[hidden], http://people.llnl.gov/schulz6 > CASC @ Lawrence Livermore National Laboratory, Livermore, USA > _______________________________________________ > mpi3-subsetting mailing list > mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting * -------------- next part -------------- An HTML attachment was scrubbed... URL: From treumann at [hidden] Fri Jun 20 13:34:58 2008 From: treumann at [hidden] (Richard Treumann) Date: Fri, 20 Jun 2008 14:34:58 -0400 Subject: [Mpi3-subsetting] MPI subsetting: charting the way forward atatelecon next week? In-Reply-To: Message-ID: I am not completely happy with the name format I have suggested for assertions. I would be glad to hear suggestions. Here is how I would cast each assertion. MPI_NO_EAGER_THROTTLE: This program does not depend on libmpi to provide an EAGER_THROTTLE semantic MPI_NO_REDUCTION_DETERMINISM This program does not depend on libmpi to provide a REDUCTION_DETERMINISM semantic MPI_NO_DATATYPE_HETEROGENEITY This program does not depend on libmpi to provide support for DATATYPE_HETEROGENEITY in communication MPI_NO_SEND_CANCEL This program does not depend on libmpi to provide a SEND_CANCEL semantic MPI_NO_ANY_SOURCE This program does not depend on libmpi to provide an ANY_SOURCE semantic MPI_NO_MPI_IO This program does not depend on libmpi to provide MPI_IO support etc In each case we are talking about something the MPI standard says must be provided by a compilant MPI implementation, just in case the application needs it. For each case there are application sthat do not need it. Does this help? Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 From alexander.supalov at [hidden] Thu Jun 26 10:36:06 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Thu, 26 Jun 2008 16:36:06 +0100 Subject: [Mpi3-subsetting] call for a report-out and session chair Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2017CA973@swsmsx413.ger.corp.intel.com> Hi everybody, As I won't be at the meeting next week, we need a volunteer to report out to the Forum and then chair the subsetting WG meeting 5:15 pm - 7:00 pm on Tuesday, June 1. Who wants to do this? Please let me know. In the meantime, I'll prepare and upload a draft report-out, and update the current proposal using the materials of our recent meeting by EOW. Best regards. Alexander -- Dr Alexander Supalov Intel GmbH Hermuelheimer Strasse 8a 50321 Bruehl, Germany Phone: +49 2232 209034 Mobile: +49 173 511 8735 Fax: +49 2232 209029 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Fri Jun 27 07:55:57 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Fri, 27 Jun 2008 13:55:57 +0100 Subject: [Mpi3-subsetting] call for a report-out and session chair In-Reply-To: <[Mpi3-subsetting] call for a report-out and session chair> Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2017CAD71@swsmsx413.ger.corp.intel.com> Hi everybody, Rich Graham kindly agreed to chair the subsetting meeting for us. A draft report-out can be found in http://svn.mpi-forum.org/trac/mpi-forum-web/attachment/wiki/SubsettingWi kiPage/MPI-3%20subsetting%2C%20take%203.pdf . I have to use a change-protected PDF for this external publication. Please comment. I've also added last meeting minutes, and am setting about updating the proposal accordingly now. Best regards. Alexander ________________________________ From: Supalov, Alexander Sent: Thursday, June 26, 2008 5:36 PM To: 'MPI 3.0 Sub-setting working group' Subject: call for a report-out and session chair Hi everybody, As I won't be at the meeting next week, we need a volunteer to report out to the Forum and then chair the subsetting WG meeting 5:15 pm - 7:00 pm on Tuesday, June 1. Who wants to do this? Please let me know. In the meantime, I'll prepare and upload a draft report-out, and update the current proposal using the materials of our recent meeting by EOW. Best regards. Alexander -- Dr Alexander Supalov Intel GmbH Hermuelheimer Strasse 8a 50321 Bruehl, Germany Phone: +49 2232 209034 Mobile: +49 173 511 8735 Fax: +49 2232 209029 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From treumann at [hidden] Mon Jun 30 14:45:19 2008 From: treumann at [hidden] (Richard Treumann) Date: Mon, 30 Jun 2008 15:45:19 -0400 Subject: [Mpi3-subsetting] Sunset WG slides In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2017CAD71@swsmsx413.ger.corp.intel.com> Message-ID: I have read the slides and have some comments on the issues related to MPI_INIT time assertions (slide 4). 1) I agree the set of assertions must be negotiated but unanimous implementor acceptance should not be needed. Defining a assertion in the standard that does not help a particular MPI implementation is pretty harmless. Defining one that does not help any implementation is useless. Implementors are in a good position to recognize assertions that can never be useful and help kill them but no implementor should have veto power.. 2) I have not heard a decent rationale for implementation specific assertions. Assertions are not about the implementation. They are about a particular application and its requirements in relation to the MPI standard. The MPI implementation simply ignores any assertion that is not useful to optimizing that implementation. 3) a) Any tool that intercepts variants of MPI_INIT can remove assertions it does not like. Removing an assertion can never make an application quit working properly. b) I am not sure what "Layered underlying libraries" means but the observation on the slide does apply to what I would call "third party libraries". If some library that will be linked into an application (say a parallel math library) makes calls to MPI_Cancel for ISEND requests than that library should check that the MPI_NO_SEND_CANCEL assertion was not used and bail out if it was. Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 "Supalov, Alexander" "MPI 3.0 Sub-setting working group" Sent by: ounces_at_lists.mpi- cc forum.org Subject Re: [Mpi3-subsetting] call for a 06/27/2008 08:55 report-out and session chair AM Please respond to "MPI 3.0 Sub-setting working group" Hi everybody, Rich Graham kindly agreed to chair the subsetting meeting for us. A draft report-out can be found in http://svn.mpi-forum.org/trac/mpi-forum-web/attachment/wiki/SubsettingWikiPage/MPI-3%20subsetting%2C%20take%203.pdf . I have to use a change-protected PDF for this external publication. Please comment. I've also added last meeting minutes, and am setting about updating the proposal accordingly now. Best regards. Alexander From: Supalov, Alexander Sent: Thursday, June 26, 2008 5:36 PM To: 'MPI 3.0 Sub-setting working group' Subject: call for a report-out and session chair Hi everybody, As I won't be at the meeting next week, we need a volunteer to report out to the Forum and then chair the subsetting WG meeting 5:15 pm - 7:00 pm on Tuesday, June 1. Who wants to do this? Please let me know. In the meantime, I'll prepare and upload a draft report-out, and update the current proposal using the materials of our recent meeting by EOW. Best regards. Alexander -- Dr Alexander Supalov Intel GmbH Hermuelheimer Strasse 8a 50321 Bruehl, Germany Phone: +49 2232 209034 Mobile: +49 173 511 8735 Fax: +49 2232 209029 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. _______________________________________________ mpi3-subsetting mailing list mpi3-subsetting_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting * -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: application/octet-stream Size: 240 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic06815.gif Type: application/octet-stream Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: application/octet-stream Size: 45 bytes Desc: not available URL: