From treumann at [hidden] Mon May 5 12:18:12 2008 From: treumann at [hidden] (Richard Treumann) Date: Mon, 5 May 2008 13:18:12 -0400 Subject: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A20148EF7A@swsmsx413.ger.corp.intel.com> Message-ID: Hi Alexander I have no objection to citing my "assertions" proposal in the subsetting discussions. I do want to keep it clear that this proposal is intended to be as simple as practical to implement, exploit and live with. "Live with" applies to 3rd party library authors or anyone else who must write MPI code but does not know and control the structure of the entire application. That guy must "live with" the decisions made by whoever coded the MPI_INIT piece. "Live with" also applies to whoever must test or certify a specific MPI implementation. Thanks - Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 "Supalov, Alexander" "MPI 2.2" Sent by: mpi-22-bounces_at_li cc sts.mpi-forum.org Subject Re: [Mpi-22] mpi-22 Digest, Vol 2, 04/26/2008 04:03 Issue 7 AM Please respond to "MPI 2.2" Dear Dick, Thank you. Would you mind if I cite your proposal in the subsets discussion? Yours looks like a good alternative to the thinking of some of us that subsets might be very rich and mutable, and to Jeff's proposal on hints I've already cited there with his permission. Best regards. Alexander From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Thursday, April 24, 2008 6:16 PM To: MPI 2.2 Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 04/24/2008 11:33:42 AM: > Hi, > > Note that this is an argument for making the assertions optional: those > who don't care don't have to use them. Those who care should use them > correctly or else. As usual. > > Best regards. > > Alexander > Hi Alexander The assertions are optional in this proposal. If this is added to the MPI standard the minimal impacts (day one impacts) are: == To application writers (none) - MPI_INIT and MPI_INIT_THREAD still work. MPI_INIT_THREAD_xxx can be passed 0 (zero) as the assertions bit vector. To MPI Implementors (small) - subroutine MPI_INIT_THREAD_xxx can be a clone of MPI_INIT_THREAD under the covers. If the Forum decides the query function is for asking what assertions are being honored, the implementation can just return "none" to every query. If there is also a query for what assertions have been made then there are a few more lines of code the implementor must write to preserve the value so it can be returned(maybe 10 lines) Writers of opaque libraries (small) - call the query function at library init time and if any assertions are found, issue an error message and kill the job. This is awkward for a library that wants to support every MPI whether it has implemented the new query function or not. == As MPI implementations begin to take advantage of assertions there is more work for the MPI implementor and the library author must begin to think about whether his customer will be upset if the library simply outlaws all assertions. The library author will never be wrong if he simply forbids assertions forever. If they become valuable he will feel the pressure to work it out. The MPI implementor will never be wrong if he adds the API but simply ignores assertions forever. If they become valuable he will feel the pressure to honor some at least. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: application/octet-stream Size: 156 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic14139.gif Type: application/octet-stream Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: application/octet-stream Size: 45 bytes Desc: not available URL: From alexander.supalov at [hidden] Mon May 5 12:28:06 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Mon, 5 May 2008 18:28:06 +0100 Subject: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 In-Reply-To: Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2014F6386@swsmsx413.ger.corp.intel.com> Dear Dick, Thank you. We can actually introduce what you propose, possibly with a query function to make it still easier to live with, as early as in MPI 2.2, as a subset precursor. Judging by the discussion in Chicago, subsets may not need much more than that in the end, possibly with a little more flags and semantics added in MPI-3. The reservation against 32- (or for that matter, 64-) bit limitation is the only one I have at the moment. Not being able to attach assertions to communicators, etc. may be missed by some advanced programmers, but here we need to be pragmatic: who will ever want to go that deep? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Monday, May 05, 2008 7:18 PM To: MPI 2.2 Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 Hi Alexander I have no objection to citing my "assertions" proposal in the subsetting discussions. I do want to keep it clear that this proposal is intended to be as simple as practical to implement, exploit and live with. "Live with" applies to 3rd party library authors or anyone else who must write MPI code but does not know and control the structure of the entire application. That guy must "live with" the decisions made by whoever coded the MPI_INIT piece. "Live with" also applies to whoever must test or certify a specific MPI implementation. Thanks - Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 "Supalov, Alexander" "Supalov, Alexander" Sent by: mpi-22-bounces_at_[hidden] 04/26/2008 04:03 AM Please respond to "MPI 2.2" To "MPI 2.2" cc Subject Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 Dear Dick, Thank you. Would you mind if I cite your proposal in the subsets discussion? Yours looks like a good alternative to the thinking of some of us that subsets might be very rich and mutable, and to Jeff's proposal on hints I've already cited there with his permission. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [ mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Thursday, April 24, 2008 6:16 PM To: MPI 2.2 Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 04/24/2008 11:33:42 AM: > Hi, > > Note that this is an argument for making the assertions optional: those > who don't care don't have to use them. Those who care should use them > correctly or else. As usual. > > Best regards. > > Alexander > Hi Alexander The assertions are optional in this proposal. If this is added to the MPI standard the minimal impacts (day one impacts) are: == To application writers (none) - MPI_INIT and MPI_INIT_THREAD still work. MPI_INIT_THREAD_xxx can be passed 0 (zero) as the assertions bit vector. To MPI Implementors (small) - subroutine MPI_INIT_THREAD_xxx can be a clone of MPI_INIT_THREAD under the covers. If the Forum decides the query function is for asking what assertions are being honored, the implementation can just return "none" to every query. If there is also a query for what assertions have been made then there are a few more lines of code the implementor must write to preserve the value so it can be returned(maybe 10 lines) Writers of opaque libraries (small) - call the query function at library init time and if any assertions are found, issue an error message and kill the job. This is awkward for a library that wants to support every MPI whether it has implemented the new query function or not. == As MPI implementations begin to take advantage of assertions there is more work for the MPI implementor and the library author must begin to think about whether his customer will be upset if the library simply outlaws all assertions. The library author will never be wrong if he simply forbids assertions forever. If they become valuable he will feel the pressure to work it out. The MPI implementor will never be wrong if he adds the API but simply ignores assertions forever. If they become valuable he will feel the pressure to honor some at least. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: application/octet-stream Size: 230 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: application/octet-stream Size: 45 bytes Desc: not available URL: From jsquyres at [hidden] Mon May 5 12:31:06 2008 From: jsquyres at [hidden] (Jeff Squyres) Date: Mon, 5 May 2008 13:31:06 -0400 Subject: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2014F6386@swsmsx413.ger.corp.intel.com> Message-ID: <2B520D06-F1FB-4AD6-B7AB-C9301C92E4C6@cisco.com> On May 5, 2008, at 1:28 PM, Supalov, Alexander wrote: > Thank you. We can actually introduce what you propose, possibly with > a query function to make it still easier to live with, as early as > in MPI 2.2, as a subset precursor. Judging by the discussion in > Chicago, subsets may not need much more than that in the end, > possibly with a little more flags and semantics added in MPI-3. > > The reservation against 32- (or for that matter, 64-) bit limitation > is the only one I have at the moment. Not being able to attach > assertions to communicators, etc. may be missed by some advanced > programmers, but here we need to be pragmatic: who will ever want to > go that deep? Libraries that build on top of MPI may want to. This was one of the prime motivations for communications, right? (private/safe communications) As Quincy stated in Chicago and on the mailing list (IIRC), what if a top-level app sets an assertion that a library using MPI can't live with? Being able to set them per-communicator might alleviate that issue. -- Jeff Squyres Cisco Systems From alexander.supalov at [hidden] Mon May 5 12:53:49 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Mon, 5 May 2008 18:53:49 +0100 Subject: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 In-Reply-To: <2B520D06-F1FB-4AD6-B7AB-C9301C92E4C6@cisco.com> Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2014F6393@swsmsx413.ger.corp.intel.com> Hi, We may want to see whether communicator-bound restriction will help down there. For example, even one communicator with dynamic processes enabled may "taint" the progress engine enough to make any saving from the absence of one-sided elsewhere disappear. On the other hand, attaching attributes to the MPI_COMM_WORLD, although not fully clean and properly timed, might be a way to spread the assertions across the job, and let libraries cut their proper part out. Same with windows and files, by the way. How and when will this pass down to the progress engine? I don't know. Do you? Best regards. Alexander -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Jeff Squyres Sent: Monday, May 05, 2008 7:31 PM To: MPI 2.2 Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 On May 5, 2008, at 1:28 PM, Supalov, Alexander wrote: > Thank you. We can actually introduce what you propose, possibly with > a query function to make it still easier to live with, as early as > in MPI 2.2, as a subset precursor. Judging by the discussion in > Chicago, subsets may not need much more than that in the end, > possibly with a little more flags and semantics added in MPI-3. > > The reservation against 32- (or for that matter, 64-) bit limitation > is the only one I have at the moment. Not being able to attach > assertions to communicators, etc. may be missed by some advanced > programmers, but here we need to be pragmatic: who will ever want to > go that deep? Libraries that build on top of MPI may want to. This was one of the prime motivations for communications, right? (private/safe communications) As Quincy stated in Chicago and on the mailing list (IIRC), what if a top-level app sets an assertion that a library using MPI can't live with? Being able to set them per-communicator might alleviate that issue. -- Jeff Squyres Cisco Systems _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. From alexander.supalov at [hidden] Mon May 5 13:02:01 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Mon, 5 May 2008 19:02:01 +0100 Subject: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2014F6393@swsmsx413.ger.corp.intel.com> Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2014F6397@swsmsx413.ger.corp.intel.com> Woops... Read "dynamic processes" rather than "one-sided". Partial update, sorry. -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Monday, May 05, 2008 7:54 PM To: MPI 2.2 Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 Hi, We may want to see whether communicator-bound restriction will help down there. For example, even one communicator with dynamic processes enabled may "taint" the progress engine enough to make any saving from the absence of one-sided elsewhere disappear. On the other hand, attaching attributes to the MPI_COMM_WORLD, although not fully clean and properly timed, might be a way to spread the assertions across the job, and let libraries cut their proper part out. Same with windows and files, by the way. How and when will this pass down to the progress engine? I don't know. Do you? Best regards. Alexander -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Jeff Squyres Sent: Monday, May 05, 2008 7:31 PM To: MPI 2.2 Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 On May 5, 2008, at 1:28 PM, Supalov, Alexander wrote: > Thank you. We can actually introduce what you propose, possibly with > a query function to make it still easier to live with, as early as > in MPI 2.2, as a subset precursor. Judging by the discussion in > Chicago, subsets may not need much more than that in the end, > possibly with a little more flags and semantics added in MPI-3. > > The reservation against 32- (or for that matter, 64-) bit limitation > is the only one I have at the moment. Not being able to attach > assertions to communicators, etc. may be missed by some advanced > programmers, but here we need to be pragmatic: who will ever want to > go that deep? Libraries that build on top of MPI may want to. This was one of the prime motivations for communications, right? (private/safe communications) As Quincy stated in Chicago and on the mailing list (IIRC), what if a top-level app sets an assertion that a library using MPI can't live with? Being able to set them per-communicator might alleviate that issue. -- Jeff Squyres Cisco Systems _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. From thakur at [hidden] Mon May 5 15:36:01 2008 From: thakur at [hidden] (Rajeev Thakur) Date: Mon, 5 May 2008 15:36:01 -0500 Subject: [Mpi-22] Call for MPI 2.2 agenda items Message-ID: <005701c8aeef$a20d3350$860add8c@mcs.anl.gov> Now that MPI 2.1 is close to completion, it is time to look at issues for MPI 2.2. Please send any items you may have for 2.2 to mpi-22_at_[hidden] and/or upload them to the wiki (and let us know). I had made a note of the following items from the last meeting that were deferred to 2.2: * support for long long in maxloc, minloc * fix meaning of first two sentences in intercommunicator barrier * Ex 5.8 uses old style MPI_UB * Many functions say "integer" instead of "nonnegative integer" * const in C++ bindings * IN, OUT, and INOUT handle types broken * MPI_CHAR, MPI::CHAR should be signed char or char? * Need 64-bit sizes for count in messages * Sizes of MPI_Aint and MPI_Offset should not be different * New predefined types MPI_AINT and MPI_OFFSET * New predefined types corresponding to new types in C99, Fortran 2003, and C++ some year From treumann at [hidden] Mon May 5 16:53:05 2008 From: treumann at [hidden] (Richard Treumann) Date: Mon, 5 May 2008 17:53:05 -0400 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2014F6386@swsmsx413.ger.corp.intel.com> Message-ID: I changed the subject to be meaningful - it had been "Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7" I do think this proposal is within the scope of the MPI 2.2 rules so I am pleased to hear this is being considered. I am 100% convinced that a query function should be part of this proposal, probably with 2 query flavors. One flavor of the query would respond with the set of assertions the application MPI_INIT_xxx call had provided and another would respond with the set of assertions the MPI implementation is actually exploiting. One library author may decide he will use if/else logic based on how the MPI implementation will behave. The application may assert MPI_NO_yyy but if the MPI implementation does not exploit MPI_NO_yyy then the library can still use the code that depends on yyy. Another library author may depend unconditionally on yyy. On an MPI implementation that does not exploit MPI_NO_yyy he may want to issue a warning that assertion MPI_NO_yyy is non-portable but let the job run. On one that does exploit MPI_NO_yyy he would abort the job. Or, he may decide to simply abort any job that asserts MPI_NO_yyy to avoid having the library suddenly quit working when the customer upgrades to an MPI implementation that does exploit MPI_NO_yyy. I think 32 assertions is probably more than enough Even a handful of defined assertions can raise testing costs. For simple cases like MPI_NO_ANY_SOURCE it is easy for a user to judge whether a piece of code is OK but if we begin to add subtle or narrow semantic assertions it gets harder. Some library providers will be tempted to say "I cannot proof read and test my code to a degree that will allow me to accept 28 specific assertions and forbid 4. I will simply forbid every subtle assertion I cannot afford to test." I predict that for large applications developed by teams and for community open source efforts the design leads will consider requiring that all parts be written to live within a few carefully chosen assertions. The design leads will not want to provide a list of 26 subtle or narrow assertions and require that everyone respect all 26 in the code they contribute. Many distinct assertions also could become a big test cost for MPI implementors and customers who must qualify an MPI implementation before trusting their business to it. If there were 64 or more assertions, how would a tester decide what combinations of assertions must be tested and then create suitable test cases? Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/05/2008 01:28:06 PM: > Dear Dick, > > Thank you. We can actually introduce what you propose, possibly with > a query function to make it still easier to live with, as early as > in MPI 2.2, as a subset precursor. Judging by the discussion in > Chicago, subsets may not need much more than that in the end, > possibly with a little more flags and semantics added in MPI-3. > > The reservation against 32- (or for that matter, 64-) bit limitation > is the only one I have at the moment. Not being able to attach > assertions to communicators, etc. may be missed by some advanced > programmers, but here we need to be pragmatic: who will ever want to > go that deep? > > Best regards. > > Alexander > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Richard Treumann > Sent: Monday, May 05, 2008 7:18 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > Hi Alexander > > I have no objection to citing my "assertions" proposal in the > subsetting discussions. I do want to keep it clear that this > proposal is intended to be as simple as practical to implement, exploit and > live with. > > "Live with" applies to 3rd party library authors or anyone else who > must write MPI code but does not know and control the structure of > the entire application. That guy must "live with" the decisions made > by whoever coded the MPI_INIT piece. "Live with" also applies to > whoever must test or certify a specific MPI implementation. > > Thanks - Dick > > Dick Treumann - MPI Team/TCEM > IBM Systems & Technology Group > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > [image removed] "Supalov, Alexander" > > > "Supalov, Alexander" > Sent by: mpi-22-bounces_at_[hidden] > 04/26/2008 04:03 AM > > Please respond to > "MPI 2.2" > > [image removed] > To > > [image removed] > "MPI 2.2" > > [image removed] > cc > > [image removed] > > [image removed] > Subject > > [image removed] > Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > > [image removed] > > [image removed] > > > Dear Dick, > > Thank you. Would you mind if I cite your proposal in the subsets > discussion? Yours looks like a good alternative to the thinking of > some of us that subsets might be very rich and mutable, and to > Jeff's proposal on hints I've already cited there with his permission. > > Best regards. > > Alexander > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Richard Treumann > Sent: Thursday, April 24, 2008 6:16 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > Dick Treumann - MPI Team/TCEM > IBM Systems & Technology Group > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > > mpi-22-bounces_at_[hidden] wrote on 04/24/2008 11:33:42 AM: > > > Hi, > > > > Note that this is an argument for making the assertions optional: those > > who don't care don't have to use them. Those who care should use them > > correctly or else. As usual. > > > > Best regards. > > > > Alexander > > > > Hi Alexander > > The assertions are optional in this proposal. If this is added to > the MPI standard the minimal impacts (day one impacts) are: > > == > To application writers (none) - MPI_INIT and MPI_INIT_THREAD still > work. MPI_INIT_THREAD_xxx can be > passed 0 (zero) as the assertions bit vector. > > To MPI Implementors (small) - subroutine MPI_INIT_THREAD_xxx can be > a clone of MPI_INIT_THREAD under the covers. If the Forum decides > the query function is for asking what assertions are being honored, > the implementation can just return "none" to every query. If there > is also a query for what assertions have been made then there are a > few more lines of code the implementor must write to preserve the > value so it can be returned(maybe 10 lines) > > Writers of opaque libraries (small) - call the query function at > library init time and if any assertions are found, issue an error > message and kill the job. This is awkward for a library that wants > to support every MPI whether it has implemented the new query function or not. > == > > As MPI implementations begin to take advantage of assertions there > is more work for the MPI implementor and the library author must > begin to think about whether his customer will be upset if the > library simply outlaws all assertions. > > The library author will never be wrong if he simply forbids > assertions forever. If they become valuable he will feel the > pressure to work it out. > > The MPI implementor will never be wrong if he adds the API but > simply ignores assertions forever. If they become valuable he will > feel the pressure to honor some at least. > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From Dries.Kimpe at [hidden] Mon May 5 17:23:06 2008 From: Dries.Kimpe at [hidden] (Dries Kimpe) Date: Tue, 6 May 2008 00:23:06 +0200 Subject: [Mpi-22] Call for MPI 2.2 agenda items In-Reply-To: <005701c8aeef$a20d3350$860add8c@mcs.anl.gov> Message-ID: <20080505222306.GA5295@mhdmobile.lan> * Rajeev Thakur [2008-05-05 15:36:01]: > * MPI_CHAR, MPI::CHAR should be signed char or char? In C and C++, char, signed char and unsigned char are all distinct types. (And in C++, wchar_t is yet another type; In C it is a typedef and thus not a new type but an alias) Char can either be signed or unsigned and this is platform/compiler dependent. So MPI_CHAR and MPI::CHAR should be whatever the compiler/platform thinks and cannot be fixed in the standard to be signed or unsigned. Dries * -------------- next part -------------- A non-text attachment was scrubbed... Name: 01-part Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From alexander.supalov at [hidden] Tue May 6 06:26:03 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Tue, 6 May 2008 12:26:03 +0100 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2014F678D@swsmsx413.ger.corp.intel.com> Dear Dick, Thanks. What about passing more verbose information to and from an MPI implementation? Say, eager threshold value, if it's meaningful? There are many less controversial examples, for example, to let the caller know how many processes are placed onto the node, etc. Simple assertions won't work in this case. Back to assertions. I thought we might consider MPI_Initialized_subset(*flag,required,*provided) or something. Setting "required" to 0 would return the set of provided assertions in "provided". Setting "required" to something else will also set the "flag" to true or false, depending on whether there's a match. The special MPI_Init_subset(*argc,***argv,required,*provided) would complement this. Regarding 32 bits, let's count (I'm taking the joint list from the subset proposal that includes your earlier points): 0,1,2,3 - MPI_THREAD_SINGLE, FUNNELED, SERIALIZED, MULTIPLE 0x00000004 - No dynamic process support 0x00000008 - No file I/O 0x00000010 - No one-sided ops 0x00000020 - No communicator & group management 0x00000040 - No non-blocking communication 0x00000080 - No message cancellation 0x00000100 - Persistent ops on both sides 0x00000200 - No heterogeneity 0x00000400 - No derived datatypes (especially those with holes) 0x00000800 - No MPI_ANY_SOURCE 0x00001000 - No message tagging 0x00002000 - No reduction order 0x00004000 - No eager buffering 0x00008000 - No mixed request types in wait/test We have 16 bits left. Note that there may be some predefined masks for well defined combinations of the above (e.g., MPI-1 = no dynamic processes, no file I/O, no threads, etc.). Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Monday, May 05, 2008 11:53 PM To: MPI 2.2 Subject: [Mpi-22] MPI_INIT assertions I changed the subject to be meaningful - it had been "Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7" I do think this proposal is within the scope of the MPI 2.2 rules so I am pleased to hear this is being considered. I am 100% convinced that a query function should be part of this proposal, probably with 2 query flavors. One flavor of the query would respond with the set of assertions the application MPI_INIT_xxx call had provided and another would respond with the set of assertions the MPI implementation is actually exploiting. One library author may decide he will use if/else logic based on how the MPI implementation will behave. The application may assert MPI_NO_yyy but if the MPI implementation does not exploit MPI_NO_yyy then the library can still use the code that depends on yyy. Another library author may depend unconditionally on yyy. On an MPI implementation that does not exploit MPI_NO_yyy he may want to issue a warning that assertion MPI_NO_yyy is non-portable but let the job run. On one that does exploit MPI_NO_yyy he would abort the job. Or, he may decide to simply abort any job that asserts MPI_NO_yyy to avoid having the library suddenly quit working when the customer upgrades to an MPI implementation that does exploit MPI_NO_yyy. I think 32 assertions is probably more than enough Even a handful of defined assertions can raise testing costs. For simple cases like MPI_NO_ANY_SOURCE it is easy for a user to judge whether a piece of code is OK but if we begin to add subtle or narrow semantic assertions it gets harder. Some library providers will be tempted to say "I cannot proof read and test my code to a degree that will allow me to accept 28 specific assertions and forbid 4. I will simply forbid every subtle assertion I cannot afford to test." I predict that for large applications developed by teams and for community open source efforts the design leads will consider requiring that all parts be written to live within a few carefully chosen assertions. The design leads will not want to provide a list of 26 subtle or narrow assertions and require that everyone respect all 26 in the code they contribute. Many distinct assertions also could become a big test cost for MPI implementors and customers who must qualify an MPI implementation before trusting their business to it. If there were 64 or more assertions, how would a tester decide what combinations of assertions must be tested and then create suitable test cases? Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/05/2008 01:28:06 PM: > Dear Dick, > > Thank you. We can actually introduce what you propose, possibly with > a query function to make it still easier to live with, as early as > in MPI 2.2, as a subset precursor. Judging by the discussion in > Chicago, subsets may not need much more than that in the end, > possibly with a little more flags and semantics added in MPI-3. > > The reservation against 32- (or for that matter, 64-) bit limitation > is the only one I have at the moment. Not being able to attach > assertions to communicators, etc. may be missed by some advanced > programmers, but here we need to be pragmatic: who will ever want to > go that deep? > > Best regards. > > Alexander > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Richard Treumann > Sent: Monday, May 05, 2008 7:18 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > Hi Alexander > > I have no objection to citing my "assertions" proposal in the > subsetting discussions. I do want to keep it clear that this > proposal is intended to be as simple as practical to implement, exploit and > live with. > > "Live with" applies to 3rd party library authors or anyone else who > must write MPI code but does not know and control the structure of > the entire application. That guy must "live with" the decisions made > by whoever coded the MPI_INIT piece. "Live with" also applies to > whoever must test or certify a specific MPI implementation. > > Thanks - Dick > > Dick Treumann - MPI Team/TCEM > IBM Systems & Technology Group > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > [image removed] "Supalov, Alexander" > > > "Supalov, Alexander" > Sent by: mpi-22-bounces_at_[hidden] > 04/26/2008 04:03 AM > > Please respond to > "MPI 2.2" > > [image removed] > To > > [image removed] > "MPI 2.2" > > [image removed] > cc > > [image removed] > > [image removed] > Subject > > [image removed] > Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > > [image removed] > > [image removed] > > > Dear Dick, > > Thank you. Would you mind if I cite your proposal in the subsets > discussion? Yours looks like a good alternative to the thinking of > some of us that subsets might be very rich and mutable, and to > Jeff's proposal on hints I've already cited there with his permission. > > Best regards. > > Alexander > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Richard Treumann > Sent: Thursday, April 24, 2008 6:16 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > Dick Treumann - MPI Team/TCEM > IBM Systems & Technology Group > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > > mpi-22-bounces_at_[hidden] wrote on 04/24/2008 11:33:42 AM: > > > Hi, > > > > Note that this is an argument for making the assertions optional: those > > who don't care don't have to use them. Those who care should use them > > correctly or else. As usual. > > > > Best regards. > > > > Alexander > > > > Hi Alexander > > The assertions are optional in this proposal. If this is added to > the MPI standard the minimal impacts (day one impacts) are: > > == > To application writers (none) - MPI_INIT and MPI_INIT_THREAD still > work. MPI_INIT_THREAD_xxx can be > passed 0 (zero) as the assertions bit vector. > > To MPI Implementors (small) - subroutine MPI_INIT_THREAD_xxx can be > a clone of MPI_INIT_THREAD under the covers. If the Forum decides > the query function is for asking what assertions are being honored, > the implementation can just return "none" to every query. If there > is also a query for what assertions have been made then there are a > few more lines of code the implementor must write to preserve the > value so it can be returned(maybe 10 lines) > > Writers of opaque libraries (small) - call the query function at > library init time and if any assertions are found, issue an error > message and kill the job. This is awkward for a library that wants > to support every MPI whether it has implemented the new query function or not. > == > > As MPI implementations begin to take advantage of assertions there > is more work for the MPI implementor and the library author must > begin to think about whether his customer will be upset if the > library simply outlaws all assertions. > > The library author will never be wrong if he simply forbids > assertions forever. If they become valuable he will feel the > pressure to work it out. > > The MPI implementor will never be wrong if he adds the API but > simply ignores assertions forever. If they become valuable he will > feel the pressure to honor some at least. > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Tue May 6 06:30:03 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Tue, 6 May 2008 12:30:03 +0100 Subject: [Mpi-22] Call for MPI 2.2 agenda items In-Reply-To: <005701c8aeef$a20d3350$860add8c@mcs.anl.gov> Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2014F6796@swsmsx413.ger.corp.intel.com> Hi, May be we add a couple of calls to pass data back and forth between the client and MPI? Either assertions proposed by Dick Treumann, or more extensible (MPI_Info based?) approach proposed by Jeff Squyres? That's admittedly connected to subsets, but this is only an addition, no backward incompatible change. MPI 2.2? Best regards. Alexander -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Rajeev Thakur Sent: Monday, May 05, 2008 10:36 PM To: mpi-forum_at_[hidden]; mpi-22_at_[hidden] Subject: [Mpi-22] Call for MPI 2.2 agenda items Now that MPI 2.1 is close to completion, it is time to look at issues for MPI 2.2. Please send any items you may have for 2.2 to mpi-22_at_[hidden] and/or upload them to the wiki (and let us know). I had made a note of the following items from the last meeting that were deferred to 2.2: * support for long long in maxloc, minloc * fix meaning of first two sentences in intercommunicator barrier * Ex 5.8 uses old style MPI_UB * Many functions say "integer" instead of "nonnegative integer" * const in C++ bindings * IN, OUT, and INOUT handle types broken * MPI_CHAR, MPI::CHAR should be signed char or char? * Need 64-bit sizes for count in messages * Sizes of MPI_Aint and MPI_Offset should not be different * New predefined types MPI_AINT and MPI_OFFSET * New predefined types corresponding to new types in C99, Fortran 2003, and C++ some year _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. From treumann at [hidden] Tue May 6 10:20:31 2008 From: treumann at [hidden] (Richard Treumann) Date: Tue, 6 May 2008 11:20:31 -0400 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2014F678D@swsmsx413.ger.corp.intel.com> Message-ID: In my view, each assertion we define in the standard should be justified by a good explanation of how it allows an MPI implementation to be more efficient and what cost/risk it adds In effect, a cost/benefit analysis. No plausible rationale - no assertion. An assertion is a statement about the application. It is not a request for libmpi to "provide something". In my proposal, an application that correctly uses some assertion must still be written to work on an MPI implementation that ignores that assertion. An application that incorrectly uses some assertion can fail or give wrong answers. You can picture an assertion as the application author granting libmpi permission to "NOT provide something" at the MPI implementors discretion. In MPI 2 we maintained a careful distinction between hints and assertions. The MPI implementation is not allowed to give wrong results or error messages when an application provides an incorrect hint. If an application could provide a hint that it would not use MPI_CANCEL on a particular communicator and then called MPI_CANCEL, it would be OK for libmpi to use a really slow CANCEL protocol but not OK for it to issue a fatal error or ignore the CANCEL. That means that the freedom to "exploit" a hint in libmpi is very limited and less care is required of the user in providing hints. Much of what is being proposed should be treated as hints and could use a different and extensible mechanism. We can consider extending the MPI_Info usage in MPI-3 to address information passing that is hint-like. The MPI standard intends applications to be platform agnostic and source level portable and we should not break that. An assertions interface for making statements about the application will not limit this kind of portability. If there are no MPI_CANCEL calls today, there are no MPI_CANCEL calls after recompilation for a new platform. If we add features for platform tuning (like how many tasks per node or eager threshhold) we could damage portability. In my first outline, I threw in MPI-IO and MPI-1SC without any rationale for how they are useful. You added: "No communicator & group management", "No non-blocking communication", "No message tagging" and "No derived datatypes". It is not clear to me they are all useful so a decent rationale would be needed for each. That makes 6 bits that may be free. The thread support level parameter in MPI_INIT_THREAD is a request to the MPI library to provide some level of thread support and I question how cleanly that maps into the "assertions" model. The assertions model may only require one bit for "MPI_NO_THREAD_CONTENTION". That frees 3 more of the bits you suggest are already taken. If we conclude that all 4 thread support options are really useful it may be best to add the assertions flag to MPI_INIT_THREAD and keep the thread support controls as they are. Even the MPI_NO_THREAD_CONTENTION bit frees up then. My original thought was that we would add a new function that adds a parameter to MPI_INIT_THREAD. I am convinced that the number of assertions defined by the MPI standard should be very limited because each brings some risk and complicates life for library writers. If we end up with as many as 32 assertions I will apologize for having offered the idea in the first place and probably advocate a no vote. If MPI 2.2 provided only MPI_NO_EAGER_THROTTLE, MPI_NO_SEND_CANCEL, MPI_NO_REQUEST_MIX, MPI_NO_ANY_SOURCE and MPI_NO_REDUCTION_ORDER I would be content and debate about adding more could wait for MPI 3.0. The assertion that the application does not try to match persistent send/recv with nonpersistent recv/send would be a nice 2.2 bonus. That accounts for 6 bits. The query function could be something as simple as: MPI_Query_assertions(int *stated, int *exploited) The author of libfred could code in his Init_fred routine: int stated_by_app, exploited_by_lib; MPI_Query_assertions( &stated_by_app, &exploited_by_lib) if (exploited_by_lib & MPI_NO_SEND_CANCEL) fatal_error("Assertion MPI_NO_SEND_CANCEL is not compatable with Fred Lib") else if (stated_by_app & MPI_NO_SEND_CANCEL) warn("Assertion MPI_NO_SEND_CANCEL is not compatable with Fred Lib. It is not exploited by your MPI but your application will not port to an MPI that does exploit MPI_NO_SEND_CANCEL"); If Fred is really cautious he can code If (stated_by_app != 0) fatal_error("Fred does not like assertions") and see if anybody asks for some assertion to be tolerated in the next release of FredLib. You mention "No eager buffering" which is different than what I proposed and is not logically an "assertion". MPI applications should not depend semantically on eager buffering today and the standard is clear that a MPI implementation which has no eager buffering can still be 100% compliant. The standard requires that IF an MPI implementation provides an eager protocol as a performance enhancement, it must also provide a safety feature which can be expensive. Many (perhaps most) applications can benefit from an eager protocol but do not need the safety feature because the application is self regulating. These applications could assert: "I do not require the safety feature." which would allow them the benefit of eager protocol without the costs of the safety feature. I can see a rationale for some kind of hints related to eager protocol buffer space and threshhold. There are many other kinds of granular "hint" support that may be useful but all that should be kept distinct from this proposal (In my opinion). Regards - Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/06/2008 07:26:03 AM: > Dear Dick, > > Thanks. What about passing more verbose information to and from an > MPI implementation? Say, eager threshold value, if it's meaningful? > There are many less controversial examples, for example, to let the > caller know how many processes are placed onto the node, etc. Simple > assertions won't work in this case. > > Back to assertions. I thought we might consider > MPI_Initialized_subset(*flag,required,*provided) or something. > Setting "required" to 0 would return the set of provided assertions > in "provided". Setting "required" to something else will also set > the "flag" to true or false, depending on whether there's a match. > > The special MPI_Init_subset(*argc,***argv,required,*provided) would > complement this. > > Regarding 32 bits, let's count (I'm taking the joint list from the > subset proposal that includes your earlier points): > > 0,1,2,3 - MPI_THREAD_SINGLE, FUNNELED, SERIALIZED, MULTIPLE > 0x00000004 - No dynamic process support > 0x00000008 - No file I/O > 0x00000010 - No one-sided ops > 0x00000020 - No communicator & group management > 0x00000040 - No non-blocking communication > 0x00000080 - No message cancellation > 0x00000100 - Persistent ops on both sides > 0x00000200 - No heterogeneity > 0x00000400 - No derived datatypes (especially those with holes) > 0x00000800 - No MPI_ANY_SOURCE > 0x00001000 - No message tagging > 0x00002000 - No reduction order > 0x00004000 - No eager buffering > 0x00008000 - No mixed request types in wait/test > > We have 16 bits left. Note that there may be some predefined masks > for well defined combinations of the above (e.g., MPI-1 = no dynamic > processes, no file I/O, no threads, etc.). > > Best regards. > > Alexander > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Richard Treumann > Sent: Monday, May 05, 2008 11:53 PM > To: MPI 2.2 > Subject: [Mpi-22] MPI_INIT assertions > I changed the subject to be meaningful - it had been "Re: [Mpi-22] > mpi-22 Digest, Vol 2, Issue 7" > > I do think this proposal is within the scope of the MPI 2.2 rules so > I am pleased to hear this is being considered. > > I am 100% convinced that a query function should be part of this > proposal, probably with 2 query flavors. One flavor of the query > would respond with the set of assertions the application > MPI_INIT_xxx call had provided and another would respond with the > set of assertions the MPI implementation is actually exploiting. > > One library author may decide he will use if/else logic based on how > the MPI implementation will behave. The application may assert > MPI_NO_yyy but if the MPI implementation does not exploit MPI_NO_yyy > then the library can still use the code that depends on yyy. Another > library author may depend unconditionally on yyy. On an MPI > implementation that does not exploit MPI_NO_yyy he may want to issue > a warning that assertion MPI_NO_yyy is non-portable but let the job > run. On one that does exploit MPI_NO_yyy he would abort the job. Or, > he may decide to simply abort any job that asserts MPI_NO_yyy to > avoid having the library suddenly quit working when the customer > upgrades to an MPI implementation that does exploit MPI_NO_yyy. > > I think 32 assertions is probably more than enough > > Even a handful of defined assertions can raise testing costs. For > simple cases like MPI_NO_ANY_SOURCE it is easy for a user to judge > whether a piece of code is OK but if we begin to add subtle or > narrow semantic assertions it gets harder. Some library providers > will be tempted to say "I cannot proof read and test my code to a > degree that will allow me to accept 28 specific assertions and > forbid 4. I will simply forbid every subtle assertion I cannot > afford to test." > > I predict that for large applications developed by teams and for > community open source efforts the design leads will consider > requiring that all parts be written to live within a few carefully > chosen assertions. The design leads will not want to provide a list > of 26 subtle or narrow assertions and require that everyone respect > all 26 in the code they contribute. > > Many distinct assertions also could become a big test cost for MPI > implementors and customers who must qualify an MPI implementation > before trusting their business to it. If there were 64 or more > assertions, how would a tester decide what combinations of > assertions must be tested and then create suitable test cases? > > Dick > > Dick Treumann - MPI Team/TCEM > IBM Systems & Technology Group > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > > mpi-22-bounces_at_[hidden] wrote on 05/05/2008 01:28:06 PM: > > > Dear Dick, > > > > Thank you. We can actually introduce what you propose, possibly with > > a query function to make it still easier to live with, as early as > > in MPI 2.2, as a subset precursor. Judging by the discussion in > > Chicago, subsets may not need much more than that in the end, > > possibly with a little more flags and semantics added in MPI-3. > > > > The reservation against 32- (or for that matter, 64-) bit limitation > > is the only one I have at the moment. Not being able to attach > > assertions to communicators, etc. may be missed by some advanced > > programmers, but here we need to be pragmatic: who will ever want to > > go that deep? > > > > Best regards. > > > > Alexander > > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > > bounces_at_[hidden]] On Behalf Of Richard Treumann > > Sent: Monday, May 05, 2008 7:18 PM > > To: MPI 2.2 > > Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > > > Hi Alexander > > > > I have no objection to citing my "assertions" proposal in the > > subsetting discussions. I do want to keep it clear that this > > proposal is intended to be as simple as practical to implement, exploit and > > live with. > > > > "Live with" applies to 3rd party library authors or anyone else who > > must write MPI code but does not know and control the structure of > > the entire application. That guy must "live with" the decisions made > > by whoever coded the MPI_INIT piece. "Live with" also applies to > > whoever must test or certify a specific MPI implementation. > > > > Thanks - Dick > > > > Dick Treumann - MPI Team/TCEM > > IBM Systems & Technology Group > > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > > Tele (845) 433-7846 Fax (845) 433-8363 > > > > [image removed] "Supalov, Alexander" > > > > > > > "Supalov, Alexander" > > Sent by: mpi-22-bounces_at_[hidden] > > 04/26/2008 04:03 AM > > > > Please respond to > > "MPI 2.2" > > > > [image removed] > > To > > > > [image removed] > > "MPI 2.2" > > > > [image removed] > > cc > > > > [image removed] > > > > [image removed] > > Subject > > > > [image removed] > > Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > > > > [image removed] > > > > [image removed] > > > > > > Dear Dick, > > > > Thank you. Would you mind if I cite your proposal in the subsets > > discussion? Yours looks like a good alternative to the thinking of > > some of us that subsets might be very rich and mutable, and to > > Jeff's proposal on hints I've already cited there with his permission. > > > > Best regards. > > > > Alexander > > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > > bounces_at_[hidden]] On Behalf Of Richard Treumann > > Sent: Thursday, April 24, 2008 6:16 PM > > To: MPI 2.2 > > Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > > Dick Treumann - MPI Team/TCEM > > IBM Systems & Technology Group > > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > > Tele (845) 433-7846 Fax (845) 433-8363 > > > > > > mpi-22-bounces_at_[hidden] wrote on 04/24/2008 11:33:42 AM: > > > > > Hi, > > > > > > Note that this is an argument for making the assertions optional: those > > > who don't care don't have to use them. Those who care should use them > > > correctly or else. As usual. > > > > > > Best regards. > > > > > > Alexander > > > > > > > Hi Alexander > > > > The assertions are optional in this proposal. If this is added to > > the MPI standard the minimal impacts (day one impacts) are: > > > > == > > To application writers (none) - MPI_INIT and MPI_INIT_THREAD still > > work. MPI_INIT_THREAD_xxx can be > > passed 0 (zero) as the assertions bit vector. > > > > To MPI Implementors (small) - subroutine MPI_INIT_THREAD_xxx can be > > a clone of MPI_INIT_THREAD under the covers. If the Forum decides > > the query function is for asking what assertions are being honored, > > the implementation can just return "none" to every query. If there > > is also a query for what assertions have been made then there are a > > few more lines of code the implementor must write to preserve the > > value so it can be returned(maybe 10 lines) > > > > Writers of opaque libraries (small) - call the query function at > > library init time and if any assertions are found, issue an error > > message and kill the job. This is awkward for a library that wants > > to support every MPI whether it has implemented the new query > function or not. > > == > > > > As MPI implementations begin to take advantage of assertions there > > is more work for the MPI implementor and the library author must > > begin to think about whether his customer will be upset if the > > library simply outlaws all assertions. > > > > The library author will never be wrong if he simply forbids > > assertions forever. If they become valuable he will feel the > > pressure to work it out. > > > > The MPI implementor will never be wrong if he adds the API but > > simply ignores assertions forever. If they become valuable he will > > feel the pressure to honor some at least. > > --------------------------------------------------------------------- > > Intel GmbH > > Dornacher Strasse 1 > > 85622 Feldkirchen/Muenchen Germany > > Sitz der Gesellschaft: Feldkirchen bei Muenchen > > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > > VAT Registration No.: DE129385895 > > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > > > This e-mail and any attachments may contain confidential material for > > the sole use of the intended recipient(s). Any review or distribution > > by others is strictly prohibited. If you are not the intended > > recipient, please contact the sender and delete all copies. > > _______________________________________________ > > mpi-22 mailing list > > mpi-22_at_[hidden] > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > --------------------------------------------------------------------- > > Intel GmbH > > Dornacher Strasse 1 > > 85622 Feldkirchen/Muenchen Germany > > Sitz der Gesellschaft: Feldkirchen bei Muenchen > > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > > VAT Registration No.: DE129385895 > > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > > > This e-mail and any attachments may contain confidential material for > > the sole use of the intended recipient(s). Any review or distribution > > by others is strictly prohibited. If you are not the intended > > recipient, please contact the sender and delete all copies. > > _______________________________________________ > > mpi-22 mailing list > > mpi-22_at_[hidden] > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From erezh at [hidden] Tue May 6 11:50:28 2008 From: erezh at [hidden] (Erez Haba) Date: Tue, 6 May 2008 09:50:28 -0700 Subject: [Mpi-22] [Mpi-forum] Call for MPI 2.2 agenda items In-Reply-To: <005701c8aeef$a20d3350$860add8c@mcs.anl.gov> Message-ID: <6B68D01C00C9994A8E150183E62A119E72BF448BA4@NA-EXMSG-C105.redmond.corp.microsoft.com> I'd like to add a proposal for globalization (not localization). That is, adding to the standard text that defines all strings are UTF-8. (I'll put a proposal together for next meeting) A second proposal is to expose a Unicode (wchar_t) variation of all API's taking strings. However, that might be more suitable for 3.0. Thanks, .Erez -----Original Message----- From: mpi-forum-bounces_at_[hidden] [mailto:mpi-forum-bounces_at_[hidden]] On Behalf Of Rajeev Thakur Sent: Monday, May 05, 2008 1:36 PM To: mpi-forum_at_[hidden]; mpi-22_at_[hidden] Subject: [Mpi-forum] Call for MPI 2.2 agenda items Now that MPI 2.1 is close to completion, it is time to look at issues for MPI 2.2. Please send any items you may have for 2.2 to mpi-22_at_[hidden] and/or upload them to the wiki (and let us know). I had made a note of the following items from the last meeting that were deferred to 2.2: * support for long long in maxloc, minloc * fix meaning of first two sentences in intercommunicator barrier * Ex 5.8 uses old style MPI_UB * Many functions say "integer" instead of "nonnegative integer" * const in C++ bindings * IN, OUT, and INOUT handle types broken * MPI_CHAR, MPI::CHAR should be signed char or char? * Need 64-bit sizes for count in messages * Sizes of MPI_Aint and MPI_Offset should not be different * New predefined types MPI_AINT and MPI_OFFSET * New predefined types corresponding to new types in C99, Fortran 2003, and C++ some year _______________________________________________ mpi-forum mailing list mpi-forum_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum From alexander.supalov at [hidden] Thu May 8 09:52:45 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Thu, 8 May 2008 15:52:45 +0100 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2015215B5@swsmsx413.ger.corp.intel.com> Dear Dick, Thank you, I think I start feeling the difference between the assertions and the hints. To be completely sure, where would you put the soft spawning info key: is this an assertion or a hint? What about other standardized info elements that materially influence the program execution or may even predetermine its success or failure (like wdir)? More generally, will the difference between assertions and hints justify the use of two different mechanisms to pass the data across the MPI boundary? Here's I think what we can save by dropping certain MPI features: * Dynamic process support: less overhead in the progress engine, easier global rank handling. * File I/O: smaller requests, easier wait/test functions. * One-sided ops: no passive target w/o MPI calls - no extra progress thread. * Communicator & group management: better memory footprint. * Non-blocking communication: easier ordering, simplified request handling. * Heterogeneity: better memory footprint, easier data handling. * Derived datatypes (especially those with holes): better memory footprint. * Message tagging: better support for stable dataflow exchanges, smaller packets. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Tuesday, May 06, 2008 5:21 PM To: MPI 2.2 Subject: Re: [Mpi-22] MPI_INIT assertions In my view, each assertion we define in the standard should be justified by a good explanation of how it allows an MPI implementation to be more efficient and what cost/risk it adds In effect, a cost/benefit analysis. No plausible rationale - no assertion. An assertion is a statement about the application. It is not a request for libmpi to "provide something". In my proposal, an application that correctly uses some assertion must still be written to work on an MPI implementation that ignores that assertion. An application that incorrectly uses some assertion can fail or give wrong answers. You can picture an assertion as the application author granting libmpi permission to "NOT provide something" at the MPI implementors discretion. In MPI 2 we maintained a careful distinction between hints and assertions. The MPI implementation is not allowed to give wrong results or error messages when an application provides an incorrect hint. If an application could provide a hint that it would not use MPI_CANCEL on a particular communicator and then called MPI_CANCEL, it would be OK for libmpi to use a really slow CANCEL protocol but not OK for it to issue a fatal error or ignore the CANCEL. That means that the freedom to "exploit" a hint in libmpi is very limited and less care is required of the user in providing hints. Much of what is being proposed should be treated as hints and could use a different and extensible mechanism. We can consider extending the MPI_Info usage in MPI-3 to address information passing that is hint-like. The MPI standard intends applications to be platform agnostic and source level portable and we should not break that. An assertions interface for making statements about the application will not limit this kind of portability. If there are no MPI_CANCEL calls today, there are no MPI_CANCEL calls after recompilation for a new platform. If we add features for platform tuning (like how many tasks per node or eager threshhold) we could damage portability. In my first outline, I threw in MPI-IO and MPI-1SC without any rationale for how they are useful. You added: "No communicator & group management", "No non-blocking communication", "No message tagging" and "No derived datatypes". It is not clear to me they are all useful so a decent rationale would be needed for each. That makes 6 bits that may be free. The thread support level parameter in MPI_INIT_THREAD is a request to the MPI library to provide some level of thread support and I question how cleanly that maps into the "assertions" model. The assertions model may only require one bit for "MPI_NO_THREAD_CONTENTION". That frees 3 more of the bits you suggest are already taken. If we conclude that all 4 thread support options are really useful it may be best to add the assertions flag to MPI_INIT_THREAD and keep the thread support controls as they are. Even the MPI_NO_THREAD_CONTENTION bit frees up then. My original thought was that we would add a new function that adds a parameter to MPI_INIT_THREAD. I am convinced that the number of assertions defined by the MPI standard should be very limited because each brings some risk and complicates life for library writers. If we end up with as many as 32 assertions I will apologize for having offered the idea in the first place and probably advocate a no vote. If MPI 2.2 provided only MPI_NO_EAGER_THROTTLE, MPI_NO_SEND_CANCEL, MPI_NO_REQUEST_MIX, MPI_NO_ANY_SOURCE and MPI_NO_REDUCTION_ORDER I would be content and debate about adding more could wait for MPI 3.0. The assertion that the application does not try to match persistent send/recv with nonpersistent recv/send would be a nice 2.2 bonus. That accounts for 6 bits. The query function could be something as simple as: MPI_Query_assertions(int *stated, int *exploited) The author of libfred could code in his Init_fred routine: int stated_by_app, exploited_by_lib; MPI_Query_assertions( &stated_by_app, &exploited_by_lib) if (exploited_by_lib & MPI_NO_SEND_CANCEL) fatal_error("Assertion MPI_NO_SEND_CANCEL is not compatable with Fred Lib") else if (stated_by_app & MPI_NO_SEND_CANCEL) warn("Assertion MPI_NO_SEND_CANCEL is not compatable with Fred Lib. It is not exploited by your MPI but your application will not port to an MPI that does exploit MPI_NO_SEND_CANCEL"); If Fred is really cautious he can code If (stated_by_app != 0) fatal_error("Fred does not like assertions") and see if anybody asks for some assertion to be tolerated in the next release of FredLib. You mention "No eager buffering" which is different than what I proposed and is not logically an "assertion". MPI applications should not depend semantically on eager buffering today and the standard is clear that a MPI implementation which has no eager buffering can still be 100% compliant. The standard requires that IF an MPI implementation provides an eager protocol as a performance enhancement, it must also provide a safety feature which can be expensive. Many (perhaps most) applications can benefit from an eager protocol but do not need the safety feature because the application is self regulating. These applications could assert: "I do not require the safety feature." which would allow them the benefit of eager protocol without the costs of the safety feature. I can see a rationale for some kind of hints related to eager protocol buffer space and threshhold. There are many other kinds of granular "hint" support that may be useful but all that should be kept distinct from this proposal (In my opinion). Regards - Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/06/2008 07:26:03 AM: > Dear Dick, > > Thanks. What about passing more verbose information to and from an > MPI implementation? Say, eager threshold value, if it's meaningful? > There are many less controversial examples, for example, to let the > caller know how many processes are placed onto the node, etc. Simple > assertions won't work in this case. > > Back to assertions. I thought we might consider > MPI_Initialized_subset(*flag,required,*provided) or something. > Setting "required" to 0 would return the set of provided assertions > in "provided". Setting "required" to something else will also set > the "flag" to true or false, depending on whether there's a match. > > The special MPI_Init_subset(*argc,***argv,required,*provided) would > complement this. > > Regarding 32 bits, let's count (I'm taking the joint list from the > subset proposal that includes your earlier points): > > 0,1,2,3 - MPI_THREAD_SINGLE, FUNNELED, SERIALIZED, MULTIPLE > 0x00000004 - No dynamic process support > 0x00000008 - No file I/O > 0x00000010 - No one-sided ops > 0x00000020 - No communicator & group management > 0x00000040 - No non-blocking communication > 0x00000080 - No message cancellation > 0x00000100 - Persistent ops on both sides > 0x00000200 - No heterogeneity > 0x00000400 - No derived datatypes (especially those with holes) > 0x00000800 - No MPI_ANY_SOURCE > 0x00001000 - No message tagging > 0x00002000 - No reduction order > 0x00004000 - No eager buffering > 0x00008000 - No mixed request types in wait/test > > We have 16 bits left. Note that there may be some predefined masks > for well defined combinations of the above (e.g., MPI-1 = no dynamic > processes, no file I/O, no threads, etc.). > > Best regards. > > Alexander > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Richard Treumann > Sent: Monday, May 05, 2008 11:53 PM > To: MPI 2.2 > Subject: [Mpi-22] MPI_INIT assertions > I changed the subject to be meaningful - it had been "Re: [Mpi-22] > mpi-22 Digest, Vol 2, Issue 7" > > I do think this proposal is within the scope of the MPI 2.2 rules so > I am pleased to hear this is being considered. > > I am 100% convinced that a query function should be part of this > proposal, probably with 2 query flavors. One flavor of the query > would respond with the set of assertions the application > MPI_INIT_xxx call had provided and another would respond with the > set of assertions the MPI implementation is actually exploiting. > > One library author may decide he will use if/else logic based on how > the MPI implementation will behave. The application may assert > MPI_NO_yyy but if the MPI implementation does not exploit MPI_NO_yyy > then the library can still use the code that depends on yyy. Another > library author may depend unconditionally on yyy. On an MPI > implementation that does not exploit MPI_NO_yyy he may want to issue > a warning that assertion MPI_NO_yyy is non-portable but let the job > run. On one that does exploit MPI_NO_yyy he would abort the job. Or, > he may decide to simply abort any job that asserts MPI_NO_yyy to > avoid having the library suddenly quit working when the customer > upgrades to an MPI implementation that does exploit MPI_NO_yyy. > > I think 32 assertions is probably more than enough > > Even a handful of defined assertions can raise testing costs. For > simple cases like MPI_NO_ANY_SOURCE it is easy for a user to judge > whether a piece of code is OK but if we begin to add subtle or > narrow semantic assertions it gets harder. Some library providers > will be tempted to say "I cannot proof read and test my code to a > degree that will allow me to accept 28 specific assertions and > forbid 4. I will simply forbid every subtle assertion I cannot > afford to test." > > I predict that for large applications developed by teams and for > community open source efforts the design leads will consider > requiring that all parts be written to live within a few carefully > chosen assertions. The design leads will not want to provide a list > of 26 subtle or narrow assertions and require that everyone respect > all 26 in the code they contribute. > > Many distinct assertions also could become a big test cost for MPI > implementors and customers who must qualify an MPI implementation > before trusting their business to it. If there were 64 or more > assertions, how would a tester decide what combinations of > assertions must be tested and then create suitable test cases? > > Dick > > Dick Treumann - MPI Team/TCEM > IBM Systems & Technology Group > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > > mpi-22-bounces_at_[hidden] wrote on 05/05/2008 01:28:06 PM: > > > Dear Dick, > > > > Thank you. We can actually introduce what you propose, possibly with > > a query function to make it still easier to live with, as early as > > in MPI 2.2, as a subset precursor. Judging by the discussion in > > Chicago, subsets may not need much more than that in the end, > > possibly with a little more flags and semantics added in MPI-3. > > > > The reservation against 32- (or for that matter, 64-) bit limitation > > is the only one I have at the moment. Not being able to attach > > assertions to communicators, etc. may be missed by some advanced > > programmers, but here we need to be pragmatic: who will ever want to > > go that deep? > > > > Best regards. > > > > Alexander > > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > > bounces_at_[hidden]] On Behalf Of Richard Treumann > > Sent: Monday, May 05, 2008 7:18 PM > > To: MPI 2.2 > > Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > > > Hi Alexander > > > > I have no objection to citing my "assertions" proposal in the > > subsetting discussions. I do want to keep it clear that this > > proposal is intended to be as simple as practical to implement, exploit and > > live with. > > > > "Live with" applies to 3rd party library authors or anyone else who > > must write MPI code but does not know and control the structure of > > the entire application. That guy must "live with" the decisions made > > by whoever coded the MPI_INIT piece. "Live with" also applies to > > whoever must test or certify a specific MPI implementation. > > > > Thanks - Dick > > > > Dick Treumann - MPI Team/TCEM > > IBM Systems & Technology Group > > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > > Tele (845) 433-7846 Fax (845) 433-8363 > > > > [image removed] "Supalov, Alexander" > > > > > > > "Supalov, Alexander" > > Sent by: mpi-22-bounces_at_[hidden] > > 04/26/2008 04:03 AM > > > > Please respond to > > "MPI 2.2" > > > > [image removed] > > To > > > > [image removed] > > "MPI 2.2" > > > > [image removed] > > cc > > > > [image removed] > > > > [image removed] > > Subject > > > > [image removed] > > Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > > > > [image removed] > > > > [image removed] > > > > > > Dear Dick, > > > > Thank you. Would you mind if I cite your proposal in the subsets > > discussion? Yours looks like a good alternative to the thinking of > > some of us that subsets might be very rich and mutable, and to > > Jeff's proposal on hints I've already cited there with his permission. > > > > Best regards. > > > > Alexander > > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > > bounces_at_[hidden]] On Behalf Of Richard Treumann > > Sent: Thursday, April 24, 2008 6:16 PM > > To: MPI 2.2 > > Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > > Dick Treumann - MPI Team/TCEM > > IBM Systems & Technology Group > > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > > Tele (845) 433-7846 Fax (845) 433-8363 > > > > > > mpi-22-bounces_at_[hidden] wrote on 04/24/2008 11:33:42 AM: > > > > > Hi, > > > > > > Note that this is an argument for making the assertions optional: those > > > who don't care don't have to use them. Those who care should use them > > > correctly or else. As usual. > > > > > > Best regards. > > > > > > Alexander > > > > > > > Hi Alexander > > > > The assertions are optional in this proposal. If this is added to > > the MPI standard the minimal impacts (day one impacts) are: > > > > == > > To application writers (none) - MPI_INIT and MPI_INIT_THREAD still > > work. MPI_INIT_THREAD_xxx can be > > passed 0 (zero) as the assertions bit vector. > > > > To MPI Implementors (small) - subroutine MPI_INIT_THREAD_xxx can be > > a clone of MPI_INIT_THREAD under the covers. If the Forum decides > > the query function is for asking what assertions are being honored, > > the implementation can just return "none" to every query. If there > > is also a query for what assertions have been made then there are a > > few more lines of code the implementor must write to preserve the > > value so it can be returned(maybe 10 lines) > > > > Writers of opaque libraries (small) - call the query function at > > library init time and if any assertions are found, issue an error > > message and kill the job. This is awkward for a library that wants > > to support every MPI whether it has implemented the new query > function or not. > > == > > > > As MPI implementations begin to take advantage of assertions there > > is more work for the MPI implementor and the library author must > > begin to think about whether his customer will be upset if the > > library simply outlaws all assertions. > > > > The library author will never be wrong if he simply forbids > > assertions forever. If they become valuable he will feel the > > pressure to work it out. > > > > The MPI implementor will never be wrong if he adds the API but > > simply ignores assertions forever. If they become valuable he will > > feel the pressure to honor some at least. > > --------------------------------------------------------------------- > > Intel GmbH > > Dornacher Strasse 1 > > 85622 Feldkirchen/Muenchen Germany > > Sitz der Gesellschaft: Feldkirchen bei Muenchen > > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > > VAT Registration No.: DE129385895 > > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > > > This e-mail and any attachments may contain confidential material for > > the sole use of the intended recipient(s). Any review or distribution > > by others is strictly prohibited. If you are not the intended > > recipient, please contact the sender and delete all copies. > > _______________________________________________ > > mpi-22 mailing list > > mpi-22_at_[hidden] > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > --------------------------------------------------------------------- > > Intel GmbH > > Dornacher Strasse 1 > > 85622 Feldkirchen/Muenchen Germany > > Sitz der Gesellschaft: Feldkirchen bei Muenchen > > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > > VAT Registration No.: DE129385895 > > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > > > This e-mail and any attachments may contain confidential material for > > the sole use of the intended recipient(s). Any review or distribution > > by others is strictly prohibited. If you are not the intended > > recipient, please contact the sender and delete all copies. > > _______________________________________________ > > mpi-22 mailing list > > mpi-22_at_[hidden] > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From treumann at [hidden] Thu May 8 12:41:22 2008 From: treumann at [hidden] (Richard Treumann) Date: Thu, 8 May 2008 13:41:22 -0400 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2015215B5@swsmsx413.ger.corp.intel.com> Message-ID: Hi Alexander First - changing the rules to let MPI_Info objects be created and manipulated before MPI_INIT is much more intrusive on implementations. If the MPI implementation uses some set of data structures to manage MPI opaque objects, those data structures probably do not exist before MPI_INIT. Passing a bit vector into MPI_INIT does not require significant reorganization of the MPI implementation but allowing MPI_Info creation and manipulation does. The concept of an "assertions" bit vector is not new for MPI (see last paragreph) so using it does not require inventing two mechanisms. You have found a couple examples that flirt with violating the concept of a "hint". For soft spawn I would argue that any soft spawn call that would have succeeded as a hard spawn must also succeed as a soft spawn no matter what hint you provide. That is, if you try to spawn with maxprocs=16 and you get 16 task your hint that you would take 8 has not resulted in semantic thin ice. If your hint said you take 32 tasks, MPI can give you 16 and again no semantic thin ice. Even if you ask for 16 on the call but agree to 8 with a hint and a hard spawn for 8 tasks "would have worked", libmpi is free to ignore the hint and fail the call because it cannot provide the 16. On wdir (+ host, arch, path, file) it gets fuzzy. An MPI implementation is free to ignore any of these and the user is free to omit them and hope the MPI implementation has reasonable defaults. It could be argued that the MPI implementation has the option of ignoring a hint when it cannot honor it and falling back to the default. That would fit the "hint" concept but I do not know if that was envisioned when these were put in MPI-2. Actually we may want to look at doing a careful catagorization of the kinds of inputs that an MPI_Info can convey. I may be making assumptions that are not justified by a strict reading of the standard when I imply that MPI_Info (key, value) pairs == hints. As part of MPI 3 we may want to specify that the MPI_Info object can carry several kinds if "info" to an MPI implementation after MPI_INIT. hints - can never break semantic or trigger error messages directives - do not break semantic but can make the difference between a run succeeding and failing (e.g.the named wdir does not exist so libmpi takes an error trying to use it) assertions - can alter semantic. If we want a way to put assertions on communicators for example I see no reason we could not use an Info object to pass them in more? In MPI_FILE_OPEN we have MPI_MODE_UNIQUE_OPEN as a mode flag and say that if the user says MPI_MODE_UNIQUE_OPEN and is wrong, his results cannot be ensured. Silent corruption of the file is possible. We do not use the word "assertion" though maybe we should have. In MPI 1sided synch operations we have an integer IN argument we call an "assertion" and it can take flages like MPI_MODE_NOCHECK, MPI_MODE_NOSTORE or MPI_MODE_NOPUT. We tell the user that if he provides an assertion that is invalid he cannot assume the program will work even though it would have without the assertion. Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 "Supalov, Alexander" "MPI 2.2" Sent by: mpi-22-bounces_at_li cc sts.mpi-forum.org Subject Re: [Mpi-22] MPI_INIT assertions 05/08/2008 10:52 AM Please respond to "MPI 2.2" Dear Dick, Thank you, I think I start feeling the difference between the assertions and the hints. To be completely sure, where would you put the soft spawning info key: is this an assertion or a hint? What about other standardized info elements that materially influence the program execution or may even predetermine its success or failure (like wdir)? More generally, will the difference between assertions and hints justify the use of two different mechanisms to pass the data across the MPI boundary? Here's I think what we can save by dropping certain MPI features: Dynamic process support: less overhead in the progress engine, easier global rank handling. File I/O: smaller requests, easier wait/test functions. One-sided ops: no passive target w/o MPI calls - no extra progress thread. Communicator & group management: better memory footprint. Non-blocking communication: easier ordering, simplified request handling. Heterogeneity: better memory footprint, easier data handling. Derived datatypes (especially those with holes): better memory footprint. Message tagging: better support for stable dataflow exchanges, smaller packets. Best regards. Alexander From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Tuesday, May 06, 2008 5:21 PM To: MPI 2.2 Subject: Re: [Mpi-22] MPI_INIT assertions In my view, each assertion we define in the standard should be justified by a good explanation of how it allows an MPI implementation to be more efficient and what cost/risk it adds In effect, a cost/benefit analysis. No plausible rationale - no assertion. An assertion is a statement about the application. It is not a request for libmpi to "provide something". In my proposal, an application that correctly uses some assertion must still be written to work on an MPI implementation that ignores that assertion. An application that incorrectly uses some assertion can fail or give wrong answers. You can picture an assertion as the application author granting libmpi permission to "NOT provide something" at the MPI implementors discretion. In MPI 2 we maintained a careful distinction between hints and assertions. The MPI implementation is not allowed to give wrong results or error messages when an application provides an incorrect hint. If an application could provide a hint that it would not use MPI_CANCEL on a particular communicator and then called MPI_CANCEL, it would be OK for libmpi to use a really slow CANCEL protocol but not OK for it to issue a fatal error or ignore the CANCEL. That means that the freedom to "exploit" a hint in libmpi is very limited and less care is required of the user in providing hints. Much of what is being proposed should be treated as hints and could use a different and extensible mechanism. We can consider extending the MPI_Info usage in MPI-3 to address information passing that is hint-like. The MPI standard intends applications to be platform agnostic and source level portable and we should not break that. An assertions interface for making statements about the application will not limit this kind of portability. If there are no MPI_CANCEL calls today, there are no MPI_CANCEL calls after recompilation for a new platform. If we add features for platform tuning (like how many tasks per node or eager threshhold) we could damage portability. In my first outline, I threw in MPI-IO and MPI-1SC without any rationale for how they are useful. You added: "No communicator & group management", "No non-blocking communication", "No message tagging" and "No derived datatypes". It is not clear to me they are all useful so a decent rationale would be needed for each. That makes 6 bits that may be free. The thread support level parameter in MPI_INIT_THREAD is a request to the MPI library to provide some level of thread support and I question how cleanly that maps into the "assertions" model. The assertions model may only require one bit for "MPI_NO_THREAD_CONTENTION". That frees 3 more of the bits you suggest are already taken. If we conclude that all 4 thread support options are really useful it may be best to add the assertions flag to MPI_INIT_THREAD and keep the thread support controls as they are. Even the MPI_NO_THREAD_CONTENTION bit frees up then. My original thought was that we would add a new function that adds a parameter to MPI_INIT_THREAD. I am convinced that the number of assertions defined by the MPI standard should be very limited because each brings some risk and complicates life for library writers. If we end up with as many as 32 assertions I will apologize for having offered the idea in the first place and probably advocate a no vote. If MPI 2.2 provided only MPI_NO_EAGER_THROTTLE, MPI_NO_SEND_CANCEL, MPI_NO_REQUEST_MIX, MPI_NO_ANY_SOURCE and MPI_NO_REDUCTION_ORDER I would be content and debate about adding more could wait for MPI 3.0. The assertion that the application does not try to match persistent send/recv with nonpersistent recv/send would be a nice 2.2 bonus. That accounts for 6 bits. The query function could be something as simple as: MPI_Query_assertions(int *stated, int *exploited) The author of libfred could code in his Init_fred routine: int stated_by_app, exploited_by_lib; MPI_Query_assertions( &stated_by_app, &exploited_by_lib) if (exploited_by_lib & MPI_NO_SEND_CANCEL) fatal_error("Assertion MPI_NO_SEND_CANCEL is not compatable with Fred Lib") else if (stated_by_app & MPI_NO_SEND_CANCEL) warn("Assertion MPI_NO_SEND_CANCEL is not compatable with Fred Lib. It is not exploited by your MPI but your application will not port to an MPI that does exploit MPI_NO_SEND_CANCEL"); If Fred is really cautious he can code If (stated_by_app != 0) fatal_error("Fred does not like assertions") and see if anybody asks for some assertion to be tolerated in the next release of FredLib. You mention "No eager buffering" which is different than what I proposed and is not logically an "assertion". MPI applications should not depend semantically on eager buffering today and the standard is clear that a MPI implementation which has no eager buffering can still be 100% compliant. The standard requires that IF an MPI implementation provides an eager protocol as a performance enhancement, it must also provide a safety feature which can be expensive. Many (perhaps most) applications can benefit from an eager protocol but do not need the safety feature because the application is self regulating. These applications could assert: "I do not require the safety feature." which would allow them the benefit of eager protocol without the costs of the safety feature. I can see a rationale for some kind of hints related to eager protocol buffer space and threshhold. There are many other kinds of granular "hint" support that may be useful but all that should be kept distinct from this proposal (In my opinion). Regards - Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/06/2008 07:26:03 AM: > Dear Dick, > > Thanks. What about passing more verbose information to and from an > MPI implementation? Say, eager threshold value, if it's meaningful? > There are many less controversial examples, for example, to let the > caller know how many processes are placed onto the node, etc. Simple > assertions won't work in this case. > > Back to assertions. I thought we might consider > MPI_Initialized_subset(*flag,required,*provided) or something. > Setting "required" to 0 would return the set of provided assertions > in "provided". Setting "required" to something else will also set > the "flag" to true or false, depending on whether there's a match. > > The special MPI_Init_subset(*argc,***argv,required,*provided) would > complement this. > > Regarding 32 bits, let's count (I'm taking the joint list from the > subset proposal that includes your earlier points): > > 0,1,2,3 - MPI_THREAD_SINGLE, FUNNELED, SERIALIZED, MULTIPLE > 0x00000004 - No dynamic process support > 0x00000008 - No file I/O > 0x00000010 - No one-sided ops > 0x00000020 - No communicator & group management > 0x00000040 - No non-blocking communication > 0x00000080 - No message cancellation > 0x00000100 - Persistent ops on both sides > 0x00000200 - No heterogeneity > 0x00000400 - No derived datatypes (especially those with holes) > 0x00000800 - No MPI_ANY_SOURCE > 0x00001000 - No message tagging > 0x00002000 - No reduction order > 0x00004000 - No eager buffering > 0x00008000 - No mixed request types in wait/test > > We have 16 bits left. Note that there may be some predefined masks > for well defined combinations of the above (e.g., MPI-1 = no dynamic > processes, no file I/O, no threads, etc.). > > Best regards. > > Alexander > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Richard Treumann > Sent: Monday, May 05, 2008 11:53 PM > To: MPI 2.2 > Subject: [Mpi-22] MPI_INIT assertions > I changed the subject to be meaningful - it had been "Re: [Mpi-22] > mpi-22 Digest, Vol 2, Issue 7" > > I do think this proposal is within the scope of the MPI 2.2 rules so > I am pleased to hear this is being considered. > > I am 100% convinced that a query function should be part of this > proposal, probably with 2 query flavors. One flavor of the query > would respond with the set of assertions the application > MPI_INIT_xxx call had provided and another would respond with the > set of assertions the MPI implementation is actually exploiting. > > One library author may decide he will use if/else logic based on how > the MPI implementation will behave. The application may assert > MPI_NO_yyy but if the MPI implementation does not exploit MPI_NO_yyy > then the library can still use the code that depends on yyy. Another > library author may depend unconditionally on yyy. On an MPI > implementation that does not exploit MPI_NO_yyy he may want to issue > a warning that assertion MPI_NO_yyy is non-portable but let the job > run. On one that does exploit MPI_NO_yyy he would abort the job. Or, > he may decide to simply abort any job that asserts MPI_NO_yyy to > avoid having the library suddenly quit working when the customer > upgrades to an MPI implementation that does exploit MPI_NO_yyy. > > I think 32 assertions is probably more than enough > > Even a handful of defined assertions can raise testing costs. For > simple cases like MPI_NO_ANY_SOURCE it is easy for a user to judge > whether a piece of code is OK but if we begin to add subtle or > narrow semantic assertions it gets harder. Some library providers > will be tempted to say "I cannot proof read and test my code to a > degree that will allow me to accept 28 specific assertions and > forbid 4. I will simply forbid every subtle assertion I cannot > afford to test." > > I predict that for large applications developed by teams and for > community open source efforts the design leads will consider > requiring that all parts be written to live within a few carefully > chosen assertions. The design leads will not want to provide a list > of 26 subtle or narrow assertions and require that everyone respect > all 26 in the code they contribute. > > Many distinct assertions also could become a big test cost for MPI > implementors and customers who must qualify an MPI implementation > before trusting their business to it. If there were 64 or more > assertions, how would a tester decide what combinations of > assertions must be tested and then create suitable test cases? > > Dick > > Dick Treumann - MPI Team/TCEM > IBM Systems & Technology Group > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > > mpi-22-bounces_at_[hidden] wrote on 05/05/2008 01:28:06 PM: > > > Dear Dick, > > > > Thank you. We can actually introduce what you propose, possibly with > > a query function to make it still easier to live with, as early as > > in MPI 2.2, as a subset precursor. Judging by the discussion in > > Chicago, subsets may not need much more than that in the end, > > possibly with a little more flags and semantics added in MPI-3. > > > > The reservation against 32- (or for that matter, 64-) bit limitation > > is the only one I have at the moment. Not being able to attach > > assertions to communicators, etc. may be missed by some advanced > > programmers, but here we need to be pragmatic: who will ever want to > > go that deep? > > > > Best regards. > > > > Alexander > > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > > bounces_at_[hidden]] On Behalf Of Richard Treumann > > Sent: Monday, May 05, 2008 7:18 PM > > To: MPI 2.2 > > Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > > > Hi Alexander > > > > I have no objection to citing my "assertions" proposal in the > > subsetting discussions. I do want to keep it clear that this > > proposal is intended to be as simple as practical to implement, exploit and > > live with. > > > > "Live with" applies to 3rd party library authors or anyone else who > > must write MPI code but does not know and control the structure of > > the entire application. That guy must "live with" the decisions made > > by whoever coded the MPI_INIT piece. "Live with" also applies to > > whoever must test or certify a specific MPI implementation. > > > > Thanks - Dick > > > > Dick Treumann - MPI Team/TCEM > > IBM Systems & Technology Group > > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > > Tele (845) 433-7846 Fax (845) 433-8363 > > > > [image removed] "Supalov, Alexander" > > > > > > > "Supalov, Alexander" > > Sent by: mpi-22-bounces_at_[hidden] > > 04/26/2008 04:03 AM > > > > Please respond to > > "MPI 2.2" > > > > [image removed] > > To > > > > [image removed] > > "MPI 2.2" > > > > [image removed] > > cc > > > > [image removed] > > > > [image removed] > > Subject > > > > [image removed] > > Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > > > > [image removed] > > > > [image removed] > > > > > > Dear Dick, > > > > Thank you. Would you mind if I cite your proposal in the subsets > > discussion? Yours looks like a good alternative to the thinking of > > some of us that subsets might be very rich and mutable, and to > > Jeff's proposal on hints I've already cited there with his permission. > > > > Best regards. > > > > Alexander > > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > > bounces_at_[hidden]] On Behalf Of Richard Treumann > > Sent: Thursday, April 24, 2008 6:16 PM > > To: MPI 2.2 > > Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > > Dick Treumann - MPI Team/TCEM > > IBM Systems & Technology Group > > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > > Tele (845) 433-7846 Fax (845) 433-8363 > > > > > > mpi-22-bounces_at_[hidden] wrote on 04/24/2008 11:33:42 AM: > > > > > Hi, > > > > > > Note that this is an argument for making the assertions optional: those > > > who don't care don't have to use them. Those who care should use them > > > correctly or else. As usual. > > > > > > Best regards. > > > > > > Alexander > > > > > > > Hi Alexander > > > > The assertions are optional in this proposal. If this is added to > > the MPI standard the minimal impacts (day one impacts) are: > > > > == > > To application writers (none) - MPI_INIT and MPI_INIT_THREAD still > > work. MPI_INIT_THREAD_xxx can be > > passed 0 (zero) as the assertions bit vector. > > > > To MPI Implementors (small) - subroutine MPI_INIT_THREAD_xxx can be > > a clone of MPI_INIT_THREAD under the covers. If the Forum decides > > the query function is for asking what assertions are being honored, > > the implementation can just return "none" to every query. If there > > is also a query for what assertions have been made then there are a > > few more lines of code the implementor must write to preserve the > > value so it can be returned(maybe 10 lines) > > > > Writers of opaque libraries (small) - call the query function at > > library init time and if any assertions are found, issue an error > > message and kill the job. This is awkward for a library that wants > > to support every MPI whether it has implemented the new query > function or not. > > == > > > > As MPI implementations begin to take advantage of assertions there > > is more work for the MPI implementor and the library author must > > begin to think about whether his customer will be upset if the > > library simply outlaws all assertions. > > > > The library author will never be wrong if he simply forbids > > assertions forever. If they become valuable he will feel the > > pressure to work it out. > > > > The MPI implementor will never be wrong if he adds the API but > > simply ignores assertions forever. If they become valuable he will > > feel the pressure to honor some at least. > > --------------------------------------------------------------------- > > Intel GmbH > > Dornacher Strasse 1 > > 85622 Feldkirchen/Muenchen Germany > > Sitz der Gesellschaft: Feldkirchen bei Muenchen > > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > > VAT Registration No.: DE129385895 > > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > > > This e-mail and any attachments may contain confidential material for > > the sole use of the intended recipient(s). Any review or distribution > > by others is strictly prohibited. If you are not the intended > > recipient, please contact the sender and delete all copies. > > _______________________________________________ > > mpi-22 mailing list > > mpi-22_at_[hidden] > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > --------------------------------------------------------------------- > > Intel GmbH > > Dornacher Strasse 1 > > 85622 Feldkirchen/Muenchen Germany > > Sitz der Gesellschaft: Feldkirchen bei Muenchen > > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > > VAT Registration No.: DE129385895 > > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > > > This e-mail and any attachments may contain confidential material for > > the sole use of the intended recipient(s). Any review or distribution > > by others is strictly prohibited. If you are not the intended > > recipient, please contact the sender and delete all copies. > > _______________________________________________ > > mpi-22 mailing list > > mpi-22_at_[hidden] > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: application/octet-stream Size: 156 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic25566.gif Type: application/octet-stream Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: application/octet-stream Size: 45 bytes Desc: not available URL: From treumann at [hidden] Thu May 8 16:22:39 2008 From: treumann at [hidden] (Richard Treumann) Date: Thu, 8 May 2008 17:22:39 -0400 Subject: [Mpi-22] Memory footprint concern In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2015215B5@swsmsx413.ger.corp.intel.com> Message-ID: Can somebody help me understand this "smaller memory footprint" issue that is part f the subsetting goal better. What systems does it affect? What does "memory footprint" really mean? In the 64 bit address space, virtual address range is not a problem. On systems I am most familiar with (AIX and I have been told Linux too), if you have a library that contains 1000 subroutines and you run a program than only calls 6 then only the pages that are touched by code for those 6 functions must get placed in real memory. The rest of the object code stays on disk. Program and library text is demand paged. The loading is on page boundries, not subroutine boundries. With a shared library, if I run a program on a node and touch 6 subroutines and you run a different program that touches those 6 and 10 more then code for all 16 subroutines may be kept in memory but the rest of the library will stay on disk. You and I will share the object code for the 6 subroutines we are both calling. Someone who wanted to make a libmpi that has MPI-1sided or MPI-IO well isolated in the library structure so simple MPI jobs would not force this extra code into memory could do that today. The user does not need to promise not to call MPI-IO subroutines for them not to to take real memory. The "subsets" would need to be devised by the MPI implementor but would be transparent to the MPI user and not dictated by the standard. The "subsets" the user did not call would remain paged out. Perhaps all static data defined by the library will come into real memory for each process but is there much reduction from being able to somehow not bring in the static data MPI-IO would require because somebody had promised not to use it? Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Thu May 8 16:52:34 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Thu, 8 May 2008 22:52:34 +0100 Subject: [Mpi-22] Memory footprint concern In-Reply-To: Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A20152173B@swsmsx413.ger.corp.intel.com> Dear Dick, By the looks of it, MPI-3 is going to be big. Petascale machines may not have OS we're accustomed to, dynamic libraries, and some other things. Smaller system libraries - and smaller MPI - may be needed there. Some of the envisioned MPI-3 features will be needed for some applications, some won't. Same with MPI-2 and MPI-1. Defining subsets may help to open a way to custom cut MPI libraries suitable for particular application classes. How subsets will be implemented is a different matter. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Thursday, May 08, 2008 11:23 PM To: MPI 2.2 Subject: [Mpi-22] Memory footprint concern Can somebody help me understand this "smaller memory footprint" issue that is part f the subsetting goal better. What systems does it affect? What does "memory footprint" really mean? In the 64 bit address space, virtual address range is not a problem. On systems I am most familiar with (AIX and I have been told Linux too), if you have a library that contains 1000 subroutines and you run a program than only calls 6 then only the pages that are touched by code for those 6 functions must get placed in real memory. The rest of the object code stays on disk. Program and library text is demand paged. The loading is on page boundries, not subroutine boundries. With a shared library, if I run a program on a node and touch 6 subroutines and you run a different program that touches those 6 and 10 more then code for all 16 subroutines may be kept in memory but the rest of the library will stay on disk. You and I will share the object code for the 6 subroutines we are both calling. Someone who wanted to make a libmpi that has MPI-1sided or MPI-IO well isolated in the library structure so simple MPI jobs would not force this extra code into memory could do that today. The user does not need to promise not to call MPI-IO subroutines for them not to to take real memory. The "subsets" would need to be devised by the MPI implementor but would be transparent to the MPI user and not dictated by the standard. The "subsets" the user did not call would remain paged out. Perhaps all static data defined by the library will come into real memory for each process but is there much reduction from being able to somehow not bring in the static data MPI-IO would require because somebody had promised not to use it? Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From treumann at [hidden] Thu May 8 20:11:43 2008 From: treumann at [hidden] (Richard Treumann) Date: Thu, 8 May 2008 21:11:43 -0400 Subject: [Mpi-22] Memory footprint concern In-Reply-To: <5ECAB1304A8B5B4CB3F9D6C01E4E21A20152173B@swsmsx413.ger.corp.intel.com> Message-ID: Hi Alexander If we would envision defining subsets in such a way that a machine with limited memory per node and un-fancy OS can clearly document what its MPI offers, maybe it is not a huge addition to complexity. For example if an MPI implementation could say it offers subsets: MPI 1.2 MPI-IO MPI-Intercomm Collectives and nothing else and the meaning would be well defined that may not be too bad. If you are expecting such a machine to have all or most of MPI-3 available and each user would select only the parts he wants at job launch time it seems to me the complexity for implementation and testing will be prohibitive. In particular if you hope subsets will give better performance because of what they leave out and really be compact because they leave out most unneeded bytes of code the number of permutations seem to me to make that a vain hope. I have been ambivalent about whether my assertions proposal really has much connection to the subsets concept mostly because I think to be useful assertions must be fairly simple. The only assertion I am 100% sure will pay off and be used is MPI_NO_EAGER_THROTTLE. There are MPI implementations today that act as if the assertion MPI_NO_EAGER_THROTTLE were present on all applications but that is not an option for a vendor MPI because if somebody contacts service and says "Your MPI violates the standard" a vendor like IBM must "fix" it. IBM MPI supports MPI_CANCEL on MPI_ISEND but at least one MPI does not because it is "too expensive". In effect they act like every application has the assertion MPI_NO_SEND_CANCELS. IBM MPI could make use of MPI_NO_REQUEST_MIX if it was available. Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 "Supalov, Alexander" "MPI 2.2" Sent by: mpi-22-bounces_at_li cc sts.mpi-forum.org Subject Re: [Mpi-22] Memory footprint 05/08/2008 05:52 concern PM Please respond to "MPI 2.2" Dear Dick, By the looks of it, MPI-3 is going to be big. Petascale machines may not have OS we're accustomed to, dynamic libraries, and some other things. Smaller system libraries - and smaller MPI - may be needed there. Some of the envisioned MPI-3 features will be needed for some applications, some won't. Same with MPI-2 and MPI-1. Defining subsets may help to open a way to custom cut MPI libraries suitable for particular application classes. How subsets will be implemented is a different matter. Best regards. Alexander From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Thursday, May 08, 2008 11:23 PM To: MPI 2.2 Subject: [Mpi-22] Memory footprint concern Can somebody help me understand this "smaller memory footprint" issue that is part f the subsetting goal better. What systems does it affect? What does "memory footprint" really mean? In the 64 bit address space, virtual address range is not a problem. On systems I am most familiar with (AIX and I have been told Linux too), if you have a library that contains 1000 subroutines and you run a program than only calls 6 then only the pages that are touched by code for those 6 functions must get placed in real memory. The rest of the object code stays on disk. Program and library text is demand paged. The loading is on page boundries, not subroutine boundries. With a shared library, if I run a program on a node and touch 6 subroutines and you run a different program that touches those 6 and 10 more then code for all 16 subroutines may be kept in memory but the rest of the library will stay on disk. You and I will share the object code for the 6 subroutines we are both calling. Someone who wanted to make a libmpi that has MPI-1sided or MPI-IO well isolated in the library structure so simple MPI jobs would not force this extra code into memory could do that today. The user does not need to promise not to call MPI-IO subroutines for them not to to take real memory. The "subsets" would need to be devised by the MPI implementor but would be transparent to the MPI user and not dictated by the standard. The "subsets" the user did not call would remain paged out. Perhaps all static data defined by the library will come into real memory for each process but is there much reduction from being able to somehow not bring in the static data MPI-IO would require because somebody had promised not to use it? Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: application/octet-stream Size: 156 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic07419.gif Type: application/octet-stream Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: application/octet-stream Size: 45 bytes Desc: not available URL: From wgropp at [hidden] Fri May 9 07:32:17 2008 From: wgropp at [hidden] (William Gropp) Date: Fri, 9 May 2008 07:32:17 -0500 Subject: [Mpi-22] Memory footprint concern In-Reply-To: Message-ID: On the memory footprint issue, I'd like to add that it is possible to structure an MPI implementation to only require the code associated with the routines that are used in the application, even when the application must be statically linked (as is the case for Blue Gene/ L, for example). This can be done even if there are actions that the application needs or wants to perform at the time of MPI_Finalize. There are some things that can be done to make this easier for the implementation, but (like Dick) I'm not convinced that a subset is required as a way to limit the memory footprint. Bill William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Fri May 9 07:54:03 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Fri, 9 May 2008 13:54:03 +0100 Subject: [Mpi-22] Memory footprint concern In-Reply-To: Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201521A3D@swsmsx413.ger.corp.intel.com> Dear Bill, Thanks. I would agree with you on that subsets are indeed not required, but their clear definition in the standard would promote equally clear and uniform implementation of the memory footprint minimizing approaches you and Dick referred to. I guess static linkage is the easiest case for this, as the linkers are smart enough nowadays to load only the required parts of the library, provided it's appropriately structured. Again, subsets may promote such a structuring. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of William Gropp Sent: Friday, May 09, 2008 2:32 PM To: MPI 2.2 Subject: Re: [Mpi-22] Memory footprint concern On the memory footprint issue, I'd like to add that it is possible to structure an MPI implementation to only require the code associated with the routines that are used in the application, even when the application must be statically linked (as is the case for Blue Gene/L, for example). This can be done even if there are actions that the application needs or wants to perform at the time of MPI_Finalize. There are some things that can be done to make this easier for the implementation, but (like Dick) I'm not convinced that a subset is required as a way to limit the memory footprint. Bill William Gropp Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From keith.d.underwood at [hidden] Fri May 9 13:34:33 2008 From: keith.d.underwood at [hidden] (Underwood, Keith D) Date: Fri, 9 May 2008 13:34:33 -0500 Subject: [Mpi-22] Memory footprint concern In-Reply-To: Message-ID: <1BABDF0B0BF60D428576BCC7646C0A8F01ACA786@rrsmsx411.amr.corp.intel.com> Imagine, for example, MPI running on a Cell SPE. Yes, it sounds crazy, but people are working on it. If you look at the state in MPI-2 today, and assume it will grow proportionally with complexity, MPI-3 could be really nasty in that context. So, while I agree that an arbitrarily large number of permutations is a terrible idea for everyone involved (implementers wouldn't leverage all of the options, ISVs wouldn't test them, people who write third party libraries would have a huge headache dealing with the arbitrary combination of features chosen by the application), it seems like it would be prudent to try to figure out how to provide some mechanisms in this direction. I don't know if this overlaps with your idea about assertions or not, but they do seem to be related. Keith ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Friday, May 09, 2008 3:12 AM To: MPI 2.2 Subject: Re: [Mpi-22] Memory footprint concern Hi Alexander If we would envision defining subsets in such a way that a machine with limited memory per node and un-fancy OS can clearly document what its MPI offers, maybe it is not a huge addition to complexity. For example if an MPI implementation could say it offers subsets: MPI 1.2 MPI-IO MPI-Intercomm Collectives and nothing else and the meaning would be well defined that may not be too bad. If you are expecting such a machine to have all or most of MPI-3 available and each user would select only the parts he wants at job launch time it seems to me the complexity for implementation and testing will be prohibitive. In particular if you hope subsets will give better performance because of what they leave out and really be compact because they leave out most unneeded bytes of code the number of permutations seem to me to make that a vain hope. I have been ambivalent about whether my assertions proposal really has much connection to the subsets concept mostly because I think to be useful assertions must be fairly simple. The only assertion I am 100% sure will pay off and be used is MPI_NO_EAGER_THROTTLE. There are MPI implementations today that act as if the assertion MPI_NO_EAGER_THROTTLE were present on all applications but that is not an option for a vendor MPI because if somebody contacts service and says "Your MPI violates the standard" a vendor like IBM must "fix" it. IBM MPI supports MPI_CANCEL on MPI_ISEND but at least one MPI does not because it is "too expensive". In effect they act like every application has the assertion MPI_NO_SEND_CANCELS. IBM MPI could make use of MPI_NO_REQUEST_MIX if it was available. Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 "Supalov, Alexander" "Supalov, Alexander" Sent by: mpi-22-bounces_at_[hidden] 05/08/2008 05:52 PM Please respond to "MPI 2.2" To "MPI 2.2" cc Subject Re: [Mpi-22] Memory footprint concern Dear Dick, By the looks of it, MPI-3 is going to be big. Petascale machines may not have OS we're accustomed to, dynamic libraries, and some other things. Smaller system libraries - and smaller MPI - may be needed there. Some of the envisioned MPI-3 features will be needed for some applications, some won't. Same with MPI-2 and MPI-1. Defining subsets may help to open a way to custom cut MPI libraries suitable for particular application classes. How subsets will be implemented is a different matter. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [ mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Thursday, May 08, 2008 11:23 PM To: MPI 2.2 Subject: [Mpi-22] Memory footprint concern Can somebody help me understand this "smaller memory footprint" issue that is part f the subsetting goal better. What systems does it affect? What does "memory footprint" really mean? In the 64 bit address space, virtual address range is not a problem. On systems I am most familiar with (AIX and I have been told Linux too), if you have a library that contains 1000 subroutines and you run a program than only calls 6 then only the pages that are touched by code for those 6 functions must get placed in real memory. The rest of the object code stays on disk. Program and library text is demand paged. The loading is on page boundries, not subroutine boundries. With a shared library, if I run a program on a node and touch 6 subroutines and you run a different program that touches those 6 and 10 more then code for all 16 subroutines may be kept in memory but the rest of the library will stay on disk. You and I will share the object code for the 6 subroutines we are both calling. Someone who wanted to make a libmpi that has MPI-1sided or MPI-IO well isolated in the library structure so simple MPI jobs would not force this extra code into memory could do that today. The user does not need to promise not to call MPI-IO subroutines for them not to to take real memory. The "subsets" would need to be devised by the MPI implementor but would be transparent to the MPI user and not dictated by the standard. The "subsets" the user did not call would remain paged out. Perhaps all static data defined by the library will come into real memory for each process but is there much reduction from being able to somehow not bring in the static data MPI-IO would require because somebody had promised not to use it? Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: application/octet-stream Size: 156 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image004.gif Type: application/octet-stream Size: 73 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image005.gif Type: application/octet-stream Size: 73 bytes Desc: not available URL: From treumann at [hidden] Fri May 9 15:48:08 2008 From: treumann at [hidden] (Richard Treumann) Date: Fri, 9 May 2008 16:48:08 -0400 Subject: [Mpi-22] Memory footprint concern In-Reply-To: <1BABDF0B0BF60D428576BCC7646C0A8F01ACA786@rrsmsx411.amr.corp.intel.com> Message-ID: Hi Keith Say the MPI implementor for the Cell SPE version of MPI did a good job of organizing his functionality (perhaps by making subsetting decisions that work for the structure of his implementation). Say there is an MPI application that only makes calls to the 6 essential MPI routines plus MPI_Allreduce and MPI_Bcast. When that application is statically bound to the MPI implementation by a smart linker that follows dependancy chains and only brings in needed subroutines don't you get the desired result? What more do you get in this case from having the MPI standard dictate the subsets and asking the user to declare what subsets he will need? If you leave it to the implementor to devise the subsets and change them as some ways of slicing and dicing libmpi prove more useful than others it may be better than having the standard lock down the dividing lines based on today's best guess. It does occur to me that the linker cannot tell if the MPI_Bcast call will need the code that supports MPI_Bcast on intercommunicators even though that is rarely used. That means if the application has an MPI_Bcast call, both (probably needed) intracomm- and (probably not needed) intercomm support enter the footprint. Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/09/2008 02:34:33 PM: > Imagine, for example, MPI running on a Cell SPE. Yes, it sounds > crazy, but people are working on it. If you look at the state in > MPI-2 today, and assume it will grow proportionally with complexity, > MPI-3 could be really nasty in that context. So, while I agree that > an arbitrarily large number of permutations is a terrible idea for > everyone involved (implementers wouldn’t leverage all of the > options, ISVs wouldn’t test them, people who write third party > libraries would have a huge headache dealing with the arbitrary > combination of features chosen by the application), it seems like it > would be prudent to try to figure out how to provide some mechanisms > in this direction. I don’t know if this overlaps with your idea > about assertions or not, but they do seem to be related. > > Keith > * -------------- next part -------------- An HTML attachment was scrubbed... URL: From keith.d.underwood at [hidden] Fri May 9 21:59:05 2008 From: keith.d.underwood at [hidden] (Underwood, Keith D) Date: Fri, 9 May 2008 21:59:05 -0500 Subject: [Mpi-22] Memory footprint concern In-Reply-To: Message-ID: <1BABDF0B0BF60D428576BCC7646C0A8F01ACA860@rrsmsx411.amr.corp.intel.com> I think Alexander hit on the answer to this: how is the user to know what is going to offer an implementation some advantage? What may look like it could offer a lot of benefit from a users perspective (ready send) could be considered low priority by the implementer. Especially in the constrained environments example, every implementer for a constrained environment could have a different perception of what they can make fast and small. So, an application that runs fine on a Cell SPE could blow up the code memory footprint on a Clearspeed. This is ultimately bad for the user. That is to say nothing of the dynamic memory footprint, which I am sure an implementer could trim with appropriate guarantees. I'm not sure official subsetting is the right answer or what the right approach for getting that functionality would be, but I am pretty sure that MPI on a Cell SPE is going to be a subset - whether it is official or not :-) Keith ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Friday, May 09, 2008 3:48 PM To: MPI 2.2 Subject: Re: [Mpi-22] Memory footprint concern Hi Keith Say the MPI implementor for the Cell SPE version of MPI did a good job of organizing his functionality (perhaps by making subsetting decisions that work for the structure of his implementation). Say there is an MPI application that only makes calls to the 6 essential MPI routines plus MPI_Allreduce and MPI_Bcast. When that application is statically bound to the MPI implementation by a smart linker that follows dependancy chains and only brings in needed subroutines don't you get the desired result? What more do you get in this case from having the MPI standard dictate the subsets and asking the user to declare what subsets he will need? If you leave it to the implementor to devise the subsets and change them as some ways of slicing and dicing libmpi prove more useful than others it may be better than having the standard lock down the dividing lines based on today's best guess. It does occur to me that the linker cannot tell if the MPI_Bcast call will need the code that supports MPI_Bcast on intercommunicators even though that is rarely used. That means if the application has an MPI_Bcast call, both (probably needed) intracomm- and (probably not needed) intercomm support enter the footprint. Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/09/2008 02:34:33 PM: > Imagine, for example, MPI running on a Cell SPE. Yes, it sounds > crazy, but people are working on it. If you look at the state in > MPI-2 today, and assume it will grow proportionally with complexity, > MPI-3 could be really nasty in that context. So, while I agree that > an arbitrarily large number of permutations is a terrible idea for > everyone involved (implementers wouldn't leverage all of the > options, ISVs wouldn't test them, people who write third party > libraries would have a huge headache dealing with the arbitrary > combination of features chosen by the application), it seems like it > would be prudent to try to figure out how to provide some mechanisms > in this direction. I don't know if this overlaps with your idea > about assertions or not, but they do seem to be related. > > Keith > * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Mon May 12 10:21:11 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Mon, 12 May 2008 16:21:11 +0100 Subject: [Mpi-22] Memory footprint concern In-Reply-To: Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201550A52@swsmsx413.ger.corp.intel.com> Dear Dick, Yes, in case of limited memory the preferred mode of subset operation would be to allow implementations reflect what part of MPI-3 they do support, rather than let application pick and choose. However, in a different context the concept of the subsets may be open to other uses, for example, by those who can shoulder the validation cost and justify this by increased performance of the subset combinations they offer. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Friday, May 09, 2008 3:12 AM To: MPI 2.2 Subject: Re: [Mpi-22] Memory footprint concern Hi Alexander If we would envision defining subsets in such a way that a machine with limited memory per node and un-fancy OS can clearly document what its MPI offers, maybe it is not a huge addition to complexity. For example if an MPI implementation could say it offers subsets: MPI 1.2 MPI-IO MPI-Intercomm Collectives and nothing else and the meaning would be well defined that may not be too bad. If you are expecting such a machine to have all or most of MPI-3 available and each user would select only the parts he wants at job launch time it seems to me the complexity for implementation and testing will be prohibitive. In particular if you hope subsets will give better performance because of what they leave out and really be compact because they leave out most unneeded bytes of code the number of permutations seem to me to make that a vain hope. I have been ambivalent about whether my assertions proposal really has much connection to the subsets concept mostly because I think to be useful assertions must be fairly simple. The only assertion I am 100% sure will pay off and be used is MPI_NO_EAGER_THROTTLE. There are MPI implementations today that act as if the assertion MPI_NO_EAGER_THROTTLE were present on all applications but that is not an option for a vendor MPI because if somebody contacts service and says "Your MPI violates the standard" a vendor like IBM must "fix" it. IBM MPI supports MPI_CANCEL on MPI_ISEND but at least one MPI does not because it is "too expensive". In effect they act like every application has the assertion MPI_NO_SEND_CANCELS. IBM MPI could make use of MPI_NO_REQUEST_MIX if it was available. Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 "Supalov, Alexander" "Supalov, Alexander" Sent by: mpi-22-bounces_at_[hidden] 05/08/2008 05:52 PM Please respond to "MPI 2.2" To "MPI 2.2" cc Subject Re: [Mpi-22] Memory footprint concern Dear Dick, By the looks of it, MPI-3 is going to be big. Petascale machines may not have OS we're accustomed to, dynamic libraries, and some other things. Smaller system libraries - and smaller MPI - may be needed there. Some of the envisioned MPI-3 features will be needed for some applications, some won't. Same with MPI-2 and MPI-1. Defining subsets may help to open a way to custom cut MPI libraries suitable for particular application classes. How subsets will be implemented is a different matter. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [ mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Thursday, May 08, 2008 11:23 PM To: MPI 2.2 Subject: [Mpi-22] Memory footprint concern Can somebody help me understand this "smaller memory footprint" issue that is part f the subsetting goal better. What systems does it affect? What does "memory footprint" really mean? In the 64 bit address space, virtual address range is not a problem. On systems I am most familiar with (AIX and I have been told Linux too), if you have a library that contains 1000 subroutines and you run a program than only calls 6 then only the pages that are touched by code for those 6 functions must get placed in real memory. The rest of the object code stays on disk. Program and library text is demand paged. The loading is on page boundries, not subroutine boundries. With a shared library, if I run a program on a node and touch 6 subroutines and you run a different program that touches those 6 and 10 more then code for all 16 subroutines may be kept in memory but the rest of the library will stay on disk. You and I will share the object code for the 6 subroutines we are both calling. Someone who wanted to make a libmpi that has MPI-1sided or MPI-IO well isolated in the library structure so simple MPI jobs would not force this extra code into memory could do that today. The user does not need to promise not to call MPI-IO subroutines for them not to to take real memory. The "subsets" would need to be devised by the MPI implementor but would be transparent to the MPI user and not dictated by the standard. The "subsets" the user did not call would remain paged out. Perhaps all static data defined by the library will come into real memory for each process but is there much reduction from being able to somehow not bring in the static data MPI-IO would require because somebody had promised not to use it? Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: application/octet-stream Size: 230 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: application/octet-stream Size: 45 bytes Desc: not available URL: From alexander.supalov at [hidden] Tue May 13 04:33:17 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Tue, 13 May 2008 10:33:17 +0100 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201550D29@swsmsx413.ger.corp.intel.com> Dear Dick, Thank you. Indeed, clear definition of the hints, directions, and assertions would be helpful, provided users can follow this. It seems to me that the selection of bit flags or Info would primarily be defined by the ease of implementation. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Thursday, May 08, 2008 7:41 PM To: MPI 2.2 Subject: Re: [Mpi-22] MPI_INIT assertions Hi Alexander First - changing the rules to let MPI_Info objects be created and manipulated before MPI_INIT is much more intrusive on implementations. If the MPI implementation uses some set of data structures to manage MPI opaque objects, those data structures probably do not exist before MPI_INIT. Passing a bit vector into MPI_INIT does not require significant reorganization of the MPI implementation but allowing MPI_Info creation and manipulation does. The concept of an "assertions" bit vector is not new for MPI (see last paragreph) so using it does not require inventing two mechanisms. You have found a couple examples that flirt with violating the concept of a "hint". For soft spawn I would argue that any soft spawn call that would have succeeded as a hard spawn must also succeed as a soft spawn no matter what hint you provide. That is, if you try to spawn with maxprocs=16 and you get 16 task your hint that you would take 8 has not resulted in semantic thin ice. If your hint said you take 32 tasks, MPI can give you 16 and again no semantic thin ice. Even if you ask for 16 on the call but agree to 8 with a hint and a hard spawn for 8 tasks "would have worked", libmpi is free to ignore the hint and fail the call because it cannot provide the 16. On wdir (+ host, arch, path, file) it gets fuzzy. An MPI implementation is free to ignore any of these and the user is free to omit them and hope the MPI implementation has reasonable defaults. It could be argued that the MPI implementation has the option of ignoring a hint when it cannot honor it and falling back to the default. That would fit the "hint" concept but I do not know if that was envisioned when these were put in MPI-2. Actually we may want to look at doing a careful catagorization of the kinds of inputs that an MPI_Info can convey. I may be making assumptions that are not justified by a strict reading of the standard when I imply that MPI_Info (key, value) pairs == hints. As part of MPI 3 we may want to specify that the MPI_Info object can carry several kinds if "info" to an MPI implementation after MPI_INIT. hints - can never break semantic or trigger error messages directives - do not break semantic but can make the difference between a run succeeding and failing (e.g.the named wdir does not exist so libmpi takes an error trying to use it) assertions - can alter semantic. If we want a way to put assertions on communicators for example I see no reason we could not use an Info object to pass them in more? In MPI_FILE_OPEN we have MPI_MODE_UNIQUE_OPEN as a mode flag and say that if the user says MPI_MODE_UNIQUE_OPEN and is wrong, his results cannot be ensured. Silent corruption of the file is possible. We do not use the word "assertion" though maybe we should have. In MPI 1sided synch operations we have an integer IN argument we call an "assertion" and it can take flages like MPI_MODE_NOCHECK, MPI_MODE_NOSTORE or MPI_MODE_NOPUT. We tell the user that if he provides an assertion that is invalid he cannot assume the program will work even though it would have without the assertion. Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 "Supalov, Alexander" "Supalov, Alexander" Sent by: mpi-22-bounces_at_[hidden] 05/08/2008 10:52 AM Please respond to "MPI 2.2" To "MPI 2.2" cc Subject Re: [Mpi-22] MPI_INIT assertions Dear Dick, Thank you, I think I start feeling the difference between the assertions and the hints. To be completely sure, where would you put the soft spawning info key: is this an assertion or a hint? What about other standardized info elements that materially influence the program execution or may even predetermine its success or failure (like wdir)? More generally, will the difference between assertions and hints justify the use of two different mechanisms to pass the data across the MPI boundary? Here's I think what we can save by dropping certain MPI features: * Dynamic process support: less overhead in the progress engine, easier global rank handling. * File I/O: smaller requests, easier wait/test functions. * One-sided ops: no passive target w/o MPI calls - no extra progress thread. * Communicator & group management: better memory footprint. * Non-blocking communication: easier ordering, simplified request handling. * Heterogeneity: better memory footprint, easier data handling. * Derived datatypes (especially those with holes): better memory footprint. * Message tagging: better support for stable dataflow exchanges, smaller packets. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [ mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Tuesday, May 06, 2008 5:21 PM To: MPI 2.2 Subject: Re: [Mpi-22] MPI_INIT assertions In my view, each assertion we define in the standard should be justified by a good explanation of how it allows an MPI implementation to be more efficient and what cost/risk it adds In effect, a cost/benefit analysis. No plausible rationale - no assertion. An assertion is a statement about the application. It is not a request for libmpi to "provide something". In my proposal, an application that correctly uses some assertion must still be written to work on an MPI implementation that ignores that assertion. An application that incorrectly uses some assertion can fail or give wrong answers. You can picture an assertion as the application author granting libmpi permission to "NOT provide something" at the MPI implementors discretion. In MPI 2 we maintained a careful distinction between hints and assertions. The MPI implementation is not allowed to give wrong results or error messages when an application provides an incorrect hint. If an application could provide a hint that it would not use MPI_CANCEL on a particular communicator and then called MPI_CANCEL, it would be OK for libmpi to use a really slow CANCEL protocol but not OK for it to issue a fatal error or ignore the CANCEL. That means that the freedom to "exploit" a hint in libmpi is very limited and less care is required of the user in providing hints. Much of what is being proposed should be treated as hints and could use a different and extensible mechanism. We can consider extending the MPI_Info usage in MPI-3 to address information passing that is hint-like. The MPI standard intends applications to be platform agnostic and source level portable and we should not break that. An assertions interface for making statements about the application will not limit this kind of portability. If there are no MPI_CANCEL calls today, there are no MPI_CANCEL calls after recompilation for a new platform. If we add features for platform tuning (like how many tasks per node or eager threshhold) we could damage portability. In my first outline, I threw in MPI-IO and MPI-1SC without any rationale for how they are useful. You added: "No communicator & group management", "No non-blocking communication", "No message tagging" and "No derived datatypes". It is not clear to me they are all useful so a decent rationale would be needed for each. That makes 6 bits that may be free. The thread support level parameter in MPI_INIT_THREAD is a request to the MPI library to provide some level of thread support and I question how cleanly that maps into the "assertions" model. The assertions model may only require one bit for "MPI_NO_THREAD_CONTENTION". That frees 3 more of the bits you suggest are already taken. If we conclude that all 4 thread support options are really useful it may be best to add the assertions flag to MPI_INIT_THREAD and keep the thread support controls as they are. Even the MPI_NO_THREAD_CONTENTION bit frees up then. My original thought was that we would add a new function that adds a parameter to MPI_INIT_THREAD. I am convinced that the number of assertions defined by the MPI standard should be very limited because each brings some risk and complicates life for library writers. If we end up with as many as 32 assertions I will apologize for having offered the idea in the first place and probably advocate a no vote. If MPI 2.2 provided only MPI_NO_EAGER_THROTTLE, MPI_NO_SEND_CANCEL, MPI_NO_REQUEST_MIX, MPI_NO_ANY_SOURCE and MPI_NO_REDUCTION_ORDER I would be content and debate about adding more could wait for MPI 3.0. The assertion that the application does not try to match persistent send/recv with nonpersistent recv/send would be a nice 2.2 bonus. That accounts for 6 bits. The query function could be something as simple as: MPI_Query_assertions(int *stated, int *exploited) The author of libfred could code in his Init_fred routine: int stated_by_app, exploited_by_lib; MPI_Query_assertions( &stated_by_app, &exploited_by_lib) if (exploited_by_lib & MPI_NO_SEND_CANCEL) fatal_error("Assertion MPI_NO_SEND_CANCEL is not compatable with Fred Lib") else if (stated_by_app & MPI_NO_SEND_CANCEL) warn("Assertion MPI_NO_SEND_CANCEL is not compatable with Fred Lib. It is not exploited by your MPI but your application will not port to an MPI that does exploit MPI_NO_SEND_CANCEL"); If Fred is really cautious he can code If (stated_by_app != 0) fatal_error("Fred does not like assertions") and see if anybody asks for some assertion to be tolerated in the next release of FredLib. You mention "No eager buffering" which is different than what I proposed and is not logically an "assertion". MPI applications should not depend semantically on eager buffering today and the standard is clear that a MPI implementation which has no eager buffering can still be 100% compliant. The standard requires that IF an MPI implementation provides an eager protocol as a performance enhancement, it must also provide a safety feature which can be expensive. Many (perhaps most) applications can benefit from an eager protocol but do not need the safety feature because the application is self regulating. These applications could assert: "I do not require the safety feature." which would allow them the benefit of eager protocol without the costs of the safety feature. I can see a rationale for some kind of hints related to eager protocol buffer space and threshhold. There are many other kinds of granular "hint" support that may be useful but all that should be kept distinct from this proposal (In my opinion). Regards - Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/06/2008 07:26:03 AM: > Dear Dick, > > Thanks. What about passing more verbose information to and from an > MPI implementation? Say, eager threshold value, if it's meaningful? > There are many less controversial examples, for example, to let the > caller know how many processes are placed onto the node, etc. Simple > assertions won't work in this case. > > Back to assertions. I thought we might consider > MPI_Initialized_subset(*flag,required,*provided) or something. > Setting "required" to 0 would return the set of provided assertions > in "provided". Setting "required" to something else will also set > the "flag" to true or false, depending on whether there's a match. > > The special MPI_Init_subset(*argc,***argv,required,*provided) would > complement this. > > Regarding 32 bits, let's count (I'm taking the joint list from the > subset proposal that includes your earlier points): > > 0,1,2,3 - MPI_THREAD_SINGLE, FUNNELED, SERIALIZED, MULTIPLE > 0x00000004 - No dynamic process support > 0x00000008 - No file I/O > 0x00000010 - No one-sided ops > 0x00000020 - No communicator & group management > 0x00000040 - No non-blocking communication > 0x00000080 - No message cancellation > 0x00000100 - Persistent ops on both sides > 0x00000200 - No heterogeneity > 0x00000400 - No derived datatypes (especially those with holes) > 0x00000800 - No MPI_ANY_SOURCE > 0x00001000 - No message tagging > 0x00002000 - No reduction order > 0x00004000 - No eager buffering > 0x00008000 - No mixed request types in wait/test > > We have 16 bits left. Note that there may be some predefined masks > for well defined combinations of the above (e.g., MPI-1 = no dynamic > processes, no file I/O, no threads, etc.). > > Best regards. > > Alexander > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Richard Treumann > Sent: Monday, May 05, 2008 11:53 PM > To: MPI 2.2 > Subject: [Mpi-22] MPI_INIT assertions > I changed the subject to be meaningful - it had been "Re: [Mpi-22] > mpi-22 Digest, Vol 2, Issue 7" > > I do think this proposal is within the scope of the MPI 2.2 rules so > I am pleased to hear this is being considered. > > I am 100% convinced that a query function should be part of this > proposal, probably with 2 query flavors. One flavor of the query > would respond with the set of assertions the application > MPI_INIT_xxx call had provided and another would respond with the > set of assertions the MPI implementation is actually exploiting. > > One library author may decide he will use if/else logic based on how > the MPI implementation will behave. The application may assert > MPI_NO_yyy but if the MPI implementation does not exploit MPI_NO_yyy > then the library can still use the code that depends on yyy. Another > library author may depend unconditionally on yyy. On an MPI > implementation that does not exploit MPI_NO_yyy he may want to issue > a warning that assertion MPI_NO_yyy is non-portable but let the job > run. On one that does exploit MPI_NO_yyy he would abort the job. Or, > he may decide to simply abort any job that asserts MPI_NO_yyy to > avoid having the library suddenly quit working when the customer > upgrades to an MPI implementation that does exploit MPI_NO_yyy. > > I think 32 assertions is probably more than enough > > Even a handful of defined assertions can raise testing costs. For > simple cases like MPI_NO_ANY_SOURCE it is easy for a user to judge > whether a piece of code is OK but if we begin to add subtle or > narrow semantic assertions it gets harder. Some library providers > will be tempted to say "I cannot proof read and test my code to a > degree that will allow me to accept 28 specific assertions and > forbid 4. I will simply forbid every subtle assertion I cannot > afford to test." > > I predict that for large applications developed by teams and for > community open source efforts the design leads will consider > requiring that all parts be written to live within a few carefully > chosen assertions. The design leads will not want to provide a list > of 26 subtle or narrow assertions and require that everyone respect > all 26 in the code they contribute. > > Many distinct assertions also could become a big test cost for MPI > implementors and customers who must qualify an MPI implementation > before trusting their business to it. If there were 64 or more > assertions, how would a tester decide what combinations of > assertions must be tested and then create suitable test cases? > > Dick > > Dick Treumann - MPI Team/TCEM > IBM Systems & Technology Group > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > > mpi-22-bounces_at_[hidden] wrote on 05/05/2008 01:28:06 PM: > > > Dear Dick, > > > > Thank you. We can actually introduce what you propose, possibly with > > a query function to make it still easier to live with, as early as > > in MPI 2.2, as a subset precursor. Judging by the discussion in > > Chicago, subsets may not need much more than that in the end, > > possibly with a little more flags and semantics added in MPI-3. > > > > The reservation against 32- (or for that matter, 64-) bit limitation > > is the only one I have at the moment. Not being able to attach > > assertions to communicators, etc. may be missed by some advanced > > programmers, but here we need to be pragmatic: who will ever want to > > go that deep? > > > > Best regards. > > > > Alexander > > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > > bounces_at_[hidden]] On Behalf Of Richard Treumann > > Sent: Monday, May 05, 2008 7:18 PM > > To: MPI 2.2 > > Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > > > Hi Alexander > > > > I have no objection to citing my "assertions" proposal in the > > subsetting discussions. I do want to keep it clear that this > > proposal is intended to be as simple as practical to implement, exploit and > > live with. > > > > "Live with" applies to 3rd party library authors or anyone else who > > must write MPI code but does not know and control the structure of > > the entire application. That guy must "live with" the decisions made > > by whoever coded the MPI_INIT piece. "Live with" also applies to > > whoever must test or certify a specific MPI implementation. > > > > Thanks - Dick > > > > Dick Treumann - MPI Team/TCEM > > IBM Systems & Technology Group > > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > > Tele (845) 433-7846 Fax (845) 433-8363 > > > > [image removed] "Supalov, Alexander" > > > > > > > "Supalov, Alexander" > > Sent by: mpi-22-bounces_at_[hidden] > > 04/26/2008 04:03 AM > > > > Please respond to > > "MPI 2.2" > > > > [image removed] > > To > > > > [image removed] > > "MPI 2.2" > > > > [image removed] > > cc > > > > [image removed] > > > > [image removed] > > Subject > > > > [image removed] > > Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > > > > [image removed] > > > > [image removed] > > > > > > Dear Dick, > > > > Thank you. Would you mind if I cite your proposal in the subsets > > discussion? Yours looks like a good alternative to the thinking of > > some of us that subsets might be very rich and mutable, and to > > Jeff's proposal on hints I've already cited there with his permission. > > > > Best regards. > > > > Alexander > > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > > bounces_at_[hidden]] On Behalf Of Richard Treumann > > Sent: Thursday, April 24, 2008 6:16 PM > > To: MPI 2.2 > > Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > > Dick Treumann - MPI Team/TCEM > > IBM Systems & Technology Group > > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > > Tele (845) 433-7846 Fax (845) 433-8363 > > > > > > mpi-22-bounces_at_[hidden] wrote on 04/24/2008 11:33:42 AM: > > > > > Hi, > > > > > > Note that this is an argument for making the assertions optional: those > > > who don't care don't have to use them. Those who care should use them > > > correctly or else. As usual. > > > > > > Best regards. > > > > > > Alexander > > > > > > > Hi Alexander > > > > The assertions are optional in this proposal. If this is added to > > the MPI standard the minimal impacts (day one impacts) are: > > > > == > > To application writers (none) - MPI_INIT and MPI_INIT_THREAD still > > work. MPI_INIT_THREAD_xxx can be > > passed 0 (zero) as the assertions bit vector. > > > > To MPI Implementors (small) - subroutine MPI_INIT_THREAD_xxx can be > > a clone of MPI_INIT_THREAD under the covers. If the Forum decides > > the query function is for asking what assertions are being honored, > > the implementation can just return "none" to every query. If there > > is also a query for what assertions have been made then there are a > > few more lines of code the implementor must write to preserve the > > value so it can be returned(maybe 10 lines) > > > > Writers of opaque libraries (small) - call the query function at > > library init time and if any assertions are found, issue an error > > message and kill the job. This is awkward for a library that wants > > to support every MPI whether it has implemented the new query > function or not. > > == > > > > As MPI implementations begin to take advantage of assertions there > > is more work for the MPI implementor and the library author must > > begin to think about whether his customer will be upset if the > > library simply outlaws all assertions. > > > > The library author will never be wrong if he simply forbids > > assertions forever. If they become valuable he will feel the > > pressure to work it out. > > > > The MPI implementor will never be wrong if he adds the API but > > simply ignores assertions forever. If they become valuable he will > > feel the pressure to honor some at least. > > --------------------------------------------------------------------- > > Intel GmbH > > Dornacher Strasse 1 > > 85622 Feldkirchen/Muenchen Germany > > Sitz der Gesellschaft: Feldkirchen bei Muenchen > > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > > VAT Registration No.: DE129385895 > > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > > > This e-mail and any attachments may contain confidential material for > > the sole use of the intended recipient(s). Any review or distribution > > by others is strictly prohibited. If you are not the intended > > recipient, please contact the sender and delete all copies. > > _______________________________________________ > > mpi-22 mailing list > > mpi-22_at_[hidden] > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > --------------------------------------------------------------------- > > Intel GmbH > > Dornacher Strasse 1 > > 85622 Feldkirchen/Muenchen Germany > > Sitz der Gesellschaft: Feldkirchen bei Muenchen > > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > > VAT Registration No.: DE129385895 > > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > > > This e-mail and any attachments may contain confidential material for > > the sole use of the intended recipient(s). Any review or distribution > > by others is strictly prohibited. If you are not the intended > > recipient, please contact the sender and delete all copies. > > _______________________________________________ > > mpi-22 mailing list > > mpi-22_at_[hidden] > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: application/octet-stream Size: 230 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: application/octet-stream Size: 45 bytes Desc: not available URL: From traff at [hidden] Tue May 13 07:54:12 2008 From: traff at [hidden] (Jesper Larsson Traeff) Date: Tue, 13 May 2008 14:54:12 +0200 Subject: [Mpi-22] Call for MPI 2.2 agenda items In-Reply-To: <005701c8aeef$a20d3350$860add8c@mcs.anl.gov> Message-ID: <20080513125412.GA1618@fourier.it.neclab.eu> Here are some items that are perhaps suitable for MPI-2.2 * MPI_Reduce_scatter_block * (possibly) built-in segmented Scan operations (new binary operators) * Correction/clarification to semantics of one-sided synchronization when switching between modes (NOTE: the proposal I posted some months back is deficient, as Rolf pointed out to me, and will have to be modified) I will try to make final proposals before the next meeting Jesper On Mon, May 05, 2008 at 03:36:01PM -0500, Rajeev Thakur wrote: > Now that MPI 2.1 is close to completion, it is time to look at issues for > MPI 2.2. Please send any items you may have for 2.2 to > mpi-22_at_[hidden] and/or upload them to the wiki (and let us know). > I had made a note of the following items from the last meeting that were > deferred to 2.2: > > * support for long long in maxloc, minloc > > * fix meaning of first two sentences in intercommunicator barrier > > * Ex 5.8 uses old style MPI_UB > > * Many functions say "integer" instead of "nonnegative integer" > > * const in C++ bindings > > * IN, OUT, and INOUT handle types broken > > * MPI_CHAR, MPI::CHAR should be signed char or char? > > * Need 64-bit sizes for count in messages > > * Sizes of MPI_Aint and MPI_Offset should not be different > > * New predefined types MPI_AINT and MPI_OFFSET > > * New predefined types corresponding to new types in C99, Fortran 2003, and > C++ some year > > > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 From bosilca at [hidden] Tue May 13 11:33:35 2008 From: bosilca at [hidden] (George Bosilca) Date: Tue, 13 May 2008 12:33:35 -0400 Subject: [Mpi-22] [Mpi-forum] Call for MPI 2.2 agenda items In-Reply-To: <20080513125412.GA1618@fourier.it.neclab.eu> Message-ID: I would like to propose an extension to the collective operators to allow them to be applied on non contiguous types, if and only if these types are composed by a single predefined type. The original matching rules (which operator support a specific predefined type) will continue to apply. As an example, this will allow users to apply a reduction operation on a trapezoidal matrix without having to define their own operator. As a result the reduction operation can benefit from all internal optimizations (such as pipelining) that usually are allowed only on contiguous types. george. On May 13, 2008, at 8:54 AM, Jesper Larsson Traeff wrote: > > Here are some items that are perhaps suitable for MPI-2.2 > > * MPI_Reduce_scatter_block > > * (possibly) built-in segmented Scan operations (new binary operators) > > * Correction/clarification to semantics of one-sided synchronization > when switching between modes (NOTE: the proposal I posted some months > back is deficient, as Rolf pointed out to me, and will have to be > modified) > > I will try to make final proposals before the next meeting > > Jesper > > On Mon, May 05, 2008 at 03:36:01PM -0500, Rajeev Thakur wrote: >> Now that MPI 2.1 is close to completion, it is time to look at >> issues for >> MPI 2.2. Please send any items you may have for 2.2 to >> mpi-22_at_[hidden] and/or upload them to the wiki (and let >> us know). >> I had made a note of the following items from the last meeting that >> were >> deferred to 2.2: >> >> * support for long long in maxloc, minloc >> >> * fix meaning of first two sentences in intercommunicator barrier >> >> * Ex 5.8 uses old style MPI_UB >> >> * Many functions say "integer" instead of "nonnegative integer" >> >> * const in C++ bindings >> >> * IN, OUT, and INOUT handle types broken >> >> * MPI_CHAR, MPI::CHAR should be signed char or char? >> >> * Need 64-bit sizes for count in messages >> >> * Sizes of MPI_Aint and MPI_Offset should not be different >> >> * New predefined types MPI_AINT and MPI_OFFSET >> >> * New predefined types corresponding to new types in C99, Fortran >> 2003, and >> C++ some year >> >> >> >> >> _______________________________________________ >> mpi-22 mailing list >> mpi-22_at_[hidden] >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > _______________________________________________ > mpi-forum mailing list > mpi-forum_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum * -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2423 bytes Desc: smime.p7s URL: From trj at [hidden] Tue May 13 17:19:07 2008 From: trj at [hidden] (Terry Jones) Date: Tue, 13 May 2008 15:19:07 -0700 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: Message-ID: The Assertions idea has much to offer. The pre MPI_Init case seems to be a clear winner, but can we generalize Assertions a little more without too heavy a cost? I would like to see Assertions extended to permit the application developer to provide additional context *during* the application run. Many MPI implementations have a whole laundry list of environment variables, but long running applications can go through several phases and a single value of a critically important environment variable may not make much sense. Perhaps Assertions could be used to change these type of tunables dynamically during the application. You can also imagine other possibilities to provide helpful context. For instance, perhaps the user could provide Assertions that would help MPI IO with read-ahead prefetching or write-behind, or even meta-data operations (e.g., later I will be creating one file per MPI task). Regards, Terry At 5:53 PM -0400 5/5/08, Richard Treumann wrote: >I changed the subject to be meaningful - it had been "Re: [Mpi-22] >mpi-22 Digest, Vol 2, Issue 7" > >I do think this proposal is within the scope of the MPI 2.2 rules so >I am pleased to hear this is being considered. > >I am 100% convinced that a query function should be part of this >proposal, probably with 2 query flavors. One flavor of the query >would respond with the set of assertions the application >MPI_INIT_xxx call had provided and another would respond with the >set of assertions the MPI implementation is actually exploiting. > >One library author may decide he will use if/else logic based on how >the MPI implementation will behave. The application may assert >MPI_NO_yyy but if the MPI implementation does not exploit MPI_NO_yyy >then the library can still use the code that depends on yyy. Another >library author may depend unconditionally on yyy. On an MPI >implementation that does not exploit MPI_NO_yyy he may want to issue >a warning that assertion MPI_NO_yyy is non-portable but let the job >run. On one that does exploit MPI_NO_yyy he would abort the job. Or, >he may decide to simply abort any job that asserts MPI_NO_yyy to >avoid having the library suddenly quit working when the customer >upgrades to an MPI implementation that does exploit MPI_NO_yyy. > >I think 32 assertions is probably more than enough > >Even a handful of defined assertions can raise testing costs. For >simple cases like MPI_NO_ANY_SOURCE it is easy for a user to judge >whether a piece of code is OK but if we begin to add subtle or >narrow semantic assertions it gets harder. Some library providers >will be tempted to say "I cannot proof read and test my code to a >degree that will allow me to accept 28 specific assertions and >forbid 4. I will simply forbid every subtle assertion I cannot >afford to test." > >I predict that for large applications developed by teams and for >community open source efforts the design leads will consider >requiring that all parts be written to live within a few carefully >chosen assertions. The design leads will not want to provide a list >of 26 subtle or narrow assertions and require that everyone respect >all 26 in the code they contribute. > >Many distinct assertions also could become a big test cost for MPI >implementors and customers who must qualify an MPI implementation >before trusting their business to it. If there were 64 or more >assertions, how would a tester decide what combinations of >assertions must be tested and then create suitable test cases? > >Dick > >Dick Treumann - MPI Team/TCEM >IBM Systems & Technology Group >Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 >Tele (845) 433-7846 Fax (845) 433-8363 > > >mpi-22-bounces_at_[hidden] wrote on 05/05/2008 01:28:06 PM: > >> Dear Dick, >> >> Thank you. We can actually introduce what you propose, possibly with >> a query function to make it still easier to live with, as early as >> in MPI 2.2, as a subset precursor. Judging by the discussion in >> Chicago, subsets may not need much more than that in the end, > > possibly with a little more flags and semantics added in MPI-3. >> >> The reservation against 32- (or for that matter, 64-) bit limitation >> is the only one I have at the moment. Not being able to attach >> assertions to communicators, etc. may be missed by some advanced >> programmers, but here we need to be pragmatic: who will ever want to >> go that deep? >> >> Best regards. >> >> Alexander >> >> From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- >> bounces_at_[hidden]] On Behalf Of Richard Treumann >> Sent: Monday, May 05, 2008 7:18 PM >> To: MPI 2.2 >> Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 > >> Hi Alexander >> >> I have no objection to citing my "assertions" proposal in the >> subsetting discussions. I do want to keep it clear that this >> proposal is intended to be as simple as practical to implement, exploit and >> live with. >> >> "Live with" applies to 3rd party library authors or anyone else who >> must write MPI code but does not know and control the structure of >> the entire application. That guy must "live with" the decisions made >> by whoever coded the MPI_INIT piece. "Live with" also applies to >> whoever must test or certify a specific MPI implementation. >> >> Thanks - Dick >> >> Dick Treumann - MPI Team/TCEM >> IBM Systems & Technology Group >> Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 >> Tele (845) 433-7846 Fax (845) 433-8363 >> >> [image removed] "Supalov, Alexander" >> > >> >> "Supalov, Alexander" >> Sent by: mpi-22-bounces_at_[hidden] >> 04/26/2008 04:03 AM >> >> Please respond to >> "MPI 2.2" >> >> [image removed] >> To >> >> [image removed] >> "MPI 2.2" >> >> [image removed] >> cc >> >> [image removed] >> >> [image removed] >> Subject >> >> [image removed] >> Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 >> >> [image removed] >> >> [image removed] >> >> >> Dear Dick, >> >> Thank you. Would you mind if I cite your proposal in the subsets >> discussion? Yours looks like a good alternative to the thinking of >> some of us that subsets might be very rich and mutable, and to >> Jeff's proposal on hints I've already cited there with his permission. >> >> Best regards. >> >> Alexander >> >> From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- >> bounces_at_[hidden]] On Behalf Of Richard Treumann >> Sent: Thursday, April 24, 2008 6:16 PM >> To: MPI 2.2 >> Subject: Re: [Mpi-22] mpi-22 Digest, Vol 2, Issue 7 >> Dick Treumann - MPI Team/TCEM >> IBM Systems & Technology Group >> Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 >> Tele (845) 433-7846 Fax (845) 433-8363 >> >> >> mpi-22-bounces_at_[hidden] wrote on 04/24/2008 11:33:42 AM: >> >> > Hi, >> > >> > Note that this is an argument for making the assertions optional: those >> > who don't care don't have to use them. Those who care should use them >> > correctly or else. As usual. >> > >> > Best regards. >> > >> > Alexander >> > >> >> Hi Alexander >> >> The assertions are optional in this proposal. If this is added to >> the MPI standard the minimal impacts (day one impacts) are: >> >> == >> To application writers (none) - MPI_INIT and MPI_INIT_THREAD still >> work. MPI_INIT_THREAD_xxx can be >> passed 0 (zero) as the assertions bit vector. >> >> To MPI Implementors (small) - subroutine MPI_INIT_THREAD_xxx can be >> a clone of MPI_INIT_THREAD under the covers. If the Forum decides >> the query function is for asking what assertions are being honored, >> the implementation can just return "none" to every query. If there >> is also a query for what assertions have been made then there are a >> few more lines of code the implementor must write to preserve the >> value so it can be returned(maybe 10 lines) >> >> Writers of opaque libraries (small) - call the query function at >> library init time and if any assertions are found, issue an error >> message and kill the job. This is awkward for a library that wants >> to support every MPI whether it has implemented the new query >>function or not. >> == >> >> As MPI implementations begin to take advantage of assertions there > > is more work for the MPI implementor and the library author must >> begin to think about whether his customer will be upset if the >> library simply outlaws all assertions. >> >> The library author will never be wrong if he simply forbids >> assertions forever. If they become valuable he will feel the >> pressure to work it out. >> >> The MPI implementor will never be wrong if he adds the API but >> simply ignores assertions forever. If they become valuable he will >> feel the pressure to honor some at least. >> --------------------------------------------------------------------- >> Intel GmbH >> Dornacher Strasse 1 >> 85622 Feldkirchen/Muenchen Germany >> Sitz der Gesellschaft: Feldkirchen bei Muenchen >> Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer >> Registergericht: Muenchen HRB 47456 Ust.-IdNr. >> VAT Registration No.: DE129385895 >> Citibank Frankfurt (BLZ 502 109 00) 600119052 >> >> This e-mail and any attachments may contain confidential material for >> the sole use of the intended recipient(s). Any review or distribution >> by others is strictly prohibited. If you are not the intended >> recipient, please contact the sender and delete all copies. >> _______________________________________________ >> mpi-22 mailing list >> mpi-22_at_[hidden] >> >>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 >> --------------------------------------------------------------------- >> Intel GmbH >> Dornacher Strasse 1 >> 85622 Feldkirchen/Muenchen Germany >> Sitz der Gesellschaft: Feldkirchen bei Muenchen >> Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer >> Registergericht: Muenchen HRB 47456 Ust.-IdNr. >> VAT Registration No.: DE129385895 >> Citibank Frankfurt (BLZ 502 109 00) 600119052 >> >> This e-mail and any attachments may contain confidential material for >> the sole use of the intended recipient(s). Any review or distribution >> by others is strictly prohibited. If you are not the intended >> recipient, please contact the sender and delete all copies. >> _______________________________________________ >> mpi-22 mailing list >> mpi-22_at_[hidden] >> >>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > >_______________________________________________ >mpi-22 mailing list >mpi-22_at_[hidden] >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From Dries.Kimpe at [hidden] Wed May 14 06:21:41 2008 From: Dries.Kimpe at [hidden] (Dries Kimpe) Date: Wed, 14 May 2008 13:21:41 +0200 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: Message-ID: <20080514112141.GA22911@mhdmobile.wlan.ntnu.no> * Terry Jones [2008-05-13 15:19:07]: > You can also imagine other possibilities to provide helpful context. For > instance, perhaps the user could provide Assertions that would help MPI IO > with read-ahead prefetching or write-behind, or even meta-data operations > (e.g., later I will be creating one file per MPI task). Things like read-ahead or write-behind clearly shouldn't be assertions but hints. (And, probably 'one file per MPI task' too -- if this is still going to be needed in 2 years) MPI already has hints that can capture some of the things mentioned above. access_style: (read_once, write_once, read_mostly, write_mostly, sequential, reverse_sequential, and random) sequential -> this can easily be used to turn on read-ahead IF THE MPI LIBRARY decides this is useful Assertions are bad -- they break compatibility -- and should only be tolerated if they provide real benefits and if the same cannot be obtained through existing mechanisms (hints, ...). In the examples mentioned, this is not the case. Dries * -------------- next part -------------- A non-text attachment was scrubbed... Name: 01-part Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From treumann at [hidden] Wed May 14 08:14:25 2008 From: treumann at [hidden] (Richard Treumann) Date: Wed, 14 May 2008 09:14:25 -0400 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: <20080514112141.GA22911@mhdmobile.wlan.ntnu.no> Message-ID: MPI_Init time assertions must be few and each must be valuable or the concept will fall like a house of cards. There is nothing in my proposal for MPI_Init time assertions that rules out providing other mechanisms in MPI 3 for giving guidance to the MPI implementation. In MPI 3 we can consider more hints and we can add the abiility to give stronger direction to MPI or provide it on a more granular basis - If it makes sense. These extra mechanisms are far to complex to consider as part of MPI 2.2. I would not use the phrase that Dries does when he says "Assertions are bad " but I agree with the the sentiment behind his statement. I think there should be a very small number of assertions defined in the standard and for each there should be a good rationale. For MPI 2.2 there should be a great rationale because we can come back in MPI 3 and add more assertions if we miss some important ones. It is much harder to remove one that turns out to be real trouble. Each assertion the the MPI Standard defines has the potential to break some piece of code that is valid MPI but that depends on semantic the assertion says is optional. The author of the routine that calls MPI_Init has the power to declare assertions and the authors of other parts of an applicaton must either live within the rules or explicitly shield against them. For project teams that develop complete applications, the decision to use an assertion belongs to the team, team leader or architect. If it is decided that an application will use a specific assertion it is the team lead who must make sure all developers understand the decision and write appropriate code. All testing will be done with the assertion in place. For third part libraries, the only option is to either forbid all assertions or explicitly pick some to allow. If there are 4 potential assertions, it is not very hard to decide for each one - "Will the library tolerate it?". If there are 50 assertions, library authors will seldom allow them all and will be more tempted to just say "No assertions allowed" because making judgements about each of 50 is too difficult. For Community developed code where people contribute source but are not under direct control of an architect or team lead, reviewing each submission for compliance with one or two assertions may be acceptable but reviewing for 50 each time somebody contributes new code is not. Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/14/2008 07:21:41 AM: > * Terry Jones [2008-05-13 15:19:07]: > > > You can also imagine other possibilities to provide helpful context. For > > instance, perhaps the user could provide Assertions that would help MPI IO > > with read-ahead prefetching or write-behind, or even meta-data operations > > (e.g., later I will be creating one file per MPI task). > > Things like read-ahead or write-behind clearly shouldn't be assertions but > hints. (And, probably 'one file per MPI task' too -- if this is still going to > be needed in 2 years) > > MPI already has hints that can capture some of the things mentioned > above. > > access_style: (read_once, write_once, read_mostly, write_mostly, sequential, > reverse_sequential, and random) > sequential -> this can easily be used to turn on read-ahead > IF THE MPI LIBRARY decides this is useful > > Assertions are bad -- they break compatibility -- and should only be > tolerated if they provide real benefits and if the same cannot be obtained > through existing mechanisms (hints, ...). > > In the examples mentioned, this is not the case. > > Dries > > [attachment "attia6gr.dat" deleted by Richard > Treumann/Poughkeepsie/IBM] _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlgraham at [hidden] Thu May 15 10:25:36 2008 From: rlgraham at [hidden] (Richard Graham) Date: Thu, 15 May 2008 11:25:36 -0400 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: Message-ID: We should be careful about making a change in MPI 2.2, knowing that we will likely turn around again in MPI 3.0 and change things again. If we are talking about changing the interface in 2.2, and then extending the assertions/hints in 3.0, this seems fine, but if we may want to change the interface yet again in 3.0, we should rethink things. I will add that if we are going to add some sort of ³info² argument to ŒMPI_Init()¹, we should deprecate MPI_Init() and MPI_Init_thread(), and include the threading specification in the ³info² object. Finally, before we decide on how to pass hints/assertions (if we do) to MPI_Init(), we should define a consistent way across the standard for passing information between the application and the library, as this is not the only instance where this is useful, and a uniform way of doing this makes things much easier on users. Rich On 5/14/08 9:14 AM, "Richard Treumann" wrote: > MPI_Init time assertions must be few and each must be valuable or the concept > will fall like a house of cards. > > There is nothing in my proposal for MPI_Init time assertions that rules out > providing other mechanisms in MPI 3 for giving guidance to the MPI > implementation. In MPI 3 we can consider more hints and we can add the > abiility to give stronger direction to MPI or provide it on a more granular > basis - If it makes sense. These extra mechanisms are far to complex to > consider as part of MPI 2.2. > > I would not use the phrase that Dries does when he says "Assertions are bad" > but I agree with the the sentiment behind his statement. I think there should > be a very small number of assertions defined in the standard and for each > there should be a good rationale. For MPI 2.2 there should be a great > rationale because we can come back in MPI 3 and add more assertions if we miss > some important ones. It is much harder to remove one that turns out to be real > trouble. > > Each assertion the the MPI Standard defines has the potential to break some > piece of code that is valid MPI but that depends on semantic the assertion > says is optional. The author of the routine that calls MPI_Init has the power > to declare assertions and the authors of other parts of an applicaton must > either live within the rules or explicitly shield against them. > > For project teams that develop complete applications, the decision to use an > assertion belongs to the team, team leader or architect. If it is decided > that an application will use a specific assertion it is the team lead who > must make sure all developers understand the decision and write appropriate > code. All testing will be done with the assertion in place. > > For third part libraries, the only option is to either forbid all assertions > or explicitly pick some to allow. If there are 4 potential assertions, it is > not very hard to decide for each one - "Will the library tolerate it?". If > there are 50 assertions, library authors will seldom allow them all and will > be more tempted to just say "No assertions allowed" because making judgements > about each of 50 is too difficult. > > For Community developed code where people contribute source but are not under > direct control of an architect or team lead, reviewing each submission for > compliance with one or two assertions may be acceptable but reviewing for 50 > each time somebody contributes new code is not. > > > > Dick Treumann - MPI Team/TCEM > IBM Systems & Technology Group > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > > mpi-22-bounces_at_[hidden] wrote on 05/14/2008 07:21:41 AM: > >> > * Terry Jones [2008-05-13 15:19:07]: >> > >>> > > You can also imagine other possibilities to provide helpful context. For >>> > > instance, perhaps the user could provide Assertions that would help MPI IO >>> > > with read-ahead prefetching or write-behind, or even meta-data >>> operations >>> > > (e.g., later I will be creating one file per MPI task). >> > >> > Things like read-ahead or write-behind clearly shouldn't be assertions but >> > hints. (And, probably 'one file per MPI task' too -- if this is still going >> to >> > be needed in 2 years) >> > >> > MPI already has hints that can capture some of the things mentioned >> > above. >> > >> > access_style: (read_once, write_once, read_mostly, write_mostly, >> sequential, >> > reverse_sequential, and random) >> > sequential -> this can easily be used to turn on read-ahead >> > IF THE MPI LIBRARY decides this is useful >> > >> > Assertions are bad -- they break compatibility -- and should only be >> > tolerated if they provide real benefits and if the same cannot be obtained >> > through existing mechanisms (hints, ...). >> > >> > In the examples mentioned, this is not the case. >> > >> > Dries >> > >> > [attachment "attia6gr.dat" deleted by Richard >> > Treumann/Poughkeepsie/IBM] _______________________________________________ >> > mpi-22 mailing list >> > mpi-22_at_[hidden] >> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Thu May 15 10:31:11 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Thu, 15 May 2008 16:31:11 +0100 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A201583399@swsmsx413.ger.corp.intel.com> Hi, We can pretty well pass data back and forth via MPI_COMM_WORLD attribute already now. This may be too late for assertions proposed by Rick, but seems adequate for everything else, provided we can describe how an implementation should react to the attribute setting action, and what attributes it should attach to the MPI_COMM_WORLD by default. Same with windows and files. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Graham Sent: Thursday, May 15, 2008 5:26 PM To: MPI 2.2 Subject: Re: [Mpi-22] MPI_INIT assertions We should be careful about making a change in MPI 2.2, knowing that we will likely turn around again in MPI 3.0 and change things again. If we are talking about changing the interface in 2.2, and then extending the assertions/hints in 3.0, this seems fine, but if we may want to change the interface yet again in 3.0, we should rethink things. I will add that if we are going to add some sort of "info" argument to 'MPI_Init()', we should deprecate MPI_Init() and MPI_Init_thread(), and include the threading specification in the "info" object. Finally, before we decide on how to pass hints/assertions (if we do) to MPI_Init(), we should define a consistent way across the standard for passing information between the application and the library, as this is not the only instance where this is useful, and a uniform way of doing this makes things much easier on users. Rich On 5/14/08 9:14 AM, "Richard Treumann" wrote: MPI_Init time assertions must be few and each must be valuable or the concept will fall like a house of cards. There is nothing in my proposal for MPI_Init time assertions that rules out providing other mechanisms in MPI 3 for giving guidance to the MPI implementation. In MPI 3 we can consider more hints and we can add the abiility to give stronger direction to MPI or provide it on a more granular basis - If it makes sense. These extra mechanisms are far to complex to consider as part of MPI 2.2. I would not use the phrase that Dries does when he says "Assertions are bad" but I agree with the the sentiment behind his statement. I think there should be a very small number of assertions defined in the standard and for each there should be a good rationale. For MPI 2.2 there should be a great rationale because we can come back in MPI 3 and add more assertions if we miss some important ones. It is much harder to remove one that turns out to be real trouble. Each assertion the the MPI Standard defines has the potential to break some piece of code that is valid MPI but that depends on semantic the assertion says is optional. The author of the routine that calls MPI_Init has the power to declare assertions and the authors of other parts of an applicaton must either live within the rules or explicitly shield against them. For project teams that develop complete applications, the decision to use an assertion belongs to the team, team leader or architect. If it is decided that an application will use a specific assertion it is the team lead who must make sure all developers understand the decision and write appropriate code. All testing will be done with the assertion in place. For third part libraries, the only option is to either forbid all assertions or explicitly pick some to allow. If there are 4 potential assertions, it is not very hard to decide for each one - "Will the library tolerate it?". If there are 50 assertions, library authors will seldom allow them all and will be more tempted to just say "No assertions allowed" because making judgements about each of 50 is too difficult. For Community developed code where people contribute source but are not under direct control of an architect or team lead, reviewing each submission for compliance with one or two assertions may be acceptable but reviewing for 50 each time somebody contributes new code is not. Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/14/2008 07:21:41 AM: > * Terry Jones [2008-05-13 15:19:07]: > > > You can also imagine other possibilities to provide helpful context. For > > instance, perhaps the user could provide Assertions that would help MPI IO > > with read-ahead prefetching or write-behind, or even meta-data operations > > (e.g., later I will be creating one file per MPI task). > > Things like read-ahead or write-behind clearly shouldn't be assertions but > hints. (And, probably 'one file per MPI task' too -- if this is still going to > be needed in 2 years) > > MPI already has hints that can capture some of the things mentioned > above. > > access_style: (read_once, write_once, read_mostly, write_mostly, sequential, > reverse_sequential, and random) > sequential -> this can easily be used to turn on read-ahead > IF THE MPI LIBRARY decides this is useful > > Assertions are bad -- they break compatibility -- and should only be > tolerated if they provide real benefits and if the same cannot be obtained > through existing mechanisms (hints, ...). > > In the examples mentioned, this is not the case. > > Dries > > [attachment "attia6gr.dat" deleted by Richard > Treumann/Poughkeepsie/IBM] _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 ________________________________ _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Thu May 15 10:33:55 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Thu, 15 May 2008 16:33:55 +0100 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A2015833A3@swsmsx413.ger.corp.intel.com> Re. MPI_Info before MPI_Init: if special MPI initialized or owned memory is used for implementing MPI_Info, getting it initialized ahead of the MPI_Init may be tough. If bit flags are appropriate for assertions, I'd say we should go with that, and possibly fold required threading support level into them as well, as discussed earlier. ________________________________ From: Supalov, Alexander Sent: Thursday, May 15, 2008 5:31 PM To: 'MPI 2.2' Subject: RE: [Mpi-22] MPI_INIT assertions Hi, We can pretty well pass data back and forth via MPI_COMM_WORLD attribute already now. This may be too late for assertions proposed by Rick, but seems adequate for everything else, provided we can describe how an implementation should react to the attribute setting action, and what attributes it should attach to the MPI_COMM_WORLD by default. Same with windows and files. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Graham Sent: Thursday, May 15, 2008 5:26 PM To: MPI 2.2 Subject: Re: [Mpi-22] MPI_INIT assertions We should be careful about making a change in MPI 2.2, knowing that we will likely turn around again in MPI 3.0 and change things again. If we are talking about changing the interface in 2.2, and then extending the assertions/hints in 3.0, this seems fine, but if we may want to change the interface yet again in 3.0, we should rethink things. I will add that if we are going to add some sort of "info" argument to 'MPI_Init()', we should deprecate MPI_Init() and MPI_Init_thread(), and include the threading specification in the "info" object. Finally, before we decide on how to pass hints/assertions (if we do) to MPI_Init(), we should define a consistent way across the standard for passing information between the application and the library, as this is not the only instance where this is useful, and a uniform way of doing this makes things much easier on users. Rich On 5/14/08 9:14 AM, "Richard Treumann" wrote: MPI_Init time assertions must be few and each must be valuable or the concept will fall like a house of cards. There is nothing in my proposal for MPI_Init time assertions that rules out providing other mechanisms in MPI 3 for giving guidance to the MPI implementation. In MPI 3 we can consider more hints and we can add the abiility to give stronger direction to MPI or provide it on a more granular basis - If it makes sense. These extra mechanisms are far to complex to consider as part of MPI 2.2. I would not use the phrase that Dries does when he says "Assertions are bad" but I agree with the the sentiment behind his statement. I think there should be a very small number of assertions defined in the standard and for each there should be a good rationale. For MPI 2.2 there should be a great rationale because we can come back in MPI 3 and add more assertions if we miss some important ones. It is much harder to remove one that turns out to be real trouble. Each assertion the the MPI Standard defines has the potential to break some piece of code that is valid MPI but that depends on semantic the assertion says is optional. The author of the routine that calls MPI_Init has the power to declare assertions and the authors of other parts of an applicaton must either live within the rules or explicitly shield against them. For project teams that develop complete applications, the decision to use an assertion belongs to the team, team leader or architect. If it is decided that an application will use a specific assertion it is the team lead who must make sure all developers understand the decision and write appropriate code. All testing will be done with the assertion in place. For third part libraries, the only option is to either forbid all assertions or explicitly pick some to allow. If there are 4 potential assertions, it is not very hard to decide for each one - "Will the library tolerate it?". If there are 50 assertions, library authors will seldom allow them all and will be more tempted to just say "No assertions allowed" because making judgements about each of 50 is too difficult. For Community developed code where people contribute source but are not under direct control of an architect or team lead, reviewing each submission for compliance with one or two assertions may be acceptable but reviewing for 50 each time somebody contributes new code is not. Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/14/2008 07:21:41 AM: > * Terry Jones [2008-05-13 15:19:07]: > > > You can also imagine other possibilities to provide helpful context. For > > instance, perhaps the user could provide Assertions that would help MPI IO > > with read-ahead prefetching or write-behind, or even meta-data operations > > (e.g., later I will be creating one file per MPI task). > > Things like read-ahead or write-behind clearly shouldn't be assertions but > hints. (And, probably 'one file per MPI task' too -- if this is still going to > be needed in 2 years) > > MPI already has hints that can capture some of the things mentioned > above. > > access_style: (read_once, write_once, read_mostly, write_mostly, sequential, > reverse_sequential, and random) > sequential -> this can easily be used to turn on read-ahead > IF THE MPI LIBRARY decides this is useful > > Assertions are bad -- they break compatibility -- and should only be > tolerated if they provide real benefits and if the same cannot be obtained > through existing mechanisms (hints, ...). > > In the examples mentioned, this is not the case. > > Dries > > [attachment "attia6gr.dat" deleted by Richard > Treumann/Poughkeepsie/IBM] _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 ________________________________ _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From trj at [hidden] Thu May 15 11:58:17 2008 From: trj at [hidden] (Terry Jones) Date: Thu, 15 May 2008 09:58:17 -0700 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: Message-ID: If we are to allow an extensible mechanism for Assertions, then I like the idea of a "Hints/Info" like interface where one may query to find out what the implementation supports. However, an Assertions version would need to be clearly distinct from Hints to adequately convey that Assertions *may change the semantic rules* (which Hints do not). An Extensible Assertions interface would presumably allow for many optimizations by relaxing semantics that are unneeded by the application. However, for Assertions the application must be aware because they can change the outcome of the application. I mentioned read-ahead as an example. Asserting a read-ahead of a given size would permit different results for files opened Read+Write on a file system that provides strong consistency semantics. I believe Extensible Assertions would be utilized by applications because as application teams currently move from platform to platform they make themselves aware of environment variable tunables and use such tunables. I would like to see applications dynamically change these tunables during a run to accommodate different application phases efficiently. Allowing such a mechanism to alter semantics gives freedom for increased performance (albeit at the cost that the application must be aware of what they are doing, and there is added complexity for all involved). I would also support Dick's original thought of a small, pre-defined set of Assertions. This is a small step in the direction of Extensible Assertions (with smaller opportunity for misuse and 'easier to swallow' adoption), and could be designed with an interface that would later permit Extensible Assertions. But I would keep the interface sufficiently distinct from Hints to avoid confusion. Regards, Terry At 11:25 AM -0400 5/15/08, Richard Graham wrote: >We should be careful about making a change in MPI 2.2, knowing that >we will likely turn around again > in MPI 3.0 and change things again. If we are talking about >changing the interface in 2.2, and > then extending the assertions/hints in 3.0, this seems fine, but if >we may want to change the > interface yet again in 3.0, we should rethink things. >I will add that if we are going to add some sort of "info" argument >to 'MPI_Init()', we should deprecate > MPI_Init() and MPI_Init_thread(), and include the threading >specification in the "info" object. >Finally, before we decide on how to pass hints/assertions (if we do) >to MPI_Init(), we should > define a consistent way across the standard for passing information >between the application and > the library, as this is not the only instance where this is useful, >and a uniform way of doing this > makes things much easier on users. > >Rich > > >On 5/14/08 9:14 AM, "Richard Treumann" wrote: > >MPI_Init time assertions must be few and each must be valuable or >the concept will fall like a house of cards. > >There is nothing in my proposal for MPI_Init time assertions that >rules out providing other mechanisms in MPI 3 for giving guidance to >the MPI implementation. In MPI 3 we can consider more hints and we >can add the abiility to give stronger direction to MPI or provide it >on a more granular basis - If it makes sense. These extra mechanisms >are far to complex to consider as part of MPI 2.2. > >I would not use the phrase that Dries does when he says "Assertions >are bad" but I agree with the the sentiment behind his statement. I >think there should be a very small number of assertions defined in >the standard and for each there should be a good rationale. For MPI >2.2 there should be a great rationale because we can come back in >MPI 3 and add more assertions if we miss some important ones. It is >much harder to remove one that turns out to be real trouble. > >Each assertion the the MPI Standard defines has the potential to >break some piece of code that is valid MPI but that depends on >semantic the assertion says is optional. The author of the routine >that calls MPI_Init has the power to declare assertions and the >authors of other parts of an applicaton must either live within the >rules or explicitly shield against them. > >For project teams that develop complete applications, the decision >to use an assertion belongs to the team, team leader or architect. > If it is decided that an application will use a specific assertion >it is the team lead who must make sure all developers understand >the decision and write appropriate code. All testing will be done >with the assertion in place. > >For third part libraries, the only option is to either forbid all >assertions or explicitly pick some to allow. If there are 4 >potential assertions, it is not very hard to decide for each one - >"Will the library tolerate it?". If there are 50 assertions, >library authors will seldom allow them all and will be more tempted >to just say "No assertions allowed" because making judgements about >each of 50 is too difficult. > >For Community developed code where people contribute source but are >not under direct control of an architect or team lead, reviewing >each submission for compliance with one or two assertions may be >acceptable but reviewing for 50 each time somebody contributes new >code is not. > > > >Dick Treumann - MPI Team/TCEM >IBM Systems & Technology Group >Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 >Tele (845) 433-7846 Fax (845) 433-8363 > > >mpi-22-bounces_at_[hidden] wrote on 05/14/2008 07:21:41 AM: > >> * Terry Jones [2008-05-13 15:19:07]: >> >> > You can also imagine other possibilities to provide helpful context. For >> > instance, perhaps the user could provide Assertions that would help MPI IO >> > with read-ahead prefetching or write-behind, or even meta-data operations >> > (e.g., later I will be creating one file per MPI task). >> >> Things like read-ahead or write-behind clearly shouldn't be assertions but >> hints. (And, probably 'one file per MPI task' too -- if this is >>still going to >> be needed in 2 years) >> >> MPI already has hints that can capture some of the things mentioned >> above. >> >> access_style: (read_once, write_once, read_mostly, write_mostly, sequential, >> reverse_sequential, and random) >> sequential -> this can easily be used to turn on read-ahead >> IF THE MPI LIBRARY decides this is useful >> >> Assertions are bad -- they break compatibility -- and should only be >> tolerated if they provide real benefits and if the same cannot be obtained >> through existing mechanisms (hints, ...). >> >> In the examples mentioned, this is not the case. >> >> Dries >> >> [attachment "attia6gr.dat" deleted by Richard >> Treumann/Poughkeepsie/IBM] _______________________________________________ >> mpi-22 mailing list >> mpi-22_at_[hidden] >> >>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > >_______________________________________________ >mpi-22 mailing list >mpi-22_at_[hidden] >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > > >_______________________________________________ >mpi-22 mailing list >mpi-22_at_[hidden] >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Thu May 15 12:03:42 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Thu, 15 May 2008 18:03:42 +0100 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: Message-ID: <5ECAB1304A8B5B4CB3F9D6C01E4E21A20158343F@swsmsx413.ger.corp.intel.com> Hi, I bet some setting changes (like eager protocol threshold) may need to be global, or at least include all processes involved in a loosely synchronous, i.e., collective manner. Is this a part of the hints proposal? If not, changes to the settings effective on one side won't be visible on the other side. Enter chaos. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Terry Jones Sent: Thursday, May 15, 2008 6:58 PM To: MPI 2.2 Subject: Re: [Mpi-22] MPI_INIT assertions If we are to allow an extensible mechanism for Assertions, then I like the idea of a "Hints/Info" like interface where one may query to find out what the implementation supports. However, an Assertions version would need to be clearly distinct from Hints to adequately convey that Assertions *may change the semantic rules* (which Hints do not). An Extensible Assertions interface would presumably allow for many optimizations by relaxing semantics that are unneeded by the application. However, for Assertions the application must be aware because they can change the outcome of the application. I mentioned read-ahead as an example. Asserting a read-ahead of a given size would permit different results for files opened Read+Write on a file system that provides strong consistency semantics. I believe Extensible Assertions would be utilized by applications because as application teams currently move from platform to platform they make themselves aware of environment variable tunables and use such tunables. I would like to see applications dynamically change these tunables during a run to accommodate different application phases efficiently. Allowing such a mechanism to alter semantics gives freedom for increased performance (albeit at the cost that the application must be aware of what they are doing, and there is added complexity for all involved). I would also support Dick's original thought of a small, pre-defined set of Assertions. This is a small step in the direction of Extensible Assertions (with smaller opportunity for misuse and 'easier to swallow' adoption), and could be designed with an interface that would later permit Extensible Assertions. But I would keep the interface sufficiently distinct from Hints to avoid confusion. Regards, Terry At 11:25 AM -0400 5/15/08, Richard Graham wrote: We should be careful about making a change in MPI 2.2, knowing that we will likely turn around again in MPI 3.0 and change things again. If we are talking about changing the interface in 2.2, and then extending the assertions/hints in 3.0, this seems fine, but if we may want to change the interface yet again in 3.0, we should rethink things. I will add that if we are going to add some sort of "info" argument to 'MPI_Init()', we should deprecate MPI_Init() and MPI_Init_thread(), and include the threading specification in the "info" object. Finally, before we decide on how to pass hints/assertions (if we do) to MPI_Init(), we should define a consistent way across the standard for passing information between the application and the library, as this is not the only instance where this is useful, and a uniform way of doing this makes things much easier on users. Rich On 5/14/08 9:14 AM, "Richard Treumann" wrote: MPI_Init time assertions must be few and each must be valuable or the concept will fall like a house of cards. There is nothing in my proposal for MPI_Init time assertions that rules out providing other mechanisms in MPI 3 for giving guidance to the MPI implementation. In MPI 3 we can consider more hints and we can add the abiility to give stronger direction to MPI or provide it on a more granular basis - If it makes sense. These extra mechanisms are far to complex to consider as part of MPI 2.2. I would not use the phrase that Dries does when he says "Assertions are bad" but I agree with the the sentiment behind his statement. I think there should be a very small number of assertions defined in the standard and for each there should be a good rationale. For MPI 2.2 there should be a great rationale because we can come back in MPI 3 and add more assertions if we miss some important ones. It is much harder to remove one that turns out to be real trouble. Each assertion the the MPI Standard defines has the potential to break some piece of code that is valid MPI but that depends on semantic the assertion says is optional. The author of the routine that calls MPI_Init has the power to declare assertions and the authors of other parts of an applicaton must either live within the rules or explicitly shield against them. For project teams that develop complete applications, the decision to use an assertion belongs to the team, team leader or architect. If it is decided that an application will use a specific assertion it is the team lead who must make sure all developers understand the decision and write appropriate code. All testing will be done with the assertion in place. For third part libraries, the only option is to either forbid all assertions or explicitly pick some to allow. If there are 4 potential assertions, it is not very hard to decide for each one - "Will the library tolerate it?". If there are 50 assertions, library authors will seldom allow them all and will be more tempted to just say "No assertions allowed" because making judgements about each of 50 is too difficult. For Community developed code where people contribute source but are not under direct control of an architect or team lead, reviewing each submission for compliance with one or two assertions may be acceptable but reviewing for 50 each time somebody contributes new code is not. Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/14/2008 07:21:41 AM: > * Terry Jones [2008-05-13 15:19:07]: > > > You can also imagine other possibilities to provide helpful context. For > > instance, perhaps the user could provide Assertions that would help MPI IO > > with read-ahead prefetching or write-behind, or even meta-data operations > > (e.g., later I will be creating one file per MPI task). > > Things like read-ahead or write-behind clearly shouldn't be assertions but > hints. (And, probably 'one file per MPI task' too -- if this is still going to > be needed in 2 years) > > MPI already has hints that can capture some of the things mentioned > above. > > access_style: (read_once, write_once, read_mostly, write_mostly, sequential, > reverse_sequential, and random) > sequential -> this can easily be used to turn on read-ahead > IF THE MPI LIBRARY decides this is useful > > Assertions are bad -- they break compatibility -- and should only be > tolerated if they provide real benefits and if the same cannot be obtained > through existing mechanisms (hints, ...). > > In the examples mentioned, this is not the case. > > Dries > > [attachment "attia6gr.dat" deleted by Richard > Treumann/Poughkeepsie/IBM] _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 ________________________________ _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlgraham at [hidden] Thu May 15 12:33:02 2008 From: rlgraham at [hidden] (Richard Graham) Date: Thu, 15 May 2008 13:33:02 -0400 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: Message-ID: Assertions should not change semantics ­ these are defined by the standard. Assertions may provide ³help² to the implementation, what ever that means. I may be missing some thing, but I really don¹t see a difference (aside from name) between assertions an hints ­ both are ways of the user conveying information to the library. Rich On 5/15/08 12:58 PM, "Terry Jones" wrote: > > If we are to allow an extensible mechanism for Assertions, then I like the > idea of a "Hints/Info" like interface where one may query to find out what the > implementation supports. However, an Assertions version would need to be > clearly distinct from Hints to adequately convey that Assertions *may change > the semantic rules* (which Hints do not). > > An Extensible Assertions interface would presumably allow for many > optimizations by relaxing semantics that are unneeded by the application. > However, for Assertions the application must be aware because they can change > the outcome of the application. I mentioned read-ahead as an example. > Asserting a read-ahead of a given size would permit different results for > files opened Read+Write on a file system that provides strong consistency > semantics. > > I believe Extensible Assertions would be utilized by applications because as > application teams currently move from platform to platform they make > themselves aware of environment variable tunables and use such tunables. I > would like to see applications dynamically change these tunables during a run > to accommodate different application phases efficiently. Allowing such a > mechanism to alter semantics gives freedom for increased performance (albeit > at the cost that the application must be aware of what they are doing, and > there is added complexity for all involved). > > I would also support Dick's original thought of a small, pre-defined set of > Assertions. This is a small step in the direction of Extensible Assertions > (with smaller opportunity for misuse and 'easier to swallow' adoption), and > could be designed with an interface that would later permit Extensible > Assertions. But I would keep the interface sufficiently distinct from Hints to > avoid confusion. > > Regards, > Terry > > > At 11:25 AM -0400 5/15/08, Richard Graham wrote: >> We should be careful about making a change in MPI 2.2, knowing that we will >> likely turn around again >> in MPI 3.0 and change things again. If we are talking about changing the >> interface in 2.2, and >> then extending the assertions/hints in 3.0, this seems fine, but if we may >> want to change the >> interface yet again in 3.0, we should rethink things. >> I will add that if we are going to add some sort of "info" argument to >> 'MPI_Init()', we should deprecate >> MPI_Init() and MPI_Init_thread(), and include the threading specification in >> the "info" object. >> Finally, before we decide on how to pass hints/assertions (if we do) to >> MPI_Init(), we should >> define a consistent way across the standard for passing information between >> the application and >> the library, as this is not the only instance where this is useful, and a >> uniform way of doing this >> makes things much easier on users. >> >> Rich >> >> >> On 5/14/08 9:14 AM, "Richard Treumann" wrote: >>> MPI_Init time assertions must be few and each must be valuable or the >>> concept will fall like a house of cards. >>> >>> There is nothing in my proposal for MPI_Init time assertions that rules out >>> providing other mechanisms in MPI 3 for giving guidance to the MPI >>> implementation. In MPI 3 we can consider more hints and we can add the >>> abiility to give stronger direction to MPI or provide it on a more granular >>> basis - If it makes sense. These extra mechanisms are far to complex to >>> consider as part of MPI 2.2. >>> >>> I would not use the phrase that Dries does when he says "Assertions are bad" >>> but I agree with the the sentiment behind his statement. I think there >>> should be a very small number of assertions defined in the standard and for >>> each there should be a good rationale. For MPI 2.2 there should be a great >>> rationale because we can come back in MPI 3 and add more assertions if we >>> miss some important ones. It is much harder to remove one that turns out to >>> be real trouble. >>> >>> Each assertion the the MPI Standard defines has the potential to break some >>> piece of code that is valid MPI but that depends on semantic the assertion >>> says is optional. The author of the routine that calls MPI_Init has the >>> power to declare assertions and the authors of other parts of an applicaton >>> must either live within the rules or explicitly shield against them. >>> >>> For project teams that develop complete applications, the decision to use an >>> assertion belongs to the team, team leader or architect. If it is decided >>> that an application will use a specific assertion it is the team lead who >>> must make sure all developers understand the decision and write appropriate >>> code. All testing will be done with the assertion in place. >>> >>> For third part libraries, the only option is to either forbid all assertions >>> or explicitly pick some to allow. If there are 4 potential assertions, it >>> is not very hard to decide for each one - "Will the library tolerate it?". >>> If there are 50 assertions, library authors will seldom allow them all and >>> will be more tempted to just say "No assertions allowed" because making >>> judgements about each of 50 is too difficult. >>> >>> For Community developed code where people contribute source but are not >>> under direct control of an architect or team lead, reviewing each submission >>> for compliance with one or two assertions may be acceptable but reviewing >>> for 50 each time somebody contributes new code is not. >>> >>> >>> >>> Dick Treumann - MPI Team/TCEM >>> IBM Systems & Technology Group >>> Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 >>> Tele (845) 433-7846 Fax (845) 433-8363 >>> >>> >>> mpi-22-bounces_at_[hidden] wrote on 05/14/2008 07:21:41 AM: >>> >>>> > * Terry Jones [2008-05-13 15:19:07]: >>>> > >>>>> > > You can also imagine other possibilities to provide helpful context. For >>>>> > > instance, perhaps the user could provide Assertions that would help >>>>> MPI IO >>>>> > > with read-ahead prefetching or write-behind, or even meta-data >>>>> operations >>>>> > > (e.g., later I will be creating one file per MPI task). >>>> > >>>> > Things like read-ahead or write-behind clearly shouldn't be assertions >>>> but >>>> > hints. (And, probably 'one file per MPI task' too -- if this is still >>>> going to >>>> > be needed in 2 years) >>>> > >>>> > MPI already has hints that can capture some of the things mentioned >>>> > above. >>>> > >>>> > access_style: (read_once, write_once, read_mostly, write_mostly, >>>> sequential, >>>> > reverse_sequential, and random) >>>> > sequential -> this can easily be used to turn on read-ahead >>>> > IF THE MPI LIBRARY decides this is useful >>>> > >>>> > Assertions are bad -- they break compatibility -- and should only be >>>> > tolerated if they provide real benefits and if the same cannot be >>>> obtained >>>> > through existing mechanisms (hints, ...). >>>> > >>>> > In the examples mentioned, this is not the case. >>>> > >>>> > Dries >>>> > >>>> > [attachment "attia6gr.dat" deleted by Richard >>>> > Treumann/Poughkeepsie/IBM] >>>> _______________________________________________ >>>> > mpi-22 mailing list >>>> > mpi-22_at_[hidden] >>>> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 >>>> >>> >>> _______________________________________________ >>> mpi-22 mailing list >>> mpi-22_at_[hidden] >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 >>> >> >> >> _______________________________________________ >> mpi-22 mailing list >> mpi-22_at_[hidden] >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From treumann at [hidden] Thu May 15 16:44:19 2008 From: treumann at [hidden] (Richard Treumann) Date: Thu, 15 May 2008 17:44:19 -0400 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: Message-ID: Hi Rich The difference between a hint and an assertion is that an hint cannot change semantic in any way. An assertion will allow libmpi to relax a specific semantic guarantee. Lets take a simple case: The application wants to say it will not try to cancel any MPI_ISEND so it passes MPI_NO_SEND_CANCEL to MPI_INIT If MPI_INIT were passed MPI_NO_SEND_CANCEL as a hint and then the application went ahead and called MPI_CANCEL anyway, it would be fine for the MPI implementation to use a much slower MPI_CANCEL algorithm than if the hint had not been given. It would not be OK for the implementation to hang or raise an error or cancel the wrong message. If MPI_Init were passed MPI_NO_SEND_CANCEL as an assertion and then the application went ahead and called MPI_CANCEL anyway, it would be OK for the MPI implementation to raise an error for the MPI_CANCEL call. Every MPI Implementation must still be prepared to do MPI_CANCEL of an ISEND correctly if MPI_CANCEL is called by an application that did not provide the assertion. Adding assertion support in MPI 2.2 cannot break a correct MPI 2.1 application. The MPI_NO_SEND_CANCEL is only useful for cases where libmpi must devote extra memory, longer code paths or extra space in message headers across an entire run just to have the information it needs to do MPI_CANCEL correctly is somebody happens to call it. If a particular MPI implementation does not incur extra costs to be ready for a potential MPI_CANCEL then the implementation will ignore this assertion. Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/15/2008 01:33:02 PM: > Assertions should not change semantics – these are defined by the > standard. Assertions > may provide “help” to the implementation, what ever that means. I > may be missing some thing, > but I really don’t see a difference (aside from name) between > assertions an hints – both are > ways of the user conveying information to the library. > > Rich > * -------------- next part -------------- An HTML attachment was scrubbed... URL: From rlgraham at [hidden] Thu May 15 16:51:48 2008 From: rlgraham at [hidden] (Richard Graham) Date: Thu, 15 May 2008 17:51:48 -0400 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: Message-ID: I think we are splitting hairs on wording here. In one case the user is saying ³I don¹t care about the performance of cancel² and in the other ³I promise not to call cancel². Neither is changing the semantics of MPI. Rich On 5/15/08 5:44 PM, "Richard Treumann" wrote: > Hi Rich > > The difference between a hint and an assertion is that an hint cannot change > semantic in any way. An assertion will allow libmpi to relax a specific > semantic guarantee. Lets take a simple case: > > The application wants to say it will not try to cancel any MPI_ISEND so it > passes MPI_NO_SEND_CANCEL to MPI_INIT > > If MPI_INIT were passed MPI_NO_SEND_CANCEL as a hint and then the application > went ahead and called MPI_CANCEL anyway, it would be fine for the MPI > implementation to use a much slower MPI_CANCEL algorithm than if the hint had > not been given. It would not be OK for the implementation to hang or raise an > error or cancel the wrong message. > > If MPI_Init were passed MPI_NO_SEND_CANCEL as an assertion and then the > application went ahead and called MPI_CANCEL anyway, it would be OK for the > MPI implementation to raise an error for the MPI_CANCEL call. > > Every MPI Implementation must still be prepared to do MPI_CANCEL of an ISEND > correctly if MPI_CANCEL is called by an application that did not provide the > assertion. Adding assertion support in MPI 2.2 cannot break a correct MPI 2.1 > application. > > The MPI_NO_SEND_CANCEL is only useful for cases where libmpi must devote extra > memory, longer code paths or extra space in message headers across an entire > run just to have the information it needs to do MPI_CANCEL correctly is > somebody happens to call it. If a particular MPI implementation does not incur > extra costs to be ready for a potential MPI_CANCEL then the implementation > will ignore this assertion. > > Dick > > > > Dick Treumann - MPI Team/TCEM > IBM Systems & Technology Group > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > > mpi-22-bounces_at_[hidden] wrote on 05/15/2008 01:33:02 PM: > >> > Assertions should not change semantics ­ these are defined by the >> > standard. Assertions >> > may provide ³help² to the implementation, what ever that means. I >> > may be missing some thing, >> > but I really don¹t see a difference (aside from name) between >> > assertions an hints ­ both are >> > ways of the user conveying information to the library. >> > >> > Rich >> > > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From apittman at [hidden] Thu May 15 16:59:51 2008 From: apittman at [hidden] (Ashley Pittman) Date: Thu, 15 May 2008 22:59:51 +0100 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: Message-ID: <1210888791.8300.15.camel@bruce.priv.wark.uk.streamline-computing.com> On Thu, 2008-05-15 at 17:44 -0400, Richard Treumann wrote: > > The MPI_NO_SEND_CANCEL is only useful for cases where libmpi must > devote extra memory, longer code paths or extra space in message > headers across an entire run just to have the information it needs to > do MPI_CANCEL correctly is somebody happens to call it. If a > particular MPI implementation does not incur extra costs to be ready > for a potential MPI_CANCEL then the implementation will ignore this > assertion. To me I see little benefit in offering MPI_NO_SEND_CANCEL as a hint, all that does is say to the library that a cancel is unlikely to happen and we all know that anyway. As a cancel is still possible I see no opportunity for optimisation of any kind making itself possible, all the extra space in headers and longer code paths would still be needed anyway. As an assertion it would offer some scope for library writers to make simplifications leading to better performance, as a hint I can't see how it would change anything. Ashley, From treumann at [hidden] Fri May 16 07:18:43 2008 From: treumann at [hidden] (Richard Treumann) Date: Fri, 16 May 2008 08:18:43 -0400 Subject: [Mpi-22] MPI_INIT assertions In-Reply-To: <1210888791.8300.15.camel@bruce.priv.wark.uk.streamline-computing.com> Message-ID: Thank you Ashley - This is exactly the distinction between assertions and hints and the reason this narrow proposal advocates assertions and does not advocate hints. An MPI_NO_SEND_CANCEL hint still requires the MPI implementation to invest whatever it takes, across the entire application life to do MPI_CANCEL correctly. An MPI_NO_SEND_CANCEL assertion allows the MPI implementation to forgo those costs and just refuse to try to MPI_CANCEL if the application has an MPI_CANCEL call. The assertion does relax a semantic guarentee (I say relax rather than change because "change" has broader implications but an assertion does change semantics in a narrow and well defined way). An MPI Implementation that is prepared to do MPI_CANCEL correctly has a different semantic than one that will issue an error message if MPI_CANCEL is called. As a footnote - MPI 2 defined some hints and MPI 3 may define more hints. That is OK. There are cases where a hint will allow an MPI implementation to give better performance without changing its semantic. In a situation where a hint is enough, the MPI standard should avoid defining an assertion. Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/15/2008 05:59:51 PM: > > To me I see little benefit in offering MPI_NO_SEND_CANCEL as a hint, all > that does is say to the library that a cancel is unlikely to happen and > we all know that anyway. As a cancel is still possible I see no > opportunity for optimisation of any kind making itself possible, all the > extra space in headers and longer code paths would still be needed > anyway. > > As an assertion it would offer some scope for library writers to make > simplifications leading to better performance, as a hint I can't see how > it would change anything. > > Ashley, > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From treumann at [hidden] Fri May 16 13:51:48 2008 From: treumann at [hidden] (Richard Treumann) Date: Fri, 16 May 2008 14:51:48 -0400 Subject: [Mpi-22] MPI_INIT_ASSERTED proposal for MPI-2.2 Message-ID: Proposal for MPI Initialization time assertions in MPI 2.2 Define two new MPI functions: MPI_INIT_ASSERTED(assertions, threads, provided) [ IN assertions ] assertions about application behavior [ IN threads] desired level of thread support (integer) [ OUT provided] provided level of thread support (integer) int MPI_Init_asserted(int *argc, char *((*argv)[]), int assertions, int threads, int *provided) MPI_INIT_ASSERTED(ASSERTIONS, THREADS, PROVIDED, IERROR) INTEGER ASSERTIONS, THREADS, PROVIDED, IERROR (I will need help for C++ bindings ) The assertions arguments is to be provided as the logical OR of any subset of the assertions defined by the MPI standard. Zero is valid for an application that does not wish to make any assertion. The assertions are constants defined in mpi.h. MPI_QUERY_ASSERTIONS(stated, exploited) [ OUT stated] assertions as give in the MPI_INIT_ASSERTED call [ OUT exploited ] assertions the MPI implementation is exploiting int MPI_Query_assertions(int *stated, int *exploited) MPI_QUERY_ASSERTIONS(STATED, EXPLOITED, IERROR) INTEGER STATED, EXPLOITED, IERROR The outputs, STATED and EXPLOITED, can be checked for specific assertions with logical masks. For example to see if the application has used MPI_NO_SEND_CANCEL, library code could use: MPI_Query_assertions(&stated, &exploited); if (stated & MPI_NO_SEND_CANCEL) .... To confirm that no assertions are being exploited the library could use if (exploited) ..... error ==================================================================== Rationale There are certain semantic guarantees or features required by the MPI standard that can add significant memory overheads or performance costs. If an application does not depend on one of these features and can declare that fact at MPI initialization, the MPI implementation may be able to provide better performance. Assertions on MPI_INIT_ASSERTED let an application declare it does not require a specific feature and allow the implementation to omit support for that feature if there is a savings from doing so. When an application calls MPI_INIT_ASSERTED with a particular assertion the MPI implementation may depend on the application to act as the assertion promises. The behavior of an MPI application that uses or depends on some feature of MPI after asserting it does not is undefined. The application cannot depend on the assertion being exploited. The MPI application must be valid for an MPI implementation that ignores the assertion. Advise to implementors - When it is practical, a high quality MPI implementation will raise an error if an application violates an assertion that was provided at initialization It will not be practical for the MPI implementation to detect all assertion violations. End advise to implementors - The following assertions are available (defined in mpi.h) MPI_NO_EAGER_THROTTLE The MPI standard allows an implementation to send some messages eagerly, meaning the message can complete at the sender before a matching receive is posted. The implementation is required to hold the message data until the matching receive gets posted. If there is no space to buffer the data the send must block. The MPI job must not fail because it has run out of space or lose the data. Providing this guarantee can demand an expensive flow control. In many well structured applications there will never be a need to buffer significant numbers of eager messages but eager sends do help performance. These applications can tell the MPI implementation the guarantee is optional. MPI_NO_REDUCTION_DETERMINISM The MPI standard either requires or strongly urges that an MPI_REDUCE/MPI_ALLREDUCE give exactly the same answer every time. There are applications that do not need a "same answer" promise as long as they can assume they will get a "correct" answer. Maybe they can be provided a faster reduction algorithm. MPI_NO_DATATYPE_HETEROGENEITY MPI_Datatypes that are made from multiple types can be optimized if it is known there will never be a need to translate the data between heterogeneous nodes. A structure made of an MPI_INT followed by an MPI_FLOAT is likely to be 8 contiguous bytes with an extent of 8. An MPI implementation that "knows" it will not need to deal with data conversion can simplify the datatype commit logic and resulting internal representation by discarding the MPI_INT/MPI_FLOAT distinctions and just recording that the type is 8 bytes with a stride of 8. ============================== If we add MPI_INIT_ASSERTED to the MPI standard, MPI_INIT and MPI_INIT_THREAD continue to work as before. MPI_INIT_THREAD is equivalent to MPI_INIT_ASSERTED with zero as the assertion. We can consider deprecating the older initialization calls. Above are the assertions I have decided make most sense for the first pass. There are others that may be justified but a rationale is needed. If MPI 2.2 defines support for the assertion above we can consider others as part of MPI 3.0, Below are some That may be justified but will need a better rationale. MPI_NO_THREAD_CONTENTION (I am not sure this different from asking for MPI_THREAD_SERIALIZED but here is what it would mean) If the MPI library may be called by contending threads it must do something to protect internal state from conflicting updates. Whether this involves mutex locks or some other scheme, it is likely to have a performance cost. This assertion promises that the application will not make conflicting MPI calls. The MPI implementation can bypass any logic that defends state against multiple concurrent calls to MPI. The application may be multi-threaded in other aspects. MPI_NO_SEND_CANCEL To support MPI_CANCEL of a send, the MPI implementation may need to add extra information to every message header. If the application can assert that it does not try to cancel any sends then perhaps less state must be maintained and smaller message headers can be used. MPI_NO_ANY_SOURCE Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From apittman at [hidden] Wed May 21 10:40:58 2008 From: apittman at [hidden] (Ashley Pittman) Date: Wed, 21 May 2008 16:40:58 +0100 Subject: [Mpi-22] MPI_INIT_ASSERTED proposal for MPI-2.2 In-Reply-To: Message-ID: <1211384458.6385.81.camel@bruce.priv.wark.uk.streamline-computing.com> On Fri, 2008-05-16 at 14:51 -0400, Richard Treumann wrote: > MPI_NO_REDUCTION_DETERMINISM > > The MPI standard either requires or strongly urges that an > MPI_REDUCE/MPI_ALLREDUCE give exactly the > same answer every time. There are applications that do not need a > "same answer" promise as long as > they can assume they will get a "correct" answer. Maybe they can be > provided a faster reduction algorithm. This makes zero sense as a hint and would make the standard contradictory if adopted. I've already made it clear I would prefer assertions and this option would be useful in that context, if hints turn out the be chosen over assertions then this flag should be dropped. Clarification of the standard over the results of floating point reduction results would probably be a good thing anyway, for example is it OK for the different communicators of the same size but with different process layouts to give different answers given the same input data? I suspect the standard forbids this although without it it's probably impossible to do any shared memory optimisations. Ashley Pittman.