From wgropp at [hidden] Fri Dec 5 09:00:38 2008 From: wgropp at [hidden] (William Gropp) Date: Fri, 5 Dec 2008 09:00:38 -0600 Subject: [Mpi-22] MPI 2.2 Reminders Message-ID: <5C46E80D-F037-4FD4-A69B-86A40452E340@illinois.edu> Our next MPI 2.2 discussion is in just over a week, and we have a lot to do. Please have a look at the wiki for the following (see https://svn.mpi-forum.org/trac/mpi-forum-web/report for the reports): 1) For your own tickets, make sure that they are up to date. If reviewers are needed, please contact them and encourage them to submit their reviews. If revised text is needed, please update the text and if you would like early feedback, notify this list. 2) For items on which you are a reviewer (particularly MPI 2.1 chapter authors), please review the relevant tickets. When you are reviewing text, make sure that you are looking at the current MPI 2.1 standard; check page and line numbers as well as the context to ensure that the change is correct. 3) For other tickets, review them and raise any issues that you have *now* so that we can start the discussion before our meeting. Thanks! Bill William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign From bronis at [hidden] Fri Dec 5 10:00:42 2008 From: bronis at [hidden] (Bronis R. de Supinski) Date: Fri, 5 Dec 2008 08:00:42 -0800 (PST) Subject: [Mpi-22] MPI 2.2 Reminders In-Reply-To: <5C46E80D-F037-4FD4-A69B-86A40452E340@illinois.edu> Message-ID: Bill and Jeff: It will be really useful if the tickets listed reviewers. In the least we need some way to map tickets to reviewers (I am thinking about chapter authors in particular), Could Jeff add a field (or fields) that the owner is responsible for completing that indicates the reviewers? As it is I am really unclear over what I am supposed to review. I am guessing there are at least one or two for my chapter... Bronis On Fri, 5 Dec 2008, William Gropp wrote: > Our next MPI 2.2 discussion is in just over a week, and we have a lot > to do. Please have a look at the wiki for the following (see https:// svn.mpi-forum.org/trac/mpi-forum-web/report > for the reports): > > 1) For your own tickets, make sure that they are up to date. If > reviewers are needed, please contact them and encourage them to submit > their reviews. If revised text is needed, please update the text and > if you would like early feedback, notify this list. > > 2) For items on which you are a reviewer (particularly MPI 2.1 chapter > authors), please review the relevant tickets. When you are reviewing > text, make sure that you are looking at the current MPI 2.1 standard; > check page and line numbers as well as the context to ensure that the > change is correct. > > 3) For other tickets, review them and raise any issues that you have > *now* so that we can start the discussion before our meeting. > > Thanks! > > Bill > > William Gropp > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Paul and Cynthia Saylor Professor of Computer Science > University of Illinois Urbana-Champaign > > > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http:// lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > From wgropp at [hidden] Fri Dec 5 10:07:18 2008 From: wgropp at [hidden] (William Gropp) Date: Fri, 5 Dec 2008 10:07:18 -0600 Subject: [Mpi-22] MPI 2.2 Reminders In-Reply-To: Message-ID: As an immediate step, we can do the following: Authors of tickets that are ready for review should do the following: (a) add a comment to the ticket with the names of the reviewers (the relevant chapter author and 3 other volunteers) (b) contact the reviewers to remind them Having a separate, searchable field would be nice, but this would capture the data now. Bill On Dec 5, 2008, at 10:00 AM, Bronis R. de Supinski wrote: > > Bill and Jeff: > > It will be really useful if the tickets listed reviewers. In the least > we need some way to map tickets to reviewers (I am thinking about > chapter > authors in particular), Could Jeff add a field (or fields) that the > owner is responsible for completing that indicates the reviewers? > > As it is I am really unclear over what I am supposed to review. I am > guessing there are at least one or two for my chapter... > > Bronis > > > On Fri, 5 Dec 2008, William Gropp wrote: > >> Our next MPI 2.2 discussion is in just over a week, and we have a lot >> to do. Please have a look at the wiki for the following (see https:// >> svn.mpi-forum.org/trac/mpi-forum-web/report >> for the reports): >> >> 1) For your own tickets, make sure that they are up to date. If >> reviewers are needed, please contact them and encourage them to >> submit >> their reviews. If revised text is needed, please update the text and >> if you would like early feedback, notify this list. >> >> 2) For items on which you are a reviewer (particularly MPI 2.1 >> chapter >> authors), please review the relevant tickets. When you are reviewing >> text, make sure that you are looking at the current MPI 2.1 standard; >> check page and line numbers as well as the context to ensure that the >> change is correct. >> >> 3) For other tickets, review them and raise any issues that you have >> *now* so that we can start the discussion before our meeting. >> >> Thanks! >> >> Bill >> >> William Gropp >> Deputy Director for Research >> Institute for Advanced Computing Applications and Technologies >> Paul and Cynthia Saylor Professor of Computer Science >> University of Illinois Urbana-Champaign >> >> >> >> >> _______________________________________________ >> mpi-22 mailing list >> mpi-22_at_[hidden] >> http:// lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 >> >> > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign From jsquyres at [hidden] Fri Dec 5 10:10:00 2008 From: jsquyres at [hidden] (Jeff Squyres) Date: Fri, 5 Dec 2008 11:10:00 -0500 Subject: [Mpi-22] MPI 2.2 Reminders In-Reply-To: Message-ID: FWIW, I've been telling people to use the "CC" field to fill in the email addresses of reviewers. Not only does this make a searchable list of reviewers (by e-mail address, at least), but it also makes them get emails when the ticket changes. It ain't perfect, but it seems to work... On Dec 5, 2008, at 11:07 AM, William Gropp wrote: > As an immediate step, we can do the following: > > Authors of tickets that are ready for review should do the following: > (a) add a comment to the ticket with the names of the reviewers (the > relevant chapter author and 3 other volunteers) > (b) contact the reviewers to remind them > > Having a separate, searchable field would be nice, but this would > capture the data now. > > Bill > > On Dec 5, 2008, at 10:00 AM, Bronis R. de Supinski wrote: > >> >> Bill and Jeff: >> >> It will be really useful if the tickets listed reviewers. In the >> least >> we need some way to map tickets to reviewers (I am thinking about >> chapter >> authors in particular), Could Jeff add a field (or fields) that the >> owner is responsible for completing that indicates the reviewers? >> >> As it is I am really unclear over what I am supposed to review. I am >> guessing there are at least one or two for my chapter... >> >> Bronis >> >> >> On Fri, 5 Dec 2008, William Gropp wrote: >> >>> Our next MPI 2.2 discussion is in just over a week, and we have a >>> lot >>> to do. Please have a look at the wiki for the following (see https:// >>> svn.mpi-forum.org/trac/mpi-forum-web/report >>> for the reports): >>> >>> 1) For your own tickets, make sure that they are up to date. If >>> reviewers are needed, please contact them and encourage them to >>> submit >>> their reviews. If revised text is needed, please update the text >>> and >>> if you would like early feedback, notify this list. >>> >>> 2) For items on which you are a reviewer (particularly MPI 2.1 >>> chapter >>> authors), please review the relevant tickets. When you are >>> reviewing >>> text, make sure that you are looking at the current MPI 2.1 >>> standard; >>> check page and line numbers as well as the context to ensure that >>> the >>> change is correct. >>> >>> 3) For other tickets, review them and raise any issues that you have >>> *now* so that we can start the discussion before our meeting. >>> >>> Thanks! >>> >>> Bill >>> >>> William Gropp >>> Deputy Director for Research >>> Institute for Advanced Computing Applications and Technologies >>> Paul and Cynthia Saylor Professor of Computer Science >>> University of Illinois Urbana-Champaign >>> >>> >>> >>> >>> _______________________________________________ >>> mpi-22 mailing list >>> mpi-22_at_[hidden] >>> http:// lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 >>> >>> >> _______________________________________________ >> mpi-22 mailing list >> mpi-22_at_[hidden] >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > William Gropp > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Paul and Cynthia Saylor Professor of Computer Science > University of Illinois Urbana-Champaign > > > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 -- Jeff Squyres Cisco Systems From alexander.supalov at [hidden] Fri Dec 5 10:13:14 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Fri, 5 Dec 2008 16:13:14 +0000 Subject: [Mpi-22] MPI 2.2 Reminders In-Reply-To: Message-ID: <928CFBE8E7CB0040959E56B4EA41A77E08FD0F37@irsmsx504.ger.corp.intel.com> Hi, It would be great to learn how to identify tickets that are up for review in the first place. All colors and fields I can see in the standard reports don't seem to help much in this. Best regards. Alexander -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of William Gropp Sent: Friday, December 05, 2008 5:07 PM To: Bronis R. de Supinski; MPI 2.2 Subject: Re: [Mpi-22] MPI 2.2 Reminders As an immediate step, we can do the following: Authors of tickets that are ready for review should do the following: (a) add a comment to the ticket with the names of the reviewers (the relevant chapter author and 3 other volunteers) (b) contact the reviewers to remind them Having a separate, searchable field would be nice, but this would capture the data now. Bill On Dec 5, 2008, at 10:00 AM, Bronis R. de Supinski wrote: > > Bill and Jeff: > > It will be really useful if the tickets listed reviewers. In the least > we need some way to map tickets to reviewers (I am thinking about > chapter > authors in particular), Could Jeff add a field (or fields) that the > owner is responsible for completing that indicates the reviewers? > > As it is I am really unclear over what I am supposed to review. I am > guessing there are at least one or two for my chapter... > > Bronis > > > On Fri, 5 Dec 2008, William Gropp wrote: > >> Our next MPI 2.2 discussion is in just over a week, and we have a lot >> to do. Please have a look at the wiki for the following (see https:// >> svn.mpi-forum.org/trac/mpi-forum-web/report >> for the reports): >> >> 1) For your own tickets, make sure that they are up to date. If >> reviewers are needed, please contact them and encourage them to >> submit >> their reviews. If revised text is needed, please update the text and >> if you would like early feedback, notify this list. >> >> 2) For items on which you are a reviewer (particularly MPI 2.1 >> chapter >> authors), please review the relevant tickets. When you are reviewing >> text, make sure that you are looking at the current MPI 2.1 standard; >> check page and line numbers as well as the context to ensure that the >> change is correct. >> >> 3) For other tickets, review them and raise any issues that you have >> *now* so that we can start the discussion before our meeting. >> >> Thanks! >> >> Bill >> >> William Gropp >> Deputy Director for Research >> Institute for Advanced Computing Applications and Technologies >> Paul and Cynthia Saylor Professor of Computer Science >> University of Illinois Urbana-Champaign >> >> >> >> >> _______________________________________________ >> mpi-22 mailing list >> mpi-22_at_[hidden] >> http:// lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 >> >> > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. From bronis at [hidden] Fri Dec 5 12:34:33 2008 From: bronis at [hidden] (Bronis R. de Supinski) Date: Fri, 5 Dec 2008 10:34:33 -0800 (PST) Subject: [Mpi-22] MPI 2.2 Reminders In-Reply-To: Message-ID: Both solutions (Jeff's and Bill's -- particularly b) work for me. At this point, I have received no emails (either automatic or not) so I will assume nothing is ready for my review until I do. On Fri, 5 Dec 2008, Jeff Squyres wrote: > FWIW, I've been telling people to use the "CC" field to fill in the > email addresses of reviewers. > > Not only does this make a searchable list of reviewers (by e-mail > address, at least), but it also makes them get emails when the ticket > changes. > > It ain't perfect, but it seems to work... > > > On Dec 5, 2008, at 11:07 AM, William Gropp wrote: > > > As an immediate step, we can do the following: > > > > Authors of tickets that are ready for review should do the following: > > (a) add a comment to the ticket with the names of the reviewers (the > > relevant chapter author and 3 other volunteers) > > (b) contact the reviewers to remind them > > > > Having a separate, searchable field would be nice, but this would > > capture the data now. > > > > Bill > > > > On Dec 5, 2008, at 10:00 AM, Bronis R. de Supinski wrote: > > > >> > >> Bill and Jeff: > >> > >> It will be really useful if the tickets listed reviewers. In the > >> least > >> we need some way to map tickets to reviewers (I am thinking about > >> chapter > >> authors in particular), Could Jeff add a field (or fields) that the > >> owner is responsible for completing that indicates the reviewers? > >> > >> As it is I am really unclear over what I am supposed to review. I am > >> guessing there are at least one or two for my chapter... > >> > >> Bronis > >> > >> > >> On Fri, 5 Dec 2008, William Gropp wrote: > >> > >>> Our next MPI 2.2 discussion is in just over a week, and we have a > >>> lot > >>> to do. Please have a look at the wiki for the following (see https:// > >>> svn.mpi-forum.org/trac/mpi-forum-web/report > >>> for the reports): > >>> > >>> 1) For your own tickets, make sure that they are up to date. If > >>> reviewers are needed, please contact them and encourage them to > >>> submit > >>> their reviews. If revised text is needed, please update the text > >>> and > >>> if you would like early feedback, notify this list. > >>> > >>> 2) For items on which you are a reviewer (particularly MPI 2.1 > >>> chapter > >>> authors), please review the relevant tickets. When you are > >>> reviewing > >>> text, make sure that you are looking at the current MPI 2.1 > >>> standard; > >>> check page and line numbers as well as the context to ensure that > >>> the > >>> change is correct. > >>> > >>> 3) For other tickets, review them and raise any issues that you have > >>> *now* so that we can start the discussion before our meeting. > >>> > >>> Thanks! > >>> > >>> Bill > >>> > >>> William Gropp > >>> Deputy Director for Research > >>> Institute for Advanced Computing Applications and Technologies > >>> Paul and Cynthia Saylor Professor of Computer Science > >>> University of Illinois Urbana-Champaign > >>> > >>> > >>> > >>> > >>> _______________________________________________ > >>> mpi-22 mailing list > >>> mpi-22_at_[hidden] > >>> http:// lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > >>> > >>> > >> _______________________________________________ > >> mpi-22 mailing list > >> mpi-22_at_[hidden] > >> http:// lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > > > William Gropp > > Deputy Director for Research > > Institute for Advanced Computing Applications and Technologies > > Paul and Cynthia Saylor Professor of Computer Science > > University of Illinois Urbana-Champaign > > > > > > > > > > _______________________________________________ > > mpi-22 mailing list > > mpi-22_at_[hidden] > > http:// lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > > -- > Jeff Squyres > Cisco Systems > > > From thakur at [hidden] Fri Dec 5 14:37:31 2008 From: thakur at [hidden] (Rajeev Thakur) Date: Fri, 5 Dec 2008 14:37:31 -0600 Subject: [Mpi-22] MPI 2.2 Reminders In-Reply-To: <5C46E80D-F037-4FD4-A69B-86A40452E340@illinois.edu> Message-ID: BTW, I am not receiving emails for comments made to my tickets by reviewers whom I have added in the CC field, i.e., the owner of the ticket is not getting an email. Rajeev From treumann at [hidden] Fri Dec 5 15:01:48 2008 From: treumann at [hidden] (Richard Treumann) Date: Fri, 5 Dec 2008 16:01:48 -0500 Subject: [Mpi-22] MPI 2.2 Reminders In-Reply-To: <5C46E80D-F037-4FD4-A69B-86A40452E340@illinois.edu> Message-ID: Bill - I have updated my proposal for MPI initialization time assertions (Ticket 22)B. What needs to happen to put it before the Forum for another look? Unfortunately, I will not be able to attend the coming meeting to help explain the rationale in person. Dick Dick Treumann - MPI Team IBM Systems & Technology Group Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From thakur at [hidden] Fri Dec 5 15:25:23 2008 From: thakur at [hidden] (Rajeev Thakur) Date: Fri, 5 Dec 2008 15:25:23 -0600 Subject: [Mpi-22] MPI 2.2 Reminders In-Reply-To: Message-ID: <3D93D3642F224A41B7E9B761245F82DF@mcs.anl.gov> Maybe the system only had a userid for me, no email address. I have just added the email address. Let's see if I now receive emails for comments added. Rajeev > -----Original Message----- > From: mpi-22-bounces_at_[hidden] > [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Rajeev Thakur > Sent: Friday, December 05, 2008 2:38 PM > To: 'MPI 2.2' > Subject: Re: [Mpi-22] MPI 2.2 Reminders > > BTW, I am not receiving emails for comments made to my > tickets by reviewers > whom I have added in the CC field, i.e., the owner of the > ticket is not > getting an email. > > Rajeev > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > From erezh at [hidden] Fri Dec 5 15:48:10 2008 From: erezh at [hidden] (Erez Haba) Date: Fri, 5 Dec 2008 13:48:10 -0800 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) Message-ID: <6B68D01C00C9994A8E150183E62A119E7B5C9F7A5B@NA-EXMSG-C105.redmond.corp.microsoft.com> This proposal has passed 1st voting and needs reviewers. We need 3 volunteers to sign-off on this proposal, plus the 3 chapter authors to sign-off on the text. The Chapter Authors for Chapter 3: Point-to-Point Communication Chapter 5: Collective Communication Chapter 11: One-Sided Communication Please add a comment to the ticket saying that you reviewed the proposal, or please send me your comments. Send Buffer Access: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/45 Thanks, .Erez * -------------- next part -------------- An HTML attachment was scrubbed... URL: From erezh at [hidden] Fri Dec 5 15:50:50 2008 From: erezh at [hidden] (Erez Haba) Date: Fri, 5 Dec 2008 13:50:50 -0800 Subject: [Mpi-22] MPI 2.2 Reminders In-Reply-To: Message-ID: <6B68D01C00C9994A8E150183E62A119E7B5C9F7A69@NA-EXMSG-C105.redmond.corp.microsoft.com> Where can I find the list of chapter authors? -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of William Gropp Sent: Friday, December 05, 2008 8:07 AM To: Bronis R. de Supinski; MPI 2.2 Subject: Re: [Mpi-22] MPI 2.2 Reminders As an immediate step, we can do the following: Authors of tickets that are ready for review should do the following: (a) add a comment to the ticket with the names of the reviewers (the relevant chapter author and 3 other volunteers) (b) contact the reviewers to remind them Having a separate, searchable field would be nice, but this would capture the data now. Bill On Dec 5, 2008, at 10:00 AM, Bronis R. de Supinski wrote: > > Bill and Jeff: > > It will be really useful if the tickets listed reviewers. In the least > we need some way to map tickets to reviewers (I am thinking about > chapter > authors in particular), Could Jeff add a field (or fields) that the > owner is responsible for completing that indicates the reviewers? > > As it is I am really unclear over what I am supposed to review. I am > guessing there are at least one or two for my chapter... > > Bronis > > > On Fri, 5 Dec 2008, William Gropp wrote: > >> Our next MPI 2.2 discussion is in just over a week, and we have a lot >> to do. Please have a look at the wiki for the following (see https:// >> svn.mpi-forum.org/trac/mpi-forum-web/report >> for the reports): >> >> 1) For your own tickets, make sure that they are up to date. If >> reviewers are needed, please contact them and encourage them to >> submit >> their reviews. If revised text is needed, please update the text and >> if you would like early feedback, notify this list. >> >> 2) For items on which you are a reviewer (particularly MPI 2.1 >> chapter >> authors), please review the relevant tickets. When you are reviewing >> text, make sure that you are looking at the current MPI 2.1 standard; >> check page and line numbers as well as the context to ensure that the >> change is correct. >> >> 3) For other tickets, review them and raise any issues that you have >> *now* so that we can start the discussion before our meeting. >> >> Thanks! >> >> Bill >> >> William Gropp >> Deputy Director for Research >> Institute for Advanced Computing Applications and Technologies >> Paul and Cynthia Saylor Professor of Computer Science >> University of Illinois Urbana-Champaign >> >> >> >> >> _______________________________________________ >> mpi-22 mailing list >> mpi-22_at_[hidden] >> http:// lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 >> >> > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 From erezh at [hidden] Fri Dec 5 16:03:49 2008 From: erezh at [hidden] (Erez Haba) Date: Fri, 5 Dec 2008 14:03:49 -0800 Subject: [Mpi-22] please review - add const keyword to the C bindings Message-ID: <6B68D01C00C9994A8E150183E62A119E7B5C9F7A8C@NA-EXMSG-C105.redmond.corp.microsoft.com> This proposal has passed 1st voting and needs reviewers. We need 3 volunteers to sign-off on this proposal. The proposal is short so don't be intimidated. :) plus I need the chapter authors to sign-off on the changes to the API (that the text is correct). Chapter Authors (it's less than a handful lines for each chapter) Chapter 3: Point-to-Point Communication Chapter 4: Datatypes Chapter 5: Collective Communication Chapter 6: Groups, Contexts, Communicators, and Caching Chapter 7: Process Topologies Chapter 8: MPI Environmental Management Chapter 9: The Info Object Chapter 10: Process Creation and Management Chapter 11: One-Sided Communication Chapter 13: I/O Chapter 15: Deprecated Functions Chapter 16: Language Bindings Please add a comment to the ticket saying that you reviewed the proposal, or please send me your comments. Add const Keyword to the C bindings: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/46 Thanks, .Erez * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Fri Dec 5 16:05:24 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Fri, 5 Dec 2008 22:05:24 +0000 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: <6B68D01C00C9994A8E150183E62A119E7B5C9F7A5B@NA-EXMSG-C105.redmond.corp.microsoft.com> Message-ID: <928CFBE8E7CB0040959E56B4EA41A77E08FD0FE9@irsmsx504.ger.corp.intel.com> Hi, I'd like to review this proposal. Let's consider the following scenario: - In the MPI_Isend, MPI maps the send buffer into the address space of the receiving process. - In the matching MPI_Recv, the receiving process makes a copy of the mapped send buffer into the receive buffer. - Once the copy is complete, the send buffer is mapped back into the sender address space during the wait/test call. What will happen one tries to access the send buffer in between? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 10:48 PM To: MPI 2.2 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) This proposal has passed 1st voting and needs reviewers. We need 3 volunteers to sign-off on this proposal, plus the 3 chapter authors to sign-off on the text. The Chapter Authors for Chapter 3: Point-to-Point Communication Chapter 5: Collective Communication Chapter 11: One-Sided Communication Please add a comment to the ticket saying that you reviewed the proposal, or please send me your comments. Send Buffer Access: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/45 Thanks, .Erez --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From erezh at [hidden] Fri Dec 5 16:17:48 2008 From: erezh at [hidden] (Erez Haba) Date: Fri, 5 Dec 2008 14:17:48 -0800 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: <928CFBE8E7CB0040959E56B4EA41A77E08FD0FE9@irsmsx504.ger.corp.intel.com> Message-ID: <6B68D01C00C9994A8E150183E62A119E7B5C9F7AB3@NA-EXMSG-C105.redmond.corp.microsoft.com> I think that the idea is for the reviewers to check the text for any mistakes and compatibility with the existing text, rather than check for the validity of the proposal. The later as I recall is left for the MPI forum assembly. As for your question, I'm sure that you can answer it yourself. :) If the memory is still also mapped to the original process (as with shared memory) that everything is fine. If the memory is removed from the original process, than the app will get an access-violation fault. (if this system works on a page boundary, to take this action it needs to make sure that there are no other allocation on the same page) Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 2:05 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Hi, I'd like to review this proposal. Let's consider the following scenario: - In the MPI_Isend, MPI maps the send buffer into the address space of the receiving process. - In the matching MPI_Recv, the receiving process makes a copy of the mapped send buffer into the receive buffer. - Once the copy is complete, the send buffer is mapped back into the sender address space during the wait/test call. What will happen one tries to access the send buffer in between? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 10:48 PM To: MPI 2.2 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) This proposal has passed 1st voting and needs reviewers. We need 3 volunteers to sign-off on this proposal, plus the 3 chapter authors to sign-off on the text. The Chapter Authors for Chapter 3: Point-to-Point Communication Chapter 5: Collective Communication Chapter 11: One-Sided Communication Please add a comment to the ticket saying that you reviewed the proposal, or please send me your comments. Send Buffer Access: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/45 Thanks, .Erez --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Fri Dec 5 16:26:17 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Fri, 5 Dec 2008 22:26:17 +0000 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: <6B68D01C00C9994A8E150183E62A119E7B5C9F7AB3@NA-EXMSG-C105.redmond.corp.microsoft.com> Message-ID: <928CFBE8E7CB0040959E56B4EA41A77E08FD0FEB@irsmsx504.ger.corp.intel.com> Dear Erez, Thank you. I'm afraid I would need to have it explained in more detail why review may not include arguments on the substance. If something in the proposal makes one think that the proposed matter may be detrimental to the MPI standard and its implementations, I consider it one's duty to point this out. Following up on your reply: the segfault situation you described will make an MPI compliant program break. Thus, the implementation will have to keep the send buffer mapped into the sending process address space. This is a limitation on the MPI implementation that should be taken into account during the voting. Another possibility that has been pointed out earlier was that the proposed change disallows byte swap and other send buffer conversions to be done in place. At least one historical MPI implementation was doing this to a great avail. Who knows what is going to happen in the future? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 11:18 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) I think that the idea is for the reviewers to check the text for any mistakes and compatibility with the existing text, rather than check for the validity of the proposal. The later as I recall is left for the MPI forum assembly. As for your question, I'm sure that you can answer it yourself. :) If the memory is still also mapped to the original process (as with shared memory) that everything is fine. If the memory is removed from the original process, than the app will get an access-violation fault. (if this system works on a page boundary, to take this action it needs to make sure that there are no other allocation on the same page) Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 2:05 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Hi, I'd like to review this proposal. Let's consider the following scenario: - In the MPI_Isend, MPI maps the send buffer into the address space of the receiving process. - In the matching MPI_Recv, the receiving process makes a copy of the mapped send buffer into the receive buffer. - Once the copy is complete, the send buffer is mapped back into the sender address space during the wait/test call. What will happen one tries to access the send buffer in between? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 10:48 PM To: MPI 2.2 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) This proposal has passed 1st voting and needs reviewers. We need 3 volunteers to sign-off on this proposal, plus the 3 chapter authors to sign-off on the text. The Chapter Authors for Chapter 3: Point-to-Point Communication Chapter 5: Collective Communication Chapter 11: One-Sided Communication Please add a comment to the ticket saying that you reviewed the proposal, or please send me your comments. Send Buffer Access: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/45 Thanks, .Erez --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsquyres at [hidden] Fri Dec 5 16:45:06 2008 From: jsquyres at [hidden] (Jeff Squyres) Date: Fri, 5 Dec 2008 17:45:06 -0500 Subject: [Mpi-22] MPI 2.2 Reminders In-Reply-To: Message-ID: <998BE139-9914-4EBF-A7E8-92B7F2246849@cisco.com> Did you fill in your e-mail address on your Trac account? On Dec 5, 2008, at 3:37 PM, Rajeev Thakur wrote: > BTW, I am not receiving emails for comments made to my tickets by > reviewers > whom I have added in the CC field, i.e., the owner of the ticket is > not > getting an email. > > Rajeev > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 -- Jeff Squyres Cisco Systems From jsquyres at [hidden] Fri Dec 5 16:50:04 2008 From: jsquyres at [hidden] (Jeff Squyres) Date: Fri, 5 Dec 2008 17:50:04 -0500 Subject: [Mpi-22] MPI 2.2 Reminders In-Reply-To: <928CFBE8E7CB0040959E56B4EA41A77E08FD0F37@irsmsx504.ger.corp.intel.com> Message-ID: <4263FEE6-5988-43B4-BC42-3ACC97A1A636@cisco.com> I sent this diagram of ticket states around shortly after the last meeting, but I didn't get any comments back. So perhaps it would be better to send it to a wider audience. According to this diagram, any ticket in the "Not ready" / "Forum feedback requested" / "Waiting for reviews" state could be waiting for reviews. Perhaps we need something better than this (i.e., a separate bit indicating that a ticket is awaiting reviews). On Dec 5, 2008, at 11:13 AM, Supalov, Alexander wrote: > Hi, > > It would be great to learn how to identify tickets that are up for > review in the first place. All colors and fields I can see in the > standard reports don't seem to help much in this. > > Best regards. > > Alexander > > -----Original Message----- > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden] > ] On Behalf Of William Gropp > Sent: Friday, December 05, 2008 5:07 PM > To: Bronis R. de Supinski; MPI 2.2 > Subject: Re: [Mpi-22] MPI 2.2 Reminders > > As an immediate step, we can do the following: > > Authors of tickets that are ready for review should do the following: > (a) add a comment to the ticket with the names of the reviewers (the > relevant chapter author and 3 other volunteers) > (b) contact the reviewers to remind them > > Having a separate, searchable field would be nice, but this would > capture the data now. > > Bill > > On Dec 5, 2008, at 10:00 AM, Bronis R. de Supinski wrote: > >> >> Bill and Jeff: >> >> It will be really useful if the tickets listed reviewers. In the >> least >> we need some way to map tickets to reviewers (I am thinking about >> chapter >> authors in particular), Could Jeff add a field (or fields) that the >> owner is responsible for completing that indicates the reviewers? >> >> As it is I am really unclear over what I am supposed to review. I am >> guessing there are at least one or two for my chapter... >> >> Bronis >> >> >> On Fri, 5 Dec 2008, William Gropp wrote: >> >>> Our next MPI 2.2 discussion is in just over a week, and we have a >>> lot >>> to do. Please have a look at the wiki for the following (see https:// >>> svn.mpi-forum.org/trac/mpi-forum-web/report >>> for the reports): >>> >>> 1) For your own tickets, make sure that they are up to date. If >>> reviewers are needed, please contact them and encourage them to >>> submit >>> their reviews. If revised text is needed, please update the text >>> and >>> if you would like early feedback, notify this list. >>> >>> 2) For items on which you are a reviewer (particularly MPI 2.1 >>> chapter >>> authors), please review the relevant tickets. When you are >>> reviewing >>> text, make sure that you are looking at the current MPI 2.1 >>> standard; >>> check page and line numbers as well as the context to ensure that >>> the >>> change is correct. >>> >>> 3) For other tickets, review them and raise any issues that you have >>> *now* so that we can start the discussion before our meeting. >>> >>> Thanks! >>> >>> Bill -- Jeff Squyres Cisco Systems * -------------- next part -------------- A non-text attachment was scrubbed... Name: MPI-2.2_ticket_states.pdf Type: application/pdf Size: 381894 bytes Desc: MPI-2.2_ticket_states.pdf URL: From erezh at [hidden] Fri Dec 5 17:28:05 2008 From: erezh at [hidden] (Erez Haba) Date: Fri, 5 Dec 2008 15:28:05 -0800 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: <928CFBE8E7CB0040959E56B4EA41A77E08FD0FEB@irsmsx504.ger.corp.intel.com> Message-ID: <6B68D01C00C9994A8E150183E62A119E7B5C9F7B54@NA-EXMSG-C105.redmond.corp.microsoft.com> Dear Alexander, It is okay and encouraged for people to comment and argue on the proposals. You can add your comments to the ticket arguing your important points. The forum then consider that various points and vote on the proposal. However for the voting process we need people to review the text and confirm that it does not conflict with the standard and it is reasonable (from language pov) to be included in the standard. If we are willing to review the text and state that it valid for the standard, that would be great. If you have any comments on the text please send them to me. Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 2:26 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Erez, Thank you. I'm afraid I would need to have it explained in more detail why review may not include arguments on the substance. If something in the proposal makes one think that the proposed matter may be detrimental to the MPI standard and its implementations, I consider it one's duty to point this out. Following up on your reply: the segfault situation you described will make an MPI compliant program break. Thus, the implementation will have to keep the send buffer mapped into the sending process address space. This is a limitation on the MPI implementation that should be taken into account during the voting. Another possibility that has been pointed out earlier was that the proposed change disallows byte swap and other send buffer conversions to be done in place. At least one historical MPI implementation was doing this to a great avail. Who knows what is going to happen in the future? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 11:18 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) I think that the idea is for the reviewers to check the text for any mistakes and compatibility with the existing text, rather than check for the validity of the proposal. The later as I recall is left for the MPI forum assembly. As for your question, I'm sure that you can answer it yourself. :) If the memory is still also mapped to the original process (as with shared memory) that everything is fine. If the memory is removed from the original process, than the app will get an access-violation fault. (if this system works on a page boundary, to take this action it needs to make sure that there are no other allocation on the same page) Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 2:05 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Hi, I'd like to review this proposal. Let's consider the following scenario: - In the MPI_Isend, MPI maps the send buffer into the address space of the receiving process. - In the matching MPI_Recv, the receiving process makes a copy of the mapped send buffer into the receive buffer. - Once the copy is complete, the send buffer is mapped back into the sender address space during the wait/test call. What will happen one tries to access the send buffer in between? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 10:48 PM To: MPI 2.2 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) This proposal has passed 1st voting and needs reviewers. We need 3 volunteers to sign-off on this proposal, plus the 3 chapter authors to sign-off on the text. The Chapter Authors for Chapter 3: Point-to-Point Communication Chapter 5: Collective Communication Chapter 11: One-Sided Communication Please add a comment to the ticket saying that you reviewed the proposal, or please send me your comments. Send Buffer Access: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/45 Thanks, .Erez --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Fri Dec 5 17:56:24 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Fri, 5 Dec 2008 23:56:24 +0000 Subject: [Mpi-22] MPI 2.2 Reminders In-Reply-To: <4263FEE6-5988-43B4-BC42-3ACC97A1A636@cisco.com> Message-ID: <928CFBE8E7CB0040959E56B4EA41A77E08FD0FEF@irsmsx504.ger.corp.intel.com> Thanks. Can the report queries be changed to show the ticket state and priority as well? This would provide visual cue to those looking for tickets waiting for review. -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Jeff Squyres Sent: Friday, December 05, 2008 11:50 PM To: MPI 2.2 Subject: Re: [Mpi-22] MPI 2.2 Reminders I sent this diagram of ticket states around shortly after the last meeting, but I didn't get any comments back. So perhaps it would be better to send it to a wider audience. According to this diagram, any ticket in the "Not ready" / "Forum feedback requested" / "Waiting for reviews" state could be waiting for reviews. Perhaps we need something better than this (i.e., a separate bit indicating that a ticket is awaiting reviews). On Dec 5, 2008, at 11:13 AM, Supalov, Alexander wrote: > Hi, > > It would be great to learn how to identify tickets that are up for > review in the first place. All colors and fields I can see in the > standard reports don't seem to help much in this. > > Best regards. > > Alexander > > -----Original Message----- > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden] > ] On Behalf Of William Gropp > Sent: Friday, December 05, 2008 5:07 PM > To: Bronis R. de Supinski; MPI 2.2 > Subject: Re: [Mpi-22] MPI 2.2 Reminders > > As an immediate step, we can do the following: > > Authors of tickets that are ready for review should do the following: > (a) add a comment to the ticket with the names of the reviewers (the > relevant chapter author and 3 other volunteers) > (b) contact the reviewers to remind them > > Having a separate, searchable field would be nice, but this would > capture the data now. > > Bill > > On Dec 5, 2008, at 10:00 AM, Bronis R. de Supinski wrote: > >> >> Bill and Jeff: >> >> It will be really useful if the tickets listed reviewers. In the >> least >> we need some way to map tickets to reviewers (I am thinking about >> chapter >> authors in particular), Could Jeff add a field (or fields) that the >> owner is responsible for completing that indicates the reviewers? >> >> As it is I am really unclear over what I am supposed to review. I am >> guessing there are at least one or two for my chapter... >> >> Bronis >> >> >> On Fri, 5 Dec 2008, William Gropp wrote: >> >>> Our next MPI 2.2 discussion is in just over a week, and we have a >>> lot >>> to do. Please have a look at the wiki for the following (see https:// >>> svn.mpi-forum.org/trac/mpi-forum-web/report >>> for the reports): >>> >>> 1) For your own tickets, make sure that they are up to date. If >>> reviewers are needed, please contact them and encourage them to >>> submit >>> their reviews. If revised text is needed, please update the text >>> and >>> if you would like early feedback, notify this list. >>> >>> 2) For items on which you are a reviewer (particularly MPI 2.1 >>> chapter >>> authors), please review the relevant tickets. When you are >>> reviewing >>> text, make sure that you are looking at the current MPI 2.1 >>> standard; >>> check page and line numbers as well as the context to ensure that >>> the >>> change is correct. >>> >>> 3) For other tickets, review them and raise any issues that you have >>> *now* so that we can start the discussion before our meeting. >>> >>> Thanks! >>> >>> Bill -- Jeff Squyres Cisco Systems --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. From alexander.supalov at [hidden] Fri Dec 5 18:19:08 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Sat, 6 Dec 2008 00:19:08 +0000 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: <6B68D01C00C9994A8E150183E62A119E7B5C9F7B54@NA-EXMSG-C105.redmond.corp.microsoft.com> Message-ID: <928CFBE8E7CB0040959E56B4EA41A77E08FD0FF0@irsmsx504.ger.corp.intel.com> Dear Erez, Thank you. I reviewed the text and found that further polishing of its textual aspects should be suspended until the substance is clarified. I put a comment to this effect into the ticket, as well as into the dependent ticket #46. In my opinion, both tickets are not yet ready to go into the standard and should go into another round of contemplation of their possible repercussions. In both cases presumed application friendliness is traded for less freedom of implementation. Application developers who disregard the standard now will most likely continue to do this in the future, possibly in some other way. Restricting the freedom of implementation to make their life easier does not seem to be an attractive proposition to me. If any of the issues identified so far, or comparable issues we cannot fathom at the moment, surface up in one of the future HPC platforms and hinder MPI adoption or transition to MPI-2.2 there, we will have done disservice both to the MPI standard and to the community. I hope this will bear on the minds of those who're going to vote on these two items at the meeting. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Saturday, December 06, 2008 12:28 AM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Alexander, It is okay and encouraged for people to comment and argue on the proposals. You can add your comments to the ticket arguing your important points. The forum then consider that various points and vote on the proposal. However for the voting process we need people to review the text and confirm that it does not conflict with the standard and it is reasonable (from language pov) to be included in the standard. If we are willing to review the text and state that it valid for the standard, that would be great. If you have any comments on the text please send them to me. Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 2:26 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Erez, Thank you. I'm afraid I would need to have it explained in more detail why review may not include arguments on the substance. If something in the proposal makes one think that the proposed matter may be detrimental to the MPI standard and its implementations, I consider it one's duty to point this out. Following up on your reply: the segfault situation you described will make an MPI compliant program break. Thus, the implementation will have to keep the send buffer mapped into the sending process address space. This is a limitation on the MPI implementation that should be taken into account during the voting. Another possibility that has been pointed out earlier was that the proposed change disallows byte swap and other send buffer conversions to be done in place. At least one historical MPI implementation was doing this to a great avail. Who knows what is going to happen in the future? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 11:18 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) I think that the idea is for the reviewers to check the text for any mistakes and compatibility with the existing text, rather than check for the validity of the proposal. The later as I recall is left for the MPI forum assembly. As for your question, I'm sure that you can answer it yourself. :) If the memory is still also mapped to the original process (as with shared memory) that everything is fine. If the memory is removed from the original process, than the app will get an access-violation fault. (if this system works on a page boundary, to take this action it needs to make sure that there are no other allocation on the same page) Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 2:05 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Hi, I'd like to review this proposal. Let's consider the following scenario: - In the MPI_Isend, MPI maps the send buffer into the address space of the receiving process. - In the matching MPI_Recv, the receiving process makes a copy of the mapped send buffer into the receive buffer. - Once the copy is complete, the send buffer is mapped back into the sender address space during the wait/test call. What will happen one tries to access the send buffer in between? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 10:48 PM To: MPI 2.2 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) This proposal has passed 1st voting and needs reviewers. We need 3 volunteers to sign-off on this proposal, plus the 3 chapter authors to sign-off on the text. The Chapter Authors for Chapter 3: Point-to-Point Communication Chapter 5: Collective Communication Chapter 11: One-Sided Communication Please add a comment to the ticket saying that you reviewed the proposal, or please send me your comments. Send Buffer Access: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/45 Thanks, .Erez --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Sat Dec 6 01:49:19 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Sat, 6 Dec 2008 07:49:19 +0000 Subject: [Mpi-22] please review - add const keyword to the C bindings In-Reply-To: <6B68D01C00C9994A8E150183E62A119E7B5C9F7A8C@NA-EXMSG-C105.redmond.corp.microsoft.com> Message-ID: <928CFBE8E7CB0040959E56B4EA41A77E08FD0FFF@irsmsx504.ger.corp.intel.com> Hi, Pursuant to the discussion of #45 (send buffer access) that this #46 depends upon, I would suggest to suspend the review until we clear situation around #45. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 11:04 PM To: MPI 2.2 Subject: [Mpi-22] please review - add const keyword to the C bindings This proposal has passed 1st voting and needs reviewers. We need 3 volunteers to sign-off on this proposal. The proposal is short so don't be intimidated. :) plus I need the chapter authors to sign-off on the changes to the API (that the text is correct). Chapter Authors (it's less than a handful lines for each chapter) Chapter 3: Point-to-Point Communication Chapter 4: Datatypes Chapter 5: Collective Communication Chapter 6: Groups, Contexts, Communicators, and Caching Chapter 7: Process Topologies Chapter 8: MPI Environmental Management Chapter 9: The Info Object Chapter 10: Process Creation and Management Chapter 11: One-Sided Communication Chapter 13: I/O Chapter 15: Deprecated Functions Chapter 16: Language Bindings Please add a comment to the ticket saying that you reviewed the proposal, or please send me your comments. Add const Keyword to the C bindings: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/46 Thanks, .Erez --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsquyres at [hidden] Sat Dec 6 11:53:47 2008 From: jsquyres at [hidden] (Jeff Squyres) Date: Sat, 6 Dec 2008 12:53:47 -0500 Subject: [Mpi-22] MPI 2.2 Reminders In-Reply-To: <5C46E80D-F037-4FD4-A69B-86A40452E340@illinois.edu> Message-ID: I have updated all of my tickets; I tried to pick reviewers who care about the particular topic, either because they commented on it, argued about it in Chicago, or otherwise are familiar with the topic. If I couldn't find names that fit any of those criteria, I tried to "spread the wealth" and pick random Forum attendee names. Sorry (but some of my tickets are trivial text changes; it shouldn't take you long to review). :-) I also updated most of my tickets to the "waiting for reviews" status (some already had their first reading, because the "need 4 reviews" rule came in after they had their 1st reading already), so you can easily see what tickets are waiting. Thanks. On Dec 5, 2008, at 10:00 AM, William Gropp wrote: > Our next MPI 2.2 discussion is in just over a week, and we have a > lot to do. Please have a look at the wiki for the following (see https://svn.mpi-forum.org/trac/mpi-forum-web/report > for the reports): > > 1) For your own tickets, make sure that they are up to date. If > reviewers are needed, please contact them and encourage them to > submit their reviews. If revised text is needed, please update the > text and if you would like early feedback, notify this list. > > 2) For items on which you are a reviewer (particularly MPI 2.1 > chapter authors), please review the relevant tickets. When you are > reviewing text, make sure that you are looking at the current MPI > 2.1 standard; check page and line numbers as well as the context to > ensure that the change is correct. > > 3) For other tickets, review them and raise any issues that you have > *now* so that we can start the discussion before our meeting. > > Thanks! > > Bill > > William Gropp > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Paul and Cynthia Saylor Professor of Computer Science > University of Illinois Urbana-Champaign > > > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 -- Jeff Squyres Cisco Systems From alexander.supalov at [hidden] Sat Dec 6 12:09:30 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Sat, 6 Dec 2008 18:09:30 +0000 Subject: [Mpi-22] MPI 2.2 Reminders In-Reply-To: Message-ID: <928CFBE8E7CB0040959E56B4EA41A77E08FD1026@irsmsx504.ger.corp.intel.com> Dear Jeff, Thanks. How can one see that status? All statuses I can see are in the Active tickets and other query reports are "new" and "assigned". What do I miss here? Should the queries be updated globally for everybody to see your changes, or is this a matter of each individual? Next, how can I list all tickets that possibly await my review? I don't want to miss something just because I overlooked a request in my inbox. Filtering by the loosely formatted CC field may be tricky. Anyway, where is a description of the query mechanism for this ticket system? Third, I hope the Forum is OK with people volunteering to review certain tickets even if they are not explicitly invited by the authors. The way to express this would probably be to add one's address to the CC list, as suggested earlier by you. Finally, it may be helpful to bring all these "by-laws" in some system and write them down, so that everybody use the same conventions for similar actions. Best regards. Alexander -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Jeff Squyres Sent: Saturday, December 06, 2008 6:54 PM To: MPI 2.2 Subject: Re: [Mpi-22] MPI 2.2 Reminders I have updated all of my tickets; I tried to pick reviewers who care about the particular topic, either because they commented on it, argued about it in Chicago, or otherwise are familiar with the topic. If I couldn't find names that fit any of those criteria, I tried to "spread the wealth" and pick random Forum attendee names. Sorry (but some of my tickets are trivial text changes; it shouldn't take you long to review). :-) I also updated most of my tickets to the "waiting for reviews" status (some already had their first reading, because the "need 4 reviews" rule came in after they had their 1st reading already), so you can easily see what tickets are waiting. Thanks. On Dec 5, 2008, at 10:00 AM, William Gropp wrote: > Our next MPI 2.2 discussion is in just over a week, and we have a > lot to do. Please have a look at the wiki for the following (see https://svn.mpi-forum.org/trac/mpi-forum-web/report > for the reports): > > 1) For your own tickets, make sure that they are up to date. If > reviewers are needed, please contact them and encourage them to > submit their reviews. If revised text is needed, please update the > text and if you would like early feedback, notify this list. > > 2) For items on which you are a reviewer (particularly MPI 2.1 > chapter authors), please review the relevant tickets. When you are > reviewing text, make sure that you are looking at the current MPI > 2.1 standard; check page and line numbers as well as the context to > ensure that the change is correct. > > 3) For other tickets, review them and raise any issues that you have > *now* so that we can start the discussion before our meeting. > > Thanks! > > Bill > > William Gropp > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Paul and Cynthia Saylor Professor of Computer Science > University of Illinois Urbana-Champaign > > > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 -- Jeff Squyres Cisco Systems _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. From bronis at [hidden] Sat Dec 6 12:25:23 2008 From: bronis at [hidden] (Bronis R. de Supinski) Date: Sat, 6 Dec 2008 10:25:23 -0800 (PST) Subject: [Mpi-22] please review - add const keyword to the C bindings In-Reply-To: <928CFBE8E7CB0040959E56B4EA41A77E08FD0FFF@irsmsx504.ger.corp.intel.com> Message-ID: Alexander: Re: > Pursuant to the discussion of #45 (send buffer access) that this #46 > depends upon, I would suggest to suspend the review until we clear > situation around #45. While it is within your rights to suggest that we "suspend the review" of these items, I object to your timing. The issues have already had a first vote. You can raise concerns about them but I see no reason that we should cease to make progress on the wording when the vast majority (I don't recall if the vote was unanimous but it was not close) have agreed to them. Perhaps more importantly, your objections have been discussed from the first that Erez brought up the issues. Frankly, I have not seen anything in your objections that changes the extensive discussion on these issues. While it may be that you have changed your position (and Intel's?) on them, I strongly doubt that others will be convinced. In my opinion, Erez should definitely proceed with the reviews. If you want to object to the solutions, you will have an opportunity when the issues come up for their second vote the week after next. Bronis > > Best regards. > > Alexander > > ________________________________ > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Friday, December 05, 2008 11:04 PM > To: MPI 2.2 > Subject: [Mpi-22] please review - add const keyword to the C bindings > > This proposal has passed 1st voting and needs reviewers. We need 3 volunteers to sign-off on this proposal. The proposal is short so don't be intimidated. :) plus I need the chapter authors to sign-off on the changes to the API (that the text is correct). > > Chapter Authors (it's less than a handful lines for each chapter) > > Chapter 3: Point-to-Point Communication > Chapter 4: Datatypes > Chapter 5: Collective Communication > Chapter 6: Groups, Contexts, Communicators, and Caching > Chapter 7: Process Topologies > Chapter 8: MPI Environmental Management > Chapter 9: The Info Object > Chapter 10: Process Creation and Management > Chapter 11: One-Sided Communication > Chapter 13: I/O > Chapter 15: Deprecated Functions > Chapter 16: Language Bindings > > > Please add a comment to the ticket saying that you reviewed the proposal, or please send me your comments. > > Add const Keyword to the C bindings: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/46 > > Thanks, > .Erez > > > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > From alexander.supalov at [hidden] Sat Dec 6 12:42:19 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Sat, 6 Dec 2008 18:42:19 +0000 Subject: [Mpi-22] please review - add const keyword to the C bindings In-Reply-To: Message-ID: <928CFBE8E7CB0040959E56B4EA41A77E08FD1028@irsmsx504.ger.corp.intel.com> Dear Bronis, Thank you. Intel position on these issues has not changed a jot. We've been consistently against them from day 1, and consistently voted them down, for reasons I mentioned elsewhere. The new memory remapping scenario that highlights how detrimental the proposed changes may be to the MPI implementations was added after the first vote because this occurred to me after substantial contemplation. This happens to all of us once in a while I guess. In my opinion, continuing the work on wording is OK as long as the substance is OK. In this case, I believe the substance is not OK. This is why I'm suggesting to suspend the work on wording until we all agree that the matter is OK in principle. If you disagree, this will be your decision. I'm not going to be at the next meeting and am using the medium available to me to promote our point of view. I hope this is OK with the Forum at large. Best regards. Alexander -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Bronis R. de Supinski Sent: Saturday, December 06, 2008 7:25 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - add const keyword to the C bindings Alexander: Re: > Pursuant to the discussion of #45 (send buffer access) that this #46 > depends upon, I would suggest to suspend the review until we clear > situation around #45. While it is within your rights to suggest that we "suspend the review" of these items, I object to your timing. The issues have already had a first vote. You can raise concerns about them but I see no reason that we should cease to make progress on the wording when the vast majority (I don't recall if the vote was unanimous but it was not close) have agreed to them. Perhaps more importantly, your objections have been discussed from the first that Erez brought up the issues. Frankly, I have not seen anything in your objections that changes the extensive discussion on these issues. While it may be that you have changed your position (and Intel's?) on them, I strongly doubt that others will be convinced. In my opinion, Erez should definitely proceed with the reviews. If you want to object to the solutions, you will have an opportunity when the issues come up for their second vote the week after next. Bronis > > Best regards. > > Alexander > > ________________________________ > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Friday, December 05, 2008 11:04 PM > To: MPI 2.2 > Subject: [Mpi-22] please review - add const keyword to the C bindings > > This proposal has passed 1st voting and needs reviewers. We need 3 volunteers to sign-off on this proposal. The proposal is short so don't be intimidated. :) plus I need the chapter authors to sign-off on the changes to the API (that the text is correct). > > Chapter Authors (it's less than a handful lines for each chapter) > > Chapter 3: Point-to-Point Communication > Chapter 4: Datatypes > Chapter 5: Collective Communication > Chapter 6: Groups, Contexts, Communicators, and Caching > Chapter 7: Process Topologies > Chapter 8: MPI Environmental Management > Chapter 9: The Info Object > Chapter 10: Process Creation and Management > Chapter 11: One-Sided Communication > Chapter 13: I/O > Chapter 15: Deprecated Functions > Chapter 16: Language Bindings > > > Please add a comment to the ticket saying that you reviewed the proposal, or please send me your comments. > > Add const Keyword to the C bindings: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/46 > > Thanks, > .Erez > > > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. From erezh at [hidden] Sat Dec 6 12:44:41 2008 From: erezh at [hidden] (Erez Haba) Date: Sat, 6 Dec 2008 10:44:41 -0800 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: <928CFBE8E7CB0040959E56B4EA41A77E08FD0FF0@irsmsx504.ger.corp.intel.com> Message-ID: <6B68D01C00C9994A8E150183E62A119E7B5C9F7C88@NA-EXMSG-C105.redmond.corp.microsoft.com> Thanks Alexander, I respectfully decline your proposal to suspend the review of these tickets. I don't see any specific reference wrt text in your comments; and you don't bring any new issue that has not been discussed before the 1st voting. Thus I don't see any reason to suspend their review. Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 4:19 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Erez, Thank you. I reviewed the text and found that further polishing of its textual aspects should be suspended until the substance is clarified. I put a comment to this effect into the ticket, as well as into the dependent ticket #46. In my opinion, both tickets are not yet ready to go into the standard and should go into another round of contemplation of their possible repercussions. In both cases presumed application friendliness is traded for less freedom of implementation. Application developers who disregard the standard now will most likely continue to do this in the future, possibly in some other way. Restricting the freedom of implementation to make their life easier does not seem to be an attractive proposition to me. If any of the issues identified so far, or comparable issues we cannot fathom at the moment, surface up in one of the future HPC platforms and hinder MPI adoption or transition to MPI-2.2 there, we will have done disservice both to the MPI standard and to the community. I hope this will bear on the minds of those who're going to vote on these two items at the meeting. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Saturday, December 06, 2008 12:28 AM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Alexander, It is okay and encouraged for people to comment and argue on the proposals. You can add your comments to the ticket arguing your important points. The forum then consider that various points and vote on the proposal. However for the voting process we need people to review the text and confirm that it does not conflict with the standard and it is reasonable (from language pov) to be included in the standard. If we are willing to review the text and state that it valid for the standard, that would be great. If you have any comments on the text please send them to me. Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 2:26 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Erez, Thank you. I'm afraid I would need to have it explained in more detail why review may not include arguments on the substance. If something in the proposal makes one think that the proposed matter may be detrimental to the MPI standard and its implementations, I consider it one's duty to point this out. Following up on your reply: the segfault situation you described will make an MPI compliant program break. Thus, the implementation will have to keep the send buffer mapped into the sending process address space. This is a limitation on the MPI implementation that should be taken into account during the voting. Another possibility that has been pointed out earlier was that the proposed change disallows byte swap and other send buffer conversions to be done in place. At least one historical MPI implementation was doing this to a great avail. Who knows what is going to happen in the future? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 11:18 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) I think that the idea is for the reviewers to check the text for any mistakes and compatibility with the existing text, rather than check for the validity of the proposal. The later as I recall is left for the MPI forum assembly. As for your question, I'm sure that you can answer it yourself. :) If the memory is still also mapped to the original process (as with shared memory) that everything is fine. If the memory is removed from the original process, than the app will get an access-violation fault. (if this system works on a page boundary, to take this action it needs to make sure that there are no other allocation on the same page) Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 2:05 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Hi, I'd like to review this proposal. Let's consider the following scenario: - In the MPI_Isend, MPI maps the send buffer into the address space of the receiving process. - In the matching MPI_Recv, the receiving process makes a copy of the mapped send buffer into the receive buffer. - Once the copy is complete, the send buffer is mapped back into the sender address space during the wait/test call. What will happen one tries to access the send buffer in between? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 10:48 PM To: MPI 2.2 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) This proposal has passed 1st voting and needs reviewers. We need 3 volunteers to sign-off on this proposal, plus the 3 chapter authors to sign-off on the text. The Chapter Authors for Chapter 3: Point-to-Point Communication Chapter 5: Collective Communication Chapter 11: One-Sided Communication Please add a comment to the ticket saying that you reviewed the proposal, or please send me your comments. Send Buffer Access: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/45 Thanks, .Erez --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From bronis at [hidden] Sat Dec 6 12:49:45 2008 From: bronis at [hidden] (Bronis R. de Supinski) Date: Sat, 6 Dec 2008 10:49:45 -0800 (PST) Subject: [Mpi-22] please review - add const keyword to the C bindings In-Reply-To: <928CFBE8E7CB0040959E56B4EA41A77E08FD1028@irsmsx504.ger.corp.intel.com> Message-ID: Alexander: Re: > Thank you. Intel position on these issues has not changed a jot. We've > been consistently against them from day 1, and consistently voted them > down, for reasons I mentioned elsewhere. OK, then I guess Intel voted against. You were very much in the minority. > The new memory remapping scenario that highlights how detrimental the > proposed changes may be to the MPI implementations was added after the > first vote because this occurred to me after substantial contemplation. > This happens to all of us once in a while I guess. My response is that your scenario is not significantly different from the previous performance concerns. It does not fundamentally change the discussion. > In my opinion, continuing the work on wording is OK as long as the > substance is OK. In this case, I believe the substance is not OK. This > is why I'm suggesting to suspend the work on wording until we all agree > that the matter is OK in principle. Frankly, that you have confirmed Intel has previously voted against the issue makes your position weaker. You seem to want veto power but that is not how the rules work. You have one vote as a member institution. You lost the first round. You can continue to raise your objections. Perhaps that will lead to a different outcome for the second vote. However, you cannot veto further progress prior to that vote. > If you disagree, this will be your decision. I'm not going to be at the > next meeting and am using the medium available to me to promote our > point of view. I hope this is OK with the Forum at large. I stated you have the right to state your opinion. You do not have the right to order review of the issues suspended. In fact, simply requesting it is not appropriate. Bronis > Best regards. > > Alexander > > -----Original Message----- > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Bronis R. de Supinski > Sent: Saturday, December 06, 2008 7:25 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - add const keyword to the C bindings > > > Alexander: > > Re: > > Pursuant to the discussion of #45 (send buffer access) that this #46 > > depends upon, I would suggest to suspend the review until we clear > > situation around #45. > > While it is within your rights to suggest that we "suspend > the review" of these items, I object to your timing. The > issues have already had a first vote. You can raise concerns > about them but I see no reason that we should cease to > make progress on the wording when the vast majority (I > don't recall if the vote was unanimous but it was not close) > have agreed to them. > > Perhaps more importantly, your objections have been discussed > from the first that Erez brought up the issues. Frankly, I > have not seen anything in your objections that changes the > extensive discussion on these issues. While it may be that > you have changed your position (and Intel's?) on them, I > strongly doubt that others will be convinced. > > In my opinion, Erez should definitely proceed with the > reviews. If you want to object to the solutions, you will > have an opportunity when the issues come up for their second > vote the week after next. > > Bronis > > > > > > > > Best regards. > > > > Alexander > > > > ________________________________ > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba > > Sent: Friday, December 05, 2008 11:04 PM > > To: MPI 2.2 > > Subject: [Mpi-22] please review - add const keyword to the C bindings > > > > This proposal has passed 1st voting and needs reviewers. We need 3 volunteers to sign-off on this proposal. The proposal is short so don't be intimidated. :) plus I need the chapter authors to sign-off on the changes to the API (that the text is correct). > > > > Chapter Authors (it's less than a handful lines for each chapter) > > > > Chapter 3: Point-to-Point Communication > > Chapter 4: Datatypes > > Chapter 5: Collective Communication > > Chapter 6: Groups, Contexts, Communicators, and Caching > > Chapter 7: Process Topologies > > Chapter 8: MPI Environmental Management > > Chapter 9: The Info Object > > Chapter 10: Process Creation and Management > > Chapter 11: One-Sided Communication > > Chapter 13: I/O > > Chapter 15: Deprecated Functions > > Chapter 16: Language Bindings > > > > > > Please add a comment to the ticket saying that you reviewed the proposal, or please send me your comments. > > > > Add const Keyword to the C bindings: https:// svn.mpi-forum.org/trac/mpi-forum-web/ticket/46 > > > > Thanks, > > .Erez > > > > > > --------------------------------------------------------------------- > > Intel GmbH > > Dornacher Strasse 1 > > 85622 Feldkirchen/Muenchen Germany > > Sitz der Gesellschaft: Feldkirchen bei Muenchen > > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > > VAT Registration No.: DE129385895 > > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > > > This e-mail and any attachments may contain confidential material for > > the sole use of the intended recipient(s). Any review or distribution > > by others is strictly prohibited. If you are not the intended > > recipient, please contact the sender and delete all copies. > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http:// lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > > > From alexander.supalov at [hidden] Sat Dec 6 13:35:16 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Sat, 6 Dec 2008 19:35:16 +0000 Subject: [Mpi-22] please review - add const keyword to the C bindings In-Reply-To: Message-ID: <928CFBE8E7CB0040959E56B4EA41A77E08FD102D@irsmsx504.ger.corp.intel.com> Dear Bronis, Thank you. This is the first time I hear that consistency weakens one's hand. Anyway, I'm not after veto power, nor I trying to "order" or "request" anything. When we agreed to be in on it, we agreed to the rules, and will consistently follow them. All I say is that from our point of view, the matter requires more discussion before it will be ready to go into the standard. Turning back to the substance from the pure textual review would be appropriate then. This is what I suggested (see below for exact wording). I would suggest that we turn from formally following the process to making sure what we put into the standard is good for MPI and the HPC community. Looking at the text alone won't help here I'm afraid. Brainstorming possible counter-examples may. If you disagree with this suggestion, so be it. I expressed my opinion. You expressed yours. I would suggest that now we let the Forum decide what to do about this matter. Best regards. Alexander -----Original Message----- From: Bronis R. de Supinski [mailto:bronis_at_[hidden]] Sent: Saturday, December 06, 2008 7:50 PM To: Supalov, Alexander Cc: MPI 2.2 Subject: RE: [Mpi-22] please review - add const keyword to the C bindings Alexander: Re: > Thank you. Intel position on these issues has not changed a jot. We've > been consistently against them from day 1, and consistently voted them > down, for reasons I mentioned elsewhere. OK, then I guess Intel voted against. You were very much in the minority. > The new memory remapping scenario that highlights how detrimental the > proposed changes may be to the MPI implementations was added after the > first vote because this occurred to me after substantial contemplation. > This happens to all of us once in a while I guess. My response is that your scenario is not significantly different from the previous performance concerns. It does not fundamentally change the discussion. > In my opinion, continuing the work on wording is OK as long as the > substance is OK. In this case, I believe the substance is not OK. This > is why I'm suggesting to suspend the work on wording until we all agree > that the matter is OK in principle. Frankly, that you have confirmed Intel has previously voted against the issue makes your position weaker. You seem to want veto power but that is not how the rules work. You have one vote as a member institution. You lost the first round. You can continue to raise your objections. Perhaps that will lead to a different outcome for the second vote. However, you cannot veto further progress prior to that vote. > If you disagree, this will be your decision. I'm not going to be at the > next meeting and am using the medium available to me to promote our > point of view. I hope this is OK with the Forum at large. I stated you have the right to state your opinion. You do not have the right to order review of the issues suspended. In fact, simply requesting it is not appropriate. Bronis > Best regards. > > Alexander > > -----Original Message----- > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Bronis R. de Supinski > Sent: Saturday, December 06, 2008 7:25 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - add const keyword to the C bindings > > > Alexander: > > Re: > > Pursuant to the discussion of #45 (send buffer access) that this #46 > > depends upon, I would suggest to suspend the review until we clear > > situation around #45. > > While it is within your rights to suggest that we "suspend > the review" of these items, I object to your timing. The > issues have already had a first vote. You can raise concerns > about them but I see no reason that we should cease to > make progress on the wording when the vast majority (I > don't recall if the vote was unanimous but it was not close) > have agreed to them. > > Perhaps more importantly, your objections have been discussed > from the first that Erez brought up the issues. Frankly, I > have not seen anything in your objections that changes the > extensive discussion on these issues. While it may be that > you have changed your position (and Intel's?) on them, I > strongly doubt that others will be convinced. > > In my opinion, Erez should definitely proceed with the > reviews. If you want to object to the solutions, you will > have an opportunity when the issues come up for their second > vote the week after next. > > Bronis > > > > > > > > Best regards. > > > > Alexander > > > > ________________________________ > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba > > Sent: Friday, December 05, 2008 11:04 PM > > To: MPI 2.2 > > Subject: [Mpi-22] please review - add const keyword to the C bindings > > > > This proposal has passed 1st voting and needs reviewers. We need 3 volunteers to sign-off on this proposal. The proposal is short so don't be intimidated. :) plus I need the chapter authors to sign-off on the changes to the API (that the text is correct). > > > > Chapter Authors (it's less than a handful lines for each chapter) > > > > Chapter 3: Point-to-Point Communication > > Chapter 4: Datatypes > > Chapter 5: Collective Communication > > Chapter 6: Groups, Contexts, Communicators, and Caching > > Chapter 7: Process Topologies > > Chapter 8: MPI Environmental Management > > Chapter 9: The Info Object > > Chapter 10: Process Creation and Management > > Chapter 11: One-Sided Communication > > Chapter 13: I/O > > Chapter 15: Deprecated Functions > > Chapter 16: Language Bindings > > > > > > Please add a comment to the ticket saying that you reviewed the proposal, or please send me your comments. > > > > Add const Keyword to the C bindings: https:// svn.mpi-forum.org/trac/mpi-forum-web/ticket/46 > > > > Thanks, > > .Erez > > > > > > --------------------------------------------------------------------- > > Intel GmbH > > Dornacher Strasse 1 > > 85622 Feldkirchen/Muenchen Germany > > Sitz der Gesellschaft: Feldkirchen bei Muenchen > > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > > VAT Registration No.: DE129385895 > > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > > > This e-mail and any attachments may contain confidential material for > > the sole use of the intended recipient(s). Any review or distribution > > by others is strictly prohibited. If you are not the intended > > recipient, please contact the sender and delete all copies. > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http:// lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > > > --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. From alexander.supalov at [hidden] Sat Dec 6 13:43:33 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Sat, 6 Dec 2008 19:43:33 +0000 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: <6B68D01C00C9994A8E150183E62A119E7B5C9F7C88@NA-EXMSG-C105.redmond.corp.microsoft.com> Message-ID: <928CFBE8E7CB0040959E56B4EA41A77E08FD102E@irsmsx504.ger.corp.intel.com> Dear Erez, Thank you. I agree to respectfully disagree on this with you, for the following two reasons: 1) The memory remapping scenario IO brought up a couple of days ago was not discussed before the first voting as far as I can recall. If you have proof to the contrary, I would most kindly ask you to present it. If this cannot be done, I would say that a new issue has been added to the discussion, and we may need to review its substance. 2) Next, I would like to see the definition of the ticket review process that states the reviewers are supposed to only check the proposed text for correspondence with the existing standard. My opinion is that reviewers are there to bring both textual and substantial concerns up when they see the need for this. So far I've been acting on this conviction. I'm looking forward to the further discussion on the floor. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Saturday, December 06, 2008 7:45 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Thanks Alexander, I respectfully decline your proposal to suspend the review of these tickets. I don't see any specific reference wrt text in your comments; and you don't bring any new issue that has not been discussed before the 1st voting. Thus I don't see any reason to suspend their review. Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 4:19 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Erez, Thank you. I reviewed the text and found that further polishing of its textual aspects should be suspended until the substance is clarified. I put a comment to this effect into the ticket, as well as into the dependent ticket #46. In my opinion, both tickets are not yet ready to go into the standard and should go into another round of contemplation of their possible repercussions. In both cases presumed application friendliness is traded for less freedom of implementation. Application developers who disregard the standard now will most likely continue to do this in the future, possibly in some other way. Restricting the freedom of implementation to make their life easier does not seem to be an attractive proposition to me. If any of the issues identified so far, or comparable issues we cannot fathom at the moment, surface up in one of the future HPC platforms and hinder MPI adoption or transition to MPI-2.2 there, we will have done disservice both to the MPI standard and to the community. I hope this will bear on the minds of those who're going to vote on these two items at the meeting. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Saturday, December 06, 2008 12:28 AM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Alexander, It is okay and encouraged for people to comment and argue on the proposals. You can add your comments to the ticket arguing your important points. The forum then consider that various points and vote on the proposal. However for the voting process we need people to review the text and confirm that it does not conflict with the standard and it is reasonable (from language pov) to be included in the standard. If we are willing to review the text and state that it valid for the standard, that would be great. If you have any comments on the text please send them to me. Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 2:26 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Erez, Thank you. I'm afraid I would need to have it explained in more detail why review may not include arguments on the substance. If something in the proposal makes one think that the proposed matter may be detrimental to the MPI standard and its implementations, I consider it one's duty to point this out. Following up on your reply: the segfault situation you described will make an MPI compliant program break. Thus, the implementation will have to keep the send buffer mapped into the sending process address space. This is a limitation on the MPI implementation that should be taken into account during the voting. Another possibility that has been pointed out earlier was that the proposed change disallows byte swap and other send buffer conversions to be done in place. At least one historical MPI implementation was doing this to a great avail. Who knows what is going to happen in the future? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 11:18 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) I think that the idea is for the reviewers to check the text for any mistakes and compatibility with the existing text, rather than check for the validity of the proposal. The later as I recall is left for the MPI forum assembly. As for your question, I'm sure that you can answer it yourself. :) If the memory is still also mapped to the original process (as with shared memory) that everything is fine. If the memory is removed from the original process, than the app will get an access-violation fault. (if this system works on a page boundary, to take this action it needs to make sure that there are no other allocation on the same page) Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 2:05 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Hi, I'd like to review this proposal. Let's consider the following scenario: - In the MPI_Isend, MPI maps the send buffer into the address space of the receiving process. - In the matching MPI_Recv, the receiving process makes a copy of the mapped send buffer into the receive buffer. - Once the copy is complete, the send buffer is mapped back into the sender address space during the wait/test call. What will happen one tries to access the send buffer in between? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 10:48 PM To: MPI 2.2 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) This proposal has passed 1st voting and needs reviewers. We need 3 volunteers to sign-off on this proposal, plus the 3 chapter authors to sign-off on the text. The Chapter Authors for Chapter 3: Point-to-Point Communication Chapter 5: Collective Communication Chapter 11: One-Sided Communication Please add a comment to the ticket saying that you reviewed the proposal, or please send me your comments. Send Buffer Access: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/45 Thanks, .Erez --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From erezh at [hidden] Mon Dec 8 15:11:45 2008 From: erezh at [hidden] (Erez Haba) Date: Mon, 8 Dec 2008 13:11:45 -0800 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: <928CFBE8E7CB0040959E56B4EA41A77E08FD102E@irsmsx504.ger.corp.intel.com> Message-ID: <6B68D01C00C9994A8E150183E62A119E7B5C9F80CC@NA-EXMSG-C105.redmond.corp.microsoft.com> Dear Alexander, As far as I recall memory remapping from the main processor to a network device was discussed before (If I recall correctly, in the April meeting). I think that it's close enough to your scenario of remapping to a different process for the purpose of this discussion. Is your case real? Do you know of systems that do that with MPI? Or is it a hypothetical case? For the review process, we do need people to review the text; we added this requirement in the last meeting. Regardless, it does not prevent any other person from giving feedback on the proposal. I'm sure that Jeff or Bill can give you're a more formal definition of the review process. The wiki page state that: https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/mpi22/TicketWorkflow * To advance to the first reading, a proposal must be reviewed by the lead chapter author and three other reviewers. That review should check the change against the standard text to ensure that the change in context is correct; in addition, the change should be evaluated for completeness. For changes that involve multiple chapters (but are logically related and hence belong in a single ticket), the respective chapter authors must review the changes in their chapters. These reviews must be entered as comments on the ticket. MPI 2.2 Chapter Authors From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Saturday, December 06, 2008 11:44 AM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Erez, Thank you. I agree to respectfully disagree on this with you, for the following two reasons: 1) The memory remapping scenario IO brought up a couple of days ago was not discussed before the first voting as far as I can recall. If you have proof to the contrary, I would most kindly ask you to present it. If this cannot be done, I would say that a new issue has been added to the discussion, and we may need to review its substance. 2) Next, I would like to see the definition of the ticket review process that states the reviewers are supposed to only check the proposed text for correspondence with the existing standard. My opinion is that reviewers are there to bring both textual and substantial concerns up when they see the need for this. So far I've been acting on this conviction. I'm looking forward to the further discussion on the floor. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Saturday, December 06, 2008 7:45 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Thanks Alexander, I respectfully decline your proposal to suspend the review of these tickets. I don't see any specific reference wrt text in your comments; and you don't bring any new issue that has not been discussed before the 1st voting. Thus I don't see any reason to suspend their review. Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 4:19 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Erez, Thank you. I reviewed the text and found that further polishing of its textual aspects should be suspended until the substance is clarified. I put a comment to this effect into the ticket, as well as into the dependent ticket #46. In my opinion, both tickets are not yet ready to go into the standard and should go into another round of contemplation of their possible repercussions. In both cases presumed application friendliness is traded for less freedom of implementation. Application developers who disregard the standard now will most likely continue to do this in the future, possibly in some other way. Restricting the freedom of implementation to make their life easier does not seem to be an attractive proposition to me. If any of the issues identified so far, or comparable issues we cannot fathom at the moment, surface up in one of the future HPC platforms and hinder MPI adoption or transition to MPI-2.2 there, we will have done disservice both to the MPI standard and to the community. I hope this will bear on the minds of those who're going to vote on these two items at the meeting. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Saturday, December 06, 2008 12:28 AM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Alexander, It is okay and encouraged for people to comment and argue on the proposals. You can add your comments to the ticket arguing your important points. The forum then consider that various points and vote on the proposal. However for the voting process we need people to review the text and confirm that it does not conflict with the standard and it is reasonable (from language pov) to be included in the standard. If we are willing to review the text and state that it valid for the standard, that would be great. If you have any comments on the text please send them to me. Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 2:26 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Erez, Thank you. I'm afraid I would need to have it explained in more detail why review may not include arguments on the substance. If something in the proposal makes one think that the proposed matter may be detrimental to the MPI standard and its implementations, I consider it one's duty to point this out. Following up on your reply: the segfault situation you described will make an MPI compliant program break. Thus, the implementation will have to keep the send buffer mapped into the sending process address space. This is a limitation on the MPI implementation that should be taken into account during the voting. Another possibility that has been pointed out earlier was that the proposed change disallows byte swap and other send buffer conversions to be done in place. At least one historical MPI implementation was doing this to a great avail. Who knows what is going to happen in the future? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 11:18 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) I think that the idea is for the reviewers to check the text for any mistakes and compatibility with the existing text, rather than check for the validity of the proposal. The later as I recall is left for the MPI forum assembly. As for your question, I'm sure that you can answer it yourself. :) If the memory is still also mapped to the original process (as with shared memory) that everything is fine. If the memory is removed from the original process, than the app will get an access-violation fault. (if this system works on a page boundary, to take this action it needs to make sure that there are no other allocation on the same page) Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 2:05 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Hi, I'd like to review this proposal. Let's consider the following scenario: - In the MPI_Isend, MPI maps the send buffer into the address space of the receiving process. - In the matching MPI_Recv, the receiving process makes a copy of the mapped send buffer into the receive buffer. - Once the copy is complete, the send buffer is mapped back into the sender address space during the wait/test call. What will happen one tries to access the send buffer in between? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 10:48 PM To: MPI 2.2 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) This proposal has passed 1st voting and needs reviewers. We need 3 volunteers to sign-off on this proposal, plus the 3 chapter authors to sign-off on the text. The Chapter Authors for Chapter 3: Point-to-Point Communication Chapter 5: Collective Communication Chapter 11: One-Sided Communication Please add a comment to the ticket saying that you reviewed the proposal, or please send me your comments. Send Buffer Access: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/45 Thanks, .Erez --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From treumann at [hidden] Mon Dec 8 15:55:43 2008 From: treumann at [hidden] (Richard Treumann) Date: Mon, 8 Dec 2008 16:55:43 -0500 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: <6B68D01C00C9994A8E150183E62A119E7B5C9F80CC@NA-EXMSG-C105.redmond.corp.microsoft.com> Message-ID: On a single OS under AIX, IBM MPI does map the portion of send side memory that holds the send buffer into the receivers address space. We do this for both contiguous and non-contiguous buffers. The mapping lasts just long enough for the receive side CPU to do a memory copy from send buffer to receive buffer. (see patent 7,392,256), This optimization does not have any effect on the addressability of the send buffer by the sending task. In our case, at least, this optimization does not argue against the proposal. Also, Robert and I had a chat about the byte swap trick and it seems it should be both semantically cleaner and require fewer CPU cycles to do it in the receive buffer. In the receive buffer there is no question that the application must wait for the communication to complete and the swap only needs to be done once (the message flows in with bytes in sender order and the MPI_Recv does one pass of byte swaps if required. In the send buffer trick, the swaps to receiver order must be done and then a second pass is needed to undo it) Dick Dick Treumann - MPI Team IBM Systems & Technology Group Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 12/08/2008 04:11:45 PM: > [image removed] > > Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Erez Haba > > to: > > MPI 2.2 > > 12/08/2008 04:13 PM > > Sent by: > > mpi-22-bounces_at_[hidden] > > Please respond to "MPI 2.2" > > Dear Alexander, > > As far as I recall memory remapping from the main processor to a > network device was discussed before (If I recall correctly, in the > April meeting). I think that it’s close enough to your scenario of > remapping to a different process for the purpose of this discussion. > > Is your case real? Do you know of systems that do that with MPI? Or > is it a hypothetical case? > > > For the review process, we do need people to review the text; we > added this requirement in the last meeting. Regardless, it does not > prevent any other person from giving feedback on the proposal. I’m > sure that Jeff or Bill can give you’re a more formal definition of > the review process. > > The wiki page state that: > https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/mpi22/TicketWorkflow > To advance to the first reading, a proposal must be reviewed by the > lead chapter author and three other reviewers. That review should > check the change against the standard text to ensure that the change > in context is correct; in addition, the change should be evaluated > for completeness. For changes that involve multiple chapters (but > are logically related and hence belong in a single ticket), the > respective chapter authors must review the changes in their > chapters. These reviews must be entered as comments on the ticket. > MPI 2.2 Chapter Authors > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Supalov, Alexander > Sent: Saturday, December 06, 2008 11:44 AM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Dear Erez, > > Thank you. I agree to respectfully disagree on this with you, for > the following two reasons: > > 1) The memory remapping scenario IO brought up a couple of days ago > was not discussed before the first voting as far as I can recall. If > you have proof to the contrary, I would most kindly ask you to > present it. If this cannot be done, I would say that a new issue has > been added to the discussion, and we may need to review its substance. > > 2) Next, I would like to see the definition of the ticket review > process that states the reviewers are supposed to only check the > proposed text for correspondence with the existing standard. My > opinion is that reviewers are there to bring both textual and > substantial concerns up when they see the need for this. So far I've > been acting on this conviction. > > I'm looking forward to the further discussion on the floor. > > Best regards. > > Alexander > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Saturday, December 06, 2008 7:45 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > Thanks Alexander, > > I respectfully decline your proposal to suspend the review of these tickets. > I don’t see any specific reference wrt text in your comments; and > you don’t bring any new issue that has not been discussed before the 1st > voting. Thus I don’t see any reason to suspend their review. > > Thanks, > .Erez > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Supalov, Alexander > Sent: Friday, December 05, 2008 4:19 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Dear Erez, > > Thank you. I reviewed the text and found that further polishing of > its textual aspects should be suspended until the substance is > clarified. I put a comment to this effect into the ticket, as well > as into the dependent ticket #46. In my opinion, both tickets are > not yet ready to go into the standard and should go into another > round of contemplation of their possible repercussions. > > In both cases presumed application friendliness is traded for less > freedom of implementation. Application developers who disregard the > standard now will most likely continue to do this in the future, > possibly in some other way. Restricting the freedom of > implementation to make their life easier does not seem to be an > attractive proposition to me. > > If any of the issues identified so far, or comparable issues we > cannot fathom at the moment, surface up in one of the future HPC > platforms and hinder MPI adoption or transition to MPI-2.2 there, we > will have done disservice both to the MPI standard and to the > community. I hope this will bear on the minds of those who're going > to vote on these two items at the meeting. > > Best regards. > > Alexander > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Saturday, December 06, 2008 12:28 AM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > Dear Alexander, > > It is okay and encouraged for people to comment and argue on the > proposals. You can add your comments to the ticket arguing your > important points. The forum then consider that various points and > vote on the proposal. > > However for the voting process we need people to review the text and > confirm that it does not conflict with the standard and it is > reasonable (from language pov) to be included in the standard. > > If we are willing to review the text and state that it valid for the > standard, that would be great. If you have any comments on the text > please send them to me.) > > Thanks, > .Erez > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Supalov, Alexander > Sent: Friday, December 05, 2008 2:26 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Dear Erez, > > Thank you. I'm afraid I would need to have it explained in more > detail why review may not include arguments on the substance. If > something in the proposal makes one think that the proposed matter > may be detrimental to the MPI standard and its implementations, I > consider it one's duty to point this out. > > Following up on your reply: the segfault situation you described > will make an MPI compliant program break. Thus, the implementation > will have to keep the send buffer mapped into the sending process > address space. This is a limitation on the MPI implementation that > should be taken into account during the voting. > > Another possibility that has been pointed out earlier was that the > proposed change disallows byte swap and other send buffer > conversions to be done in place. At least one historical MPI > implementation was doing this to a great avail. Who knows what is > going to happen in the future? > > Best regards. > > Alexander > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Friday, December 05, 2008 11:18 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > I think that the idea is for the reviewers to check the text for any > mistakes and compatibility with the existing text, rather than check > for the validity of the proposal. The later as I recall is left for > the MPI forum assembly. > > As for your question, I’m sure that you can answer it yourself. J If > the memory is still also mapped to the original process (as with > shared memory) that everything is fine. If the memory is removed > from the original process, than the app will get an access-violation fault. > (if this system works on a page boundary, to take this action it > needs to make sure that there are no other allocation on the same page) > > Thanks, > .Erez > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Supalov, Alexander > Sent: Friday, December 05, 2008 2:05 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Hi, > > I'd like to review this proposal. Let's consider the following scenario:. > > - In the MPI_Isend, MPI maps the send buffer into the address space > of the receiving process. > - In the matching MPI_Recv, the receiving process makes a copy of > the mapped send buffer into the receive buffer. > - Once the copy is complete, the send buffer is mapped back into the > sender address space during the wait/test call. > > What will happen one tries to access the send buffer in between? > > Best regards. > > Alexander > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Friday, December 05, 2008 10:48 PM > To: MPI 2.2 > Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) > This proposal has passed 1st voting and needs reviewers. We need 3 volunteers > to sign-off on this proposal, plus the 3 chapter authors to sign-off > on the text. > > The Chapter Authors for > > Chapter 3: Point-to-Point Communication > Chapter 5: Collective Communication > Chapter 11: One-Sided Communication > > Please add a comment to the ticket saying that you reviewed the > proposal, or please send me your comments., > > Send Buffer Access: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/45 > > Thanks, > .Erez > > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052. > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052. > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052. > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052. > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From wgropp at [hidden] Mon Dec 8 15:58:24 2008 From: wgropp at [hidden] (William Gropp) Date: Mon, 8 Dec 2008 15:58:24 -0600 Subject: [Mpi-22] Voting and review process reminders Message-ID: Here is a reminder of the process for voting and reviews: Votes are taken to approve a change or addition. Votes are taken twice at separate meetings; one reason for the two votes is to allow time for reflection and for additional issues to be raised. It is acceptable to vote yes on one of the two votes and no on the other. Reviews of the proposed text are used to ensure that the text is consistent with the rest of the document. Such reviews do not review the proposal on its own technical grounds (that should be done by the Forum members prior to each vote). Rather, they look for editing errors or inconsistencies with respect to the existing document. As an example of an editing error, when reviewing one proposal that included the LaTeX source (which is to be encourage), there was a use of \mpi/ where \MPI/ was intended. As an example of an inconsistency, in reviewing ticket 55, I noticed that the proposed change was inconsistent with text already in the document. While it is hoped that such inconsistencies would be detected by the Forum members prior to the votes, the review process provides an additional check. Bill William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign From erezh at [hidden] Mon Dec 8 16:26:22 2008 From: erezh at [hidden] (Erez Haba) Date: Mon, 8 Dec 2008 14:26:22 -0800 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: Message-ID: <6B68D01C00C9994A8E150183E62A119E7B5C9F8189@NA-EXMSG-C105.redmond.corp.microsoft.com> Thanks Dick, Yes, it makes sense that the send buffer would still be addressable by the sender process as the buffer might share the pages with other memory allocations. (I assume that the mapping is done in page granularity). Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Monday, December 08, 2008 1:56 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) On a single OS under AIX, IBM MPI does map the portion of send side memory that holds the send buffer into the receivers address space. We do this for both contiguous and non-contiguous buffers. The mapping lasts just long enough for the receive side CPU to do a memory copy from send buffer to receive buffer. (see patent 7,392,256) This optimization does not have any effect on the addressability of the send buffer by the sending task. In our case, at least, this optimization does not argue against the proposal. Also, Robert and I had a chat about the byte swap trick and it seems it should be both semantically cleaner and require fewer CPU cycles to do it in the receive buffer. In the receive buffer there is no question that the application must wait for the communication to complete and the swap only needs to be done once (the message flows in with bytes in sender order and the MPI_Recv does one pass of byte swaps if required. In the send buffer trick, the swaps to receiver order must be done and then a second pass is needed to undo it) Dick Dick Treumann - MPI Team IBM Systems & Technology Group Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 12/08/2008 04:11:45 PM: > [image removed] > > Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Erez Haba > > to: > > MPI 2.2 > > 12/08/2008 04:13 PM > > Sent by: > > mpi-22-bounces_at_[hidden] > > Please respond to "MPI 2.2" > > Dear Alexander, > > As far as I recall memory remapping from the main processor to a > network device was discussed before (If I recall correctly, in the > April meeting). I think that it’s close enough to your scenario of > remapping to a different process for the purpose of this discussion. > > Is your case real? Do you know of systems that do that with MPI? Or > is it a hypothetical case? > > > For the review process, we do need people to review the text; we > added this requirement in the last meeting. Regardless, it does not > prevent any other person from giving feedback on the proposal. I’m > sure that Jeff or Bill can give you’re a more formal definition of > the review process. > > The wiki page state that: > https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/mpi22/TicketWorkflow > To advance to the first reading, a proposal must be reviewed by the > lead chapter author and three other reviewers. That review should > check the change against the standard text to ensure that the change > in context is correct; in addition, the change should be evaluated > for completeness. For changes that involve multiple chapters (but > are logically related and hence belong in a single ticket), the > respective chapter authors must review the changes in their > chapters. These reviews must be entered as comments on the ticket. > MPI 2.2 Chapter Authors > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Supalov, Alexander > Sent: Saturday, December 06, 2008 11:44 AM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Dear Erez, > > Thank you. I agree to respectfully disagree on this with you, for > the following two reasons: > > 1) The memory remapping scenario IO brought up a couple of days ago > was not discussed before the first voting as far as I can recall. If > you have proof to the contrary, I would most kindly ask you to > present it. If this cannot be done, I would say that a new issue has > been added to the discussion, and we may need to review its substance. > > 2) Next, I would like to see the definition of the ticket review > process that states the reviewers are supposed to only check the > proposed text for correspondence with the existing standard. My > opinion is that reviewers are there to bring both textual and > substantial concerns up when they see the need for this. So far I've > been acting on this conviction. > > I'm looking forward to the further discussion on the floor. > > Best regards. > > Alexander > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Saturday, December 06, 2008 7:45 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > Thanks Alexander, > > I respectfully decline your proposal to suspend the review of these tickets. > I don’t see any specific reference wrt text in your comments; and > you don’t bring any new issue that has not been discussed before the 1st > voting. Thus I don’t see any reason to suspend their review. > > Thanks, > .Erez > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Supalov, Alexander > Sent: Friday, December 05, 2008 4:19 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Dear Erez, > > Thank you. I reviewed the text and found that further polishing of > its textual aspects should be suspended until the substance is > clarified. I put a comment to this effect into the ticket, as well > as into the dependent ticket #46. In my opinion, both tickets are > not yet ready to go into the standard and should go into another > round of contemplation of their possible repercussions. > > In both cases presumed application friendliness is traded for less > freedom of implementation. Application developers who disregard the > standard now will most likely continue to do this in the future, > possibly in some other way. Restricting the freedom of > implementation to make their life easier does not seem to be an > attractive proposition to me. > > If any of the issues identified so far, or comparable issues we > cannot fathom at the moment, surface up in one of the future HPC > platforms and hinder MPI adoption or transition to MPI-2.2 there, we > will have done disservice both to the MPI standard and to the > community. I hope this will bear on the minds of those who're going > to vote on these two items at the meeting. > > Best regards. > > Alexander > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Saturday, December 06, 2008 12:28 AM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > Dear Alexander, > > It is okay and encouraged for people to comment and argue on the > proposals. You can add your comments to the ticket arguing your > important points. The forum then consider that various points and > vote on the proposal. > > However for the voting process we need people to review the text and > confirm that it does not conflict with the standard and it is > reasonable (from language pov) to be included in the standard. > > If we are willing to review the text and state that it valid for the > standard, that would be great. If you have any comments on the text > please send them to me. > > Thanks, > .Erez > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Supalov, Alexander > Sent: Friday, December 05, 2008 2:26 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Dear Erez, > > Thank you. I'm afraid I would need to have it explained in more > detail why review may not include arguments on the substance. If > something in the proposal makes one think that the proposed matter > may be detrimental to the MPI standard and its implementations, I > consider it one's duty to point this out. > > Following up on your reply: the segfault situation you described > will make an MPI compliant program break. Thus, the implementation > will have to keep the send buffer mapped into the sending process > address space. This is a limitation on the MPI implementation that > should be taken into account during the voting. > > Another possibility that has been pointed out earlier was that the > proposed change disallows byte swap and other send buffer > conversions to be done in place. At least one historical MPI > implementation was doing this to a great avail. Who knows what is > going to happen in the future? > > Best regards. > > Alexander > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Friday, December 05, 2008 11:18 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > I think that the idea is for the reviewers to check the text for any > mistakes and compatibility with the existing text, rather than check > for the validity of the proposal. The later as I recall is left for > the MPI forum assembly. > > As for your question, I’m sure that you can answer it yourself. J If > the memory is still also mapped to the original process (as with > shared memory) that everything is fine. If the memory is removed > from the original process, than the app will get an access-violation fault. > (if this system works on a page boundary, to take this action it > needs to make sure that there are no other allocation on the same page) > > Thanks, > .Erez > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Supalov, Alexander > Sent: Friday, December 05, 2008 2:05 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Hi, > > I'd like to review this proposal. Let's consider the following scenario: > > - In the MPI_Isend, MPI maps the send buffer into the address space > of the receiving process. > - In the matching MPI_Recv, the receiving process makes a copy of > the mapped send buffer into the receive buffer. > - Once the copy is complete, the send buffer is mapped back into the > sender address space during the wait/test call. > > What will happen one tries to access the send buffer in between? > > Best regards. > > Alexander > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Friday, December 05, 2008 10:48 PM > To: MPI 2.2 > Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) > This proposal has passed 1st voting and needs reviewers. We need 3 volunteers > to sign-off on this proposal, plus the 3 chapter authors to sign-off > on the text. > > The Chapter Authors for > > Chapter 3: Point-to-Point Communication > Chapter 5: Collective Communication > Chapter 11: One-Sided Communication > > Please add a comment to the ticket saying that you reviewed the > proposal, or please send me your comments. > > Send Buffer Access: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/45 > > Thanks, > .Erez > > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Mon Dec 8 16:57:56 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Mon, 8 Dec 2008 22:57:56 +0000 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: Message-ID: <928CFBE8E7CB0040959E56B4EA41A77E0900425E@irsmsx504.ger.corp.intel.com> Dear Dick, Thank you. Keeping the send buffer mapped upon the sender address space is surely legal. I wonder what will be happening to the sender that has, say, thousands of the outstanding MPI_Isends, all of which mapped their contents to the receiver memory. This will consume kernel memory space and possibly hardware resources. Will this be good for scalability? On a microkernel with so little memory to play with? I'd rather map the send buffer out. As for the send or receive side byte swapping, or, for that matter, any kind of reversible transformation - the receiver will certainly do this once. Note however that proposal #46 talks about collectives as well. If there are several receivers in a Bcast type operation, they will all be doing the transformation instead of the sender doing this once there and back again. The total expense will be higher for the receiver side transformation if more than two receivers are involved. Another couple of scenarios to consider: 3) Imagine send buffers have to pinned in the memory. To avoid doing this too often, these registrations will normally be cached. If more than one send can be used for a buffer or, for that matter, overlapping portions of the same buffer, say by different threads, access to the lookup-and-pin will have to be made atomic. This will further complicate implementation and introduce a potentially costly mutual exclusion primitive into the critical path. 4) I wonder what a const modifier will do for a buffer identifies by MPI_BOTTOM and/or a derived data type, possibly with holes in it. How will this square up with the C language sequence association rules? 5) Note also if both #45 and #46 will be introduced, there will be no way to retract this, even with the help of the MPI_INIT_ASSERTED, should we later decide to introduce assertion like MPI_NO_SEND_BUFFER_READ_ACCESS. The const modifier from #46 will make that syntactically useless. 6) Finally, what will happen in the Fortran interface? With the copy-in/copy-out possibly happening on the MPI subroutine boundary for array sections? If more than one send is allowed, the application can pretty easily exhaust any virtual memory with a couple of long enough vectors. I'm looking forward to your and everybody's opinion on those scenarios. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Monday, December 08, 2008 10:56 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) On a single OS under AIX, IBM MPI does map the portion of send side memory that holds the send buffer into the receivers address space. We do this for both contiguous and non-contiguous buffers. The mapping lasts just long enough for the receive side CPU to do a memory copy from send buffer to receive buffer. (see patent 7,392,256) This optimization does not have any effect on the addressability of the send buffer by the sending task. In our case, at least, this optimization does not argue against the proposal. Also, Robert and I had a chat about the byte swap trick and it seems it should be both semantically cleaner and require fewer CPU cycles to do it in the receive buffer. In the receive buffer there is no question that the application must wait for the communication to complete and the swap only needs to be done once (the message flows in with bytes in sender order and the MPI_Recv does one pass of byte swaps if required. In the send buffer trick, the swaps to receiver order must be done and then a second pass is needed to undo it) Dick Dick Treumann - MPI Team IBM Systems & Technology Group Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 12/08/2008 04:11:45 PM: > [image removed] > > Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Erez Haba > > to: > > MPI 2.2 > > 12/08/2008 04:13 PM > > Sent by: > > mpi-22-bounces_at_[hidden] > > Please respond to "MPI 2.2" > > Dear Alexander, > > As far as I recall memory remapping from the main processor to a > network device was discussed before (If I recall correctly, in the > April meeting). I think that it's close enough to your scenario of > remapping to a different process for the purpose of this discussion. > > Is your case real? Do you know of systems that do that with MPI? Or > is it a hypothetical case? > > > For the review process, we do need people to review the text; we > added this requirement in the last meeting. Regardless, it does not > prevent any other person from giving feedback on the proposal. I'm > sure that Jeff or Bill can give you're a more formal definition of > the review process. > > The wiki page state that: > https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/mpi22/TicketWorkflow > To advance to the first reading, a proposal must be reviewed by the > lead chapter author and three other reviewers. That review should > check the change against the standard text to ensure that the change > in context is correct; in addition, the change should be evaluated > for completeness. For changes that involve multiple chapters (but > are logically related and hence belong in a single ticket), the > respective chapter authors must review the changes in their > chapters. These reviews must be entered as comments on the ticket. > MPI 2.2 Chapter Authors > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Supalov, Alexander > Sent: Saturday, December 06, 2008 11:44 AM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Dear Erez, > > Thank you. I agree to respectfully disagree on this with you, for > the following two reasons: > > 1) The memory remapping scenario IO brought up a couple of days ago > was not discussed before the first voting as far as I can recall. If > you have proof to the contrary, I would most kindly ask you to > present it. If this cannot be done, I would say that a new issue has > been added to the discussion, and we may need to review its substance. > > 2) Next, I would like to see the definition of the ticket review > process that states the reviewers are supposed to only check the > proposed text for correspondence with the existing standard. My > opinion is that reviewers are there to bring both textual and > substantial concerns up when they see the need for this. So far I've > been acting on this conviction. > > I'm looking forward to the further discussion on the floor. > > Best regards. > > Alexander > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Saturday, December 06, 2008 7:45 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > Thanks Alexander, > > I respectfully decline your proposal to suspend the review of these tickets. > I don't see any specific reference wrt text in your comments; and > you don't bring any new issue that has not been discussed before the 1st > voting. Thus I don't see any reason to suspend their review. > > Thanks, > .Erez > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Supalov, Alexander > Sent: Friday, December 05, 2008 4:19 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Dear Erez, > > Thank you. I reviewed the text and found that further polishing of > its textual aspects should be suspended until the substance is > clarified. I put a comment to this effect into the ticket, as well > as into the dependent ticket #46. In my opinion, both tickets are > not yet ready to go into the standard and should go into another > round of contemplation of their possible repercussions. > > In both cases presumed application friendliness is traded for less > freedom of implementation. Application developers who disregard the > standard now will most likely continue to do this in the future, > possibly in some other way. Restricting the freedom of > implementation to make their life easier does not seem to be an > attractive proposition to me. > > If any of the issues identified so far, or comparable issues we > cannot fathom at the moment, surface up in one of the future HPC > platforms and hinder MPI adoption or transition to MPI-2.2 there, we > will have done disservice both to the MPI standard and to the > community. I hope this will bear on the minds of those who're going > to vote on these two items at the meeting. > > Best regards. > > Alexander > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Saturday, December 06, 2008 12:28 AM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > Dear Alexander, > > It is okay and encouraged for people to comment and argue on the > proposals. You can add your comments to the ticket arguing your > important points. The forum then consider that various points and > vote on the proposal. > > However for the voting process we need people to review the text and > confirm that it does not conflict with the standard and it is > reasonable (from language pov) to be included in the standard. > > If we are willing to review the text and state that it valid for the > standard, that would be great. If you have any comments on the text > please send them to me. > > Thanks, > .Erez > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Supalov, Alexander > Sent: Friday, December 05, 2008 2:26 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Dear Erez, > > Thank you. I'm afraid I would need to have it explained in more > detail why review may not include arguments on the substance. If > something in the proposal makes one think that the proposed matter > may be detrimental to the MPI standard and its implementations, I > consider it one's duty to point this out. > > Following up on your reply: the segfault situation you described > will make an MPI compliant program break. Thus, the implementation > will have to keep the send buffer mapped into the sending process > address space. This is a limitation on the MPI implementation that > should be taken into account during the voting. > > Another possibility that has been pointed out earlier was that the > proposed change disallows byte swap and other send buffer > conversions to be done in place. At least one historical MPI > implementation was doing this to a great avail. Who knows what is > going to happen in the future? > > Best regards. > > Alexander > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Friday, December 05, 2008 11:18 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > I think that the idea is for the reviewers to check the text for any > mistakes and compatibility with the existing text, rather than check > for the validity of the proposal. The later as I recall is left for > the MPI forum assembly. > > As for your question, I'm sure that you can answer it yourself. J If > the memory is still also mapped to the original process (as with > shared memory) that everything is fine. If the memory is removed > from the original process, than the app will get an access-violation fault. > (if this system works on a page boundary, to take this action it > needs to make sure that there are no other allocation on the same page) > > Thanks, > .Erez > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Supalov, Alexander > Sent: Friday, December 05, 2008 2:05 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Hi, > > I'd like to review this proposal. Let's consider the following scenario: > > - In the MPI_Isend, MPI maps the send buffer into the address space > of the receiving process. > - In the matching MPI_Recv, the receiving process makes a copy of > the mapped send buffer into the receive buffer. > - Once the copy is complete, the send buffer is mapped back into the > sender address space during the wait/test call. > > What will happen one tries to access the send buffer in between? > > Best regards. > > Alexander > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Friday, December 05, 2008 10:48 PM > To: MPI 2.2 > Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) > This proposal has passed 1st voting and needs reviewers. We need 3 volunteers > to sign-off on this proposal, plus the 3 chapter authors to sign-off > on the text. > > The Chapter Authors for > > Chapter 3: Point-to-Point Communication > Chapter 5: Collective Communication > Chapter 11: One-Sided Communication > > Please add a comment to the ticket saying that you reviewed the > proposal, or please send me your comments. > > Send Buffer Access: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/45 > > Thanks, > .Erez > > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Mon Dec 8 17:29:17 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Mon, 8 Dec 2008 23:29:17 +0000 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: <6B68D01C00C9994A8E150183E62A119E7B5C9F80CC@NA-EXMSG-C105.redmond.corp.microsoft.com> Message-ID: <928CFBE8E7CB0040959E56B4EA41A77E09004266@irsmsx504.ger.corp.intel.com> Dear Erez, Thank you. I hope to have replied to your first question in my reply to Dick. It also contains a couple of other concerns I have identified so far in connection to ##45 and 46. The review process definition - and Bill's clarification - are very helpful. I apparently missed this aspect because I did not attend the last meeting. It would be great to have those procedural decision communicated clearly to the Forum in some way after the meeting for the benefit of the absentees. Basing on this definition, I agree that it will be correct to decouple the textual review from the substance discussion of ##45 and 46 that I started - without diminishing the import of either, of course. I'm looking forward to the member's reaction to those concerns that I raised. I become more and more convinced that the proposed changes are going to harm more than they help. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Monday, December 08, 2008 10:12 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Alexander, As far as I recall memory remapping from the main processor to a network device was discussed before (If I recall correctly, in the April meeting). I think that it's close enough to your scenario of remapping to a different process for the purpose of this discussion. Is your case real? Do you know of systems that do that with MPI? Or is it a hypothetical case? For the review process, we do need people to review the text; we added this requirement in the last meeting. Regardless, it does not prevent any other person from giving feedback on the proposal. I'm sure that Jeff or Bill can give you're a more formal definition of the review process. The wiki page state that: https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/mpi22/TicketWorkflow * To advance to the first reading, a proposal must be reviewed by the lead chapter author and three other reviewers. That review should check the change against the standard text to ensure that the change in context is correct; in addition, the change should be evaluated for completeness. For changes that involve multiple chapters (but are logically related and hence belong in a single ticket), the respective chapter authors must review the changes in their chapters. These reviews must be entered as comments on the ticket. MPI 2.2 Chapter Authors From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Saturday, December 06, 2008 11:44 AM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Erez, Thank you. I agree to respectfully disagree on this with you, for the following two reasons: 1) The memory remapping scenario IO brought up a couple of days ago was not discussed before the first voting as far as I can recall. If you have proof to the contrary, I would most kindly ask you to present it. If this cannot be done, I would say that a new issue has been added to the discussion, and we may need to review its substance. 2) Next, I would like to see the definition of the ticket review process that states the reviewers are supposed to only check the proposed text for correspondence with the existing standard. My opinion is that reviewers are there to bring both textual and substantial concerns up when they see the need for this. So far I've been acting on this conviction. I'm looking forward to the further discussion on the floor. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Saturday, December 06, 2008 7:45 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Thanks Alexander, I respectfully decline your proposal to suspend the review of these tickets. I don't see any specific reference wrt text in your comments; and you don't bring any new issue that has not been discussed before the 1st voting. Thus I don't see any reason to suspend their review. Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 4:19 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Erez, Thank you. I reviewed the text and found that further polishing of its textual aspects should be suspended until the substance is clarified. I put a comment to this effect into the ticket, as well as into the dependent ticket #46. In my opinion, both tickets are not yet ready to go into the standard and should go into another round of contemplation of their possible repercussions. In both cases presumed application friendliness is traded for less freedom of implementation. Application developers who disregard the standard now will most likely continue to do this in the future, possibly in some other way. Restricting the freedom of implementation to make their life easier does not seem to be an attractive proposition to me. If any of the issues identified so far, or comparable issues we cannot fathom at the moment, surface up in one of the future HPC platforms and hinder MPI adoption or transition to MPI-2.2 there, we will have done disservice both to the MPI standard and to the community. I hope this will bear on the minds of those who're going to vote on these two items at the meeting. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Saturday, December 06, 2008 12:28 AM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Alexander, It is okay and encouraged for people to comment and argue on the proposals. You can add your comments to the ticket arguing your important points. The forum then consider that various points and vote on the proposal. However for the voting process we need people to review the text and confirm that it does not conflict with the standard and it is reasonable (from language pov) to be included in the standard. If we are willing to review the text and state that it valid for the standard, that would be great. If you have any comments on the text please send them to me. Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 2:26 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Erez, Thank you. I'm afraid I would need to have it explained in more detail why review may not include arguments on the substance. If something in the proposal makes one think that the proposed matter may be detrimental to the MPI standard and its implementations, I consider it one's duty to point this out. Following up on your reply: the segfault situation you described will make an MPI compliant program break. Thus, the implementation will have to keep the send buffer mapped into the sending process address space. This is a limitation on the MPI implementation that should be taken into account during the voting. Another possibility that has been pointed out earlier was that the proposed change disallows byte swap and other send buffer conversions to be done in place. At least one historical MPI implementation was doing this to a great avail. Who knows what is going to happen in the future? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 11:18 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) I think that the idea is for the reviewers to check the text for any mistakes and compatibility with the existing text, rather than check for the validity of the proposal. The later as I recall is left for the MPI forum assembly. As for your question, I'm sure that you can answer it yourself. :) If the memory is still also mapped to the original process (as with shared memory) that everything is fine. If the memory is removed from the original process, than the app will get an access-violation fault. (if this system works on a page boundary, to take this action it needs to make sure that there are no other allocation on the same page) Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Friday, December 05, 2008 2:05 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Hi, I'd like to review this proposal. Let's consider the following scenario: - In the MPI_Isend, MPI maps the send buffer into the address space of the receiving process. - In the matching MPI_Recv, the receiving process makes a copy of the mapped send buffer into the receive buffer. - Once the copy is complete, the send buffer is mapped back into the sender address space during the wait/test call. What will happen one tries to access the send buffer in between? Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Erez Haba Sent: Friday, December 05, 2008 10:48 PM To: MPI 2.2 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) This proposal has passed 1st voting and needs reviewers. We need 3 volunteers to sign-off on this proposal, plus the 3 chapter authors to sign-off on the text. The Chapter Authors for Chapter 3: Point-to-Point Communication Chapter 5: Collective Communication Chapter 11: One-Sided Communication Please add a comment to the ticket saying that you reviewed the proposal, or please send me your comments. Send Buffer Access: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/45 Thanks, .Erez --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.supalov at [hidden] Tue Dec 9 14:09:44 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Tue, 9 Dec 2008 20:09:44 +0000 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: <928CFBE8E7CB0040959E56B4EA41A77E0900425E@irsmsx504.ger.corp.intel.com> Message-ID: <928CFBE8E7CB0040959E56B4EA41A77E090047E6@irsmsx504.ger.corp.intel.com> HI everybody, Another possible complication: 7) In-place compression and/or encryption of the messages. Compression in particular can work wonders on monotonous messages, and cost less time in total than the transmission of so many giga-zeroes, for example. Again, having send buffer access allowed and const modifier attached will kill this huge optimization opportunity. Too bad. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Monday, December 08, 2008 11:58 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Dear Dick, Thank you. Keeping the send buffer mapped upon the sender address space is surely legal. I wonder what will be happening to the sender that has, say, thousands of the outstanding MPI_Isends, all of which mapped their contents to the receiver memory. This will consume kernel memory space and possibly hardware resources. Will this be good for scalability? On a microkernel with so little memory to play with? I'd rather map the send buffer out. As for the send or receive side byte swapping, or, for that matter, any kind of reversible transformation - the receiver will certainly do this once. Note however that proposal #46 talks about collectives as well. If there are several receivers in a Bcast type operation, they will all be doing the transformation instead of the sender doing this once there and back again. The total expense will be higher for the receiver side transformation if more than two receivers are involved. Another couple of scenarios to consider: 3) Imagine send buffers have to pinned in the memory. To avoid doing this too often, these registrations will normally be cached. If more than one send can be used for a buffer or, for that matter, overlapping portions of the same buffer, say by different threads, access to the lookup-and-pin will have to be made atomic. This will further complicate implementation and introduce a potentially costly mutual exclusion primitive into the critical path. 4) I wonder what a const modifier will do for a buffer identifies by MPI_BOTTOM and/or a derived data type, possibly with holes in it. How will this square up with the C language sequence association rules? 5) Note also if both #45 and #46 will be introduced, there will be no way to retract this, even with the help of the MPI_INIT_ASSERTED, should we later decide to introduce assertion like MPI_NO_SEND_BUFFER_READ_ACCESS. The const modifier from #46 will make that syntactically useless. 6) Finally, what will happen in the Fortran interface? With the copy-in/copy-out possibly happening on the MPI subroutine boundary for array sections? If more than one send is allowed, the application can pretty easily exhaust any virtual memory with a couple of long enough vectors. I'm looking forward to your and everybody's opinion on those scenarios. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Monday, December 08, 2008 10:56 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) On a single OS under AIX, IBM MPI does map the portion of send side memory that holds the send buffer into the receivers address space. We do this for both contiguous and non-contiguous buffers. The mapping lasts just long enough for the receive side CPU to do a memory copy from send buffer to receive buffer. (see patent 7,392,256) This optimization does not have any effect on the addressability of the send buffer by the sending task. In our case, at least, this optimization does not argue against the proposal. Also, Robert and I had a chat about the byte swap trick and it seems it should be both semantically cleaner and require fewer CPU cycles to do it in the receive buffer. In the receive buffer there is no question that the application must wait for the communication to complete and the swap only needs to be done once (the message flows in with bytes in sender order and the MPI_Recv does one pass of byte swaps if required. In the send buffer trick, the swaps to receiver order must be done and then a second pass is needed to undo it) Dick Dick Treumann - MPI Team IBM Systems & Technology Group Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 12/08/2008 04:11:45 PM: > [image removed] > > Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Erez Haba > > to: > > MPI 2.2 > > 12/08/2008 04:13 PM > > Sent by: > > mpi-22-bounces_at_[hidden] > > Please respond to "MPI 2.2" > > Dear Alexander, > > As far as I recall memory remapping from the main processor to a > network device was discussed before (If I recall correctly, in the > April meeting). I think that it's close enough to your scenario of > remapping to a different process for the purpose of this discussion. > > Is your case real? Do you know of systems that do that with MPI? Or > is it a hypothetical case? > > > For the review process, we do need people to review the text; we > added this requirement in the last meeting. Regardless, it does not > prevent any other person from giving feedback on the proposal. I'm > sure that Jeff or Bill can give you're a more formal definition of > the review process. > > The wiki page state that: > https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/mpi22/TicketWorkflow > To advance to the first reading, a proposal must be reviewed by the > lead chapter author and three other reviewers. That review should > check the change against the standard text to ensure that the change > in context is correct; in addition, the change should be evaluated > for completeness. For changes that involve multiple chapters (but > are logically related and hence belong in a single ticket), the > respective chapter authors must review the changes in their > chapters. These reviews must be entered as comments on the ticket. > MPI 2.2 Chapter Authors > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Supalov, Alexander > Sent: Saturday, December 06, 2008 11:44 AM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Dear Erez, > > Thank you. I agree to respectfully disagree on this with you, for > the following two reasons: > > 1) The memory remapping scenario IO brought up a couple of days ago > was not discussed before the first voting as far as I can recall. If > you have proof to the contrary, I would most kindly ask you to > present it. If this cannot be done, I would say that a new issue has > been added to the discussion, and we may need to review its substance. > > 2) Next, I would like to see the definition of the ticket review > process that states the reviewers are supposed to only check the > proposed text for correspondence with the existing standard. My > opinion is that reviewers are there to bring both textual and > substantial concerns up when they see the need for this. So far I've > been acting on this conviction. > > I'm looking forward to the further discussion on the floor. > > Best regards. > > Alexander > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Saturday, December 06, 2008 7:45 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > Thanks Alexander, > > I respectfully decline your proposal to suspend the review of these tickets. > I don't see any specific reference wrt text in your comments; and > you don't bring any new issue that has not been discussed before the 1st > voting. Thus I don't see any reason to suspend their review. > > Thanks, > .Erez > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Supalov, Alexander > Sent: Friday, December 05, 2008 4:19 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Dear Erez, > > Thank you. I reviewed the text and found that further polishing of > its textual aspects should be suspended until the substance is > clarified. I put a comment to this effect into the ticket, as well > as into the dependent ticket #46. In my opinion, both tickets are > not yet ready to go into the standard and should go into another > round of contemplation of their possible repercussions. > > In both cases presumed application friendliness is traded for less > freedom of implementation. Application developers who disregard the > standard now will most likely continue to do this in the future, > possibly in some other way. Restricting the freedom of > implementation to make their life easier does not seem to be an > attractive proposition to me. > > If any of the issues identified so far, or comparable issues we > cannot fathom at the moment, surface up in one of the future HPC > platforms and hinder MPI adoption or transition to MPI-2.2 there, we > will have done disservice both to the MPI standard and to the > community. I hope this will bear on the minds of those who're going > to vote on these two items at the meeting. > > Best regards. > > Alexander > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Saturday, December 06, 2008 12:28 AM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > Dear Alexander, > > It is okay and encouraged for people to comment and argue on the > proposals. You can add your comments to the ticket arguing your > important points. The forum then consider that various points and > vote on the proposal. > > However for the voting process we need people to review the text and > confirm that it does not conflict with the standard and it is > reasonable (from language pov) to be included in the standard. > > If we are willing to review the text and state that it valid for the > standard, that would be great. If you have any comments on the text > please send them to me. > > Thanks, > .Erez > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Supalov, Alexander > Sent: Friday, December 05, 2008 2:26 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Dear Erez, > > Thank you. I'm afraid I would need to have it explained in more > detail why review may not include arguments on the substance. If > something in the proposal makes one think that the proposed matter > may be detrimental to the MPI standard and its implementations, I > consider it one's duty to point this out. > > Following up on your reply: the segfault situation you described > will make an MPI compliant program break. Thus, the implementation > will have to keep the send buffer mapped into the sending process > address space. This is a limitation on the MPI implementation that > should be taken into account during the voting. > > Another possibility that has been pointed out earlier was that the > proposed change disallows byte swap and other send buffer > conversions to be done in place. At least one historical MPI > implementation was doing this to a great avail. Who knows what is > going to happen in the future? > > Best regards. > > Alexander > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Friday, December 05, 2008 11:18 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > I think that the idea is for the reviewers to check the text for any > mistakes and compatibility with the existing text, rather than check > for the validity of the proposal. The later as I recall is left for > the MPI forum assembly. > > As for your question, I'm sure that you can answer it yourself. J If > the memory is still also mapped to the original process (as with > shared memory) that everything is fine. If the memory is removed > from the original process, than the app will get an access-violation fault. > (if this system works on a page boundary, to take this action it > needs to make sure that there are no other allocation on the same page) > > Thanks, > .Erez > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Supalov, Alexander > Sent: Friday, December 05, 2008 2:05 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Hi, > > I'd like to review this proposal. Let's consider the following scenario: > > - In the MPI_Isend, MPI maps the send buffer into the address space > of the receiving process. > - In the matching MPI_Recv, the receiving process makes a copy of > the mapped send buffer into the receive buffer. > - Once the copy is complete, the send buffer is mapped back into the > sender address space during the wait/test call. > > What will happen one tries to access the send buffer in between? > > Best regards. > > Alexander > > > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22- > bounces_at_[hidden]] On Behalf Of Erez Haba > Sent: Friday, December 05, 2008 10:48 PM > To: MPI 2.2 > Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) > This proposal has passed 1st voting and needs reviewers. We need 3 volunteers > to sign-off on this proposal, plus the 3 chapter authors to sign-off > on the text. > > The Chapter Authors for > > Chapter 3: Point-to-Point Communication > Chapter 5: Collective Communication > Chapter 11: One-Sided Communication > > Please add a comment to the ticket saying that you reviewed the > proposal, or please send me your comments. > > Send Buffer Access: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/45 > > Thanks, > .Erez > > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From ritzdorf at [hidden] Tue Dec 9 14:58:28 2008 From: ritzdorf at [hidden] (Hubert Ritzdorf) Date: Tue, 09 Dec 2008 21:58:28 +0100 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: Message-ID: <493EDBF4.7080605@it.neclab.eu> Richard Treumann wrote: > > On a single OS under AIX, IBM MPI does map the portion of send side > memory that holds the send buffer into the receivers address space. We > do this for both contiguous and non-contiguous buffers. The mapping > lasts just long enough for the receive side CPU to do a memory copy > from send buffer to receive buffer. (see patent *7,392,256*) > > This optimization does not have any effect on the addressability of > the send buffer by the sending task. In our case, at least, this > optimization does not argue against the proposal. > This reference to the patent would be a clear indication for other implementers (such as me) to implement it using Alexander's approach (in order to avoid any problem with IBM's patent lawyers). And the change of the actual standard might limit implementers to implement Alexander's approach and might decrease the performance of their MPI implementation. (OK; 2 might's, not really a strong argument since it is not already implemented in this way; but the mapping approach seems to be quite reasonable). Or does IBM have also a patent for Alexander's approach ? Or does Intel have already a patent on this approach ? ;-) Hubert Hubert Ritzdorf NEC Europe * -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3245 bytes Desc: S/MIME Cryptographic Signature URL: From bwbarre at [hidden] Tue Dec 9 22:32:10 2008 From: bwbarre at [hidden] (Barrett, Brian W) Date: Tue, 9 Dec 2008 21:32:10 -0700 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: <10725_1228853715_mB9KFEr6011609_928CFBE8E7CB0040959E56B4EA41A77E090047E6@irsmsx504.ger.corp.intel.com> Message-ID: For reasons I'm not sure I completely understand, the discussion on this proposal has become focused on micro-optimizations for MPI implementations, rather than the original desire to fix one of the many usability complaints users have with the MPI standard. I personally believe that the send buffer restriction places an onerous burden on the developer of modern MPI applications. While it's straight-forward to ensure a buffer is not used multiple times for simple ghost cell transfers in basic physics codes, it can be more challenging in more advanced applications. We already know of one application in the wild that reuses send buffers (the Parallel BGL). I'm certain that if we polled application writers, we'd find even more which violate the restriction. Even as an MPI implementer who knows what he's doing (in theory, anyway), I find this restriction can be a difficult problem to work around. Further, this is a difficult issue to detect. Yes, a correctness-checking library can track buffers in use. But consider that buffer reuse is likely to be quite transient, and correctness-checking libraries frequently have an extraordinary performance penalty. Normal development cycle testing likely won't cause the issue to appear, since no current mainstream MPI implementation enforces the rule and I'm doubtful that situation will change. Finally, from a user's perspective, it must look rather capricious of the standards committee that a send buffer can't be reused but a one-sided buffer can. I certainly couldn't justify to a user why the standard has such a contradiction. Do we as a committee have the right to be capricious on this issue? Certainly. Should we then wonder why there are so many people running around preaching the evils of MPI? Certainly not. So in my opinion the question really should be framed not as "Is there any optimization that such a change prohibits?", but "Does the usability of the standard outweigh the loss of *potential* optimizations?". I have to think that eliminating a common source of confusion for users is worth losing the ability to increase the performance of sending so many giga-zeros of data. Brian -- Brian W. Barrett Dept. 1422: Scalable Computer Architectures Sandia National Laboratories From bwbarre at [hidden] Tue Dec 9 23:36:05 2008 From: bwbarre at [hidden] (Barrett, Brian W) Date: Tue, 9 Dec 2008 22:36:05 -0700 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: <10725_1228853715_mB9KFEr6011609_928CFBE8E7CB0040959E56B4EA41A77E090047E6@irsmsx504.ger.corp.intel.com> Message-ID: Ok, now that I've gotten my user's point of view off my chest, back to my view as an MPI implementer. I've tried to respond to all Alexander's objections. If I missed one, I apologize. > 1) The memory remapping scenario IO brought up a couple of days ago Unless someone's done something fun with operating system and architecture design I'm not aware of, remapping has one problem that always must be considered... Send buffers are not required to be (and frequently aren't) page aligned or a multiple of page size. Therefore, completely removing the send buffer from the user's process has the problem of also taking legal addresses with it (which would violate the standard). IBM's solution is elegant in that it allows remapping without removing from the sender's process space. Sandia has a solution called SMARTMAP that is both not patented and allows single copy transfers in shared memory environments. Point number 2 was a procedural argument. I believe others are in a better position than I to comment on this. My understanding, however, is that a technical objection can cause the vote to fail, but is not grounds on preventing a vote (particularly a second vote). If it were, we'd never get anything done. > 3) Imagine send buffers have to pinned in the memory. To avoid doing this too > often, these registrations will normally be cached. If more than one send can > be used for a buffer or, for that matter, overlapping portions of the same > buffer, say by different threads, access to the lookup-and-pin will have to be > made atomic. This will further complicate implementation and introduce a > potentially costly mutual exclusion primitive into the critical path. The caching problem already exists. Consider a case where a large send is completed, then multiple small sends occur within that base and bound after the first is completed. This situation is perfectly legal, happens in codes in the wild, and must be dealt with by MPI implementations. If that's not enough, consider a case where the buffer is part of an active Window (which is legal, as long as the buffers in use for communication don't overlap). All these cases certainly should be handled by an MPI today. > 4) I wonder what a const modifier will do for a buffer identifies by > MPI_BOTTOM and/or a derived data type, possibly with holes in it. How will > this square up with the C language sequence association rules? This sounds like an issue for the const proposal, which is different from the send buffer access proposal. I'm not sure I have enough data to form an opinion on the const proposal, but I'm fairly sure we can discuss the send buffer access proposal without considering this issue. > 5) Note also if both #45 and #46 will be introduced, there will be no way to > retract this, even with the help of the MPI_INIT_ASSERTED, should we later > decide to introduce assertion like MPI_NO_SEND_BUFFER_READ_ACCESS. The const > modifier from #46 will make that syntactically useless. If both are passed, that might be true. It could be argued the const proposal depends on the access proposal. However, it can not be rationally argued that the access proposal in any way depends upon the const proposal. The send buffer access proposal can certainly be passed and an assert added later (at whatever point the init_assert proposal is integrated into the standard) that allows MPI implementations to modify the send buffer. You raise a good point about the const proposal. But it has absolutely no bearing on the send buffer access proposal. > 6) Finally, what will happen in the Fortran interface? With the > copy-in/copy-out possibly happening on the MPI subroutine boundary for array > sections? If more than one send is allowed, the application can pretty easily > exhaust any virtual memory with a couple of long enough vectors. How does that change from today? Today users send multiple buffers at the same time, and seem to cope with memory exhaustion issues just fine. So soon they might be able to remove the data copy they've had to make at the user level to work around the MPI access restriction, so there's actually less virtual memory in use. Seems like a win to me. > 7) In-place compression and/or encryption of the messages. Compression in > particular can work wonders on monotonous messages, and cost less time in > total than the transmission of so many giga-zeroes, for example. Again, having > send buffer access allowed and const modifier attached will kill this huge > optimization opportunity. Too bad. While I hope you're joking about the giga-zeros, you do raise a valid concern, in that there are a number of optimizations regarding compression, encryption, and endian-swapping that may be eliminated by this proposal. On the flip side, as I argued in a previous e-mail, the user gains quite a bit in usability. We have to balance these two factors. Since users know where my office is, I tend to lean towards making their lives easier, particularly when it doesn't cause extra work for me. But I already sent an e-mail on that point... Our experience with Open MPI was that the potential for performance in other parts of the MPI (collectives, etc.) far outweighed any send-side tricks we could think of (and you haven't brought up any we didn't think of). So if we wanted to do compression or encryption, it would be done with send-side bounce buffers. Since a software pipeline would practically be required to get good performance, the bounce buffer would not have to scale with the size of the communication buffer but instead with the properties of the network pipeline. Of course, my opinion would be that it would be much simpler and much higher performance to support compression or encryption as part of the NIC as the data is streamed to the network. Otherwise, you're burning memory bandwidth doing the extra copy (even in the modify the send buffer case), and memory bandwidth is a precious resource for HPC applications. One other point to consider. If I was a user, I'd expect that my one-sided traffic also be compressed, encrypted, or endian-swapped. The standard already requires multiple accesses be legal for one-sided communication. So you're going to have a situation where some communication can use a send-modify implementation and some can not. I'm not familiar with how Intel's MPI is architected, but Open MPI is architected such that decisions such as compression, encryption, and endian-swapping would be made at a low enough level that the code path is the same whether the message is a point-to-point send or a one-sided put. Since that's some of the most complicated code in Open MPI, I can't foresee adding a second code path just to get a (dubious) performance benefit. Brian -- Brian W. Barrett Dept. 1422: Scalable Computer Architectures Sandia National Laboratories From alexander.supalov at [hidden] Wed Dec 10 08:51:21 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Wed, 10 Dec 2008 14:51:21 +0000 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: Message-ID: <928CFBE8E7CB0040959E56B4EA41A77E09004BDD@irsmsx504.ger.corp.intel.com> Dear Brian, Thank you. From the implementor's point of you, to be consistent, we probably should prohibit access to the engaged 1-sided buffers as well. Best regards. Alexander -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Barrett, Brian W Sent: Wednesday, December 10, 2008 5:32 AM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) For reasons I'm not sure I completely understand, the discussion on this proposal has become focused on micro-optimizations for MPI implementations, rather than the original desire to fix one of the many usability complaints users have with the MPI standard. I personally believe that the send buffer restriction places an onerous burden on the developer of modern MPI applications. While it's straight-forward to ensure a buffer is not used multiple times for simple ghost cell transfers in basic physics codes, it can be more challenging in more advanced applications. We already know of one application in the wild that reuses send buffers (the Parallel BGL). I'm certain that if we polled application writers, we'd find even more which violate the restriction. Even as an MPI implementer who knows what he's doing (in theory, anyway), I find this restriction can be a difficult problem to work around. Further, this is a difficult issue to detect. Yes, a correctness-checking library can track buffers in use. But consider that buffer reuse is likely to be quite transient, and correctness-checking libraries frequently have an extraordinary performance penalty. Normal development cycle testing likely won't cause the issue to appear, since no current mainstream MPI implementation enforces the rule and I'm doubtful that situation will change. Finally, from a user's perspective, it must look rather capricious of the standards committee that a send buffer can't be reused but a one-sided buffer can. I certainly couldn't justify to a user why the standard has such a contradiction. Do we as a committee have the right to be capricious on this issue? Certainly. Should we then wonder why there are so many people running around preaching the evils of MPI? Certainly not. So in my opinion the question really should be framed not as "Is there any optimization that such a change prohibits?", but "Does the usability of the standard outweigh the loss of *potential* optimizations?". I have to think that eliminating a common source of confusion for users is worth losing the ability to increase the performance of sending so many giga-zeros of data. Brian -- Brian W. Barrett Dept. 1422: Scalable Computer Architectures Sandia National Laboratories _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. From wgropp at [hidden] Wed Dec 10 10:24:27 2008 From: wgropp at [hidden] (William Gropp) Date: Wed, 10 Dec 2008 10:24:27 -0600 Subject: [Mpi-22] MPI 2.2 Reminders In-Reply-To: <6B68D01C00C9994A8E150183E62A119E7B5C9F7A69@NA-EXMSG-C105.redmond.corp.microsoft.com> Message-ID: <57BB1841-578D-4D34-8E76-13A4677343AA@illinois.edu> I've added some information on the review process, including a list of chapter authors and the state diagram that Jeff created, to the 2.2 wiki page. Bill On Dec 5, 2008, at 3:50 PM, Erez Haba wrote: > Where can I find the list of chapter authors? > > -----Original Message----- > From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden] > ] On Behalf Of William Gropp > Sent: Friday, December 05, 2008 8:07 AM > To: Bronis R. de Supinski; MPI 2.2 > Subject: Re: [Mpi-22] MPI 2.2 Reminders > > As an immediate step, we can do the following: > > Authors of tickets that are ready for review should do the following: > (a) add a comment to the ticket with the names of the reviewers (the > relevant chapter author and 3 other volunteers) > (b) contact the reviewers to remind them > > Having a separate, searchable field would be nice, but this would > capture the data now. > > Bill > > On Dec 5, 2008, at 10:00 AM, Bronis R. de Supinski wrote: > >> >> Bill and Jeff: >> >> It will be really useful if the tickets listed reviewers. In the >> least >> we need some way to map tickets to reviewers (I am thinking about >> chapter >> authors in particular), Could Jeff add a field (or fields) that the >> owner is responsible for completing that indicates the reviewers? >> >> As it is I am really unclear over what I am supposed to review. I am >> guessing there are at least one or two for my chapter... >> >> Bronis >> >> >> On Fri, 5 Dec 2008, William Gropp wrote: >> >>> Our next MPI 2.2 discussion is in just over a week, and we have a >>> lot >>> to do. Please have a look at the wiki for the following (see https:// >>> svn.mpi-forum.org/trac/mpi-forum-web/report >>> for the reports): >>> >>> 1) For your own tickets, make sure that they are up to date. If >>> reviewers are needed, please contact them and encourage them to >>> submit >>> their reviews. If revised text is needed, please update the text >>> and >>> if you would like early feedback, notify this list. >>> >>> 2) For items on which you are a reviewer (particularly MPI 2.1 >>> chapter >>> authors), please review the relevant tickets. When you are >>> reviewing >>> text, make sure that you are looking at the current MPI 2.1 >>> standard; >>> check page and line numbers as well as the context to ensure that >>> the >>> change is correct. >>> >>> 3) For other tickets, review them and raise any issues that you have >>> *now* so that we can start the discussion before our meeting. >>> >>> Thanks! >>> >>> Bill >>> >>> William Gropp >>> Deputy Director for Research >>> Institute for Advanced Computing Applications and Technologies >>> Paul and Cynthia Saylor Professor of Computer Science >>> University of Illinois Urbana-Champaign >>> >>> >>> >>> >>> _______________________________________________ >>> mpi-22 mailing list >>> mpi-22_at_[hidden] >>> http:// lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 >>> >>> >> _______________________________________________ >> mpi-22 mailing list >> mpi-22_at_[hidden] >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > William Gropp > Deputy Director for Research > Institute for Advanced Computing Applications and Technologies > Paul and Cynthia Saylor Professor of Computer Science > University of Illinois Urbana-Champaign > > > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign From alexander.supalov at [hidden] Wed Dec 10 10:50:33 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Wed, 10 Dec 2008 16:50:33 +0000 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: Message-ID: <928CFBE8E7CB0040959E56B4EA41A77E0CB5753B@irsmsx504.ger.corp.intel.com> Dear Brian, Thanks. Now that I've already got my implementor's view out on the balance between the user friendliness and implementation freedom in the reply to your other letter, let me follow you on the way back to the proposal and counter-arguments. I reply below using AS> as a prefix. Before I do this, let me sum up my position: 1) The send access proposal restricts freedom of implementation. This statement is based on historic precedent and several realistic, forward looking scenarios mentioned earlier in this trail. The only argument in favor of 1) is the desire to make an unknown number of incorrect MPI programs valid. I've never seen any data on how many applications were involved, by the way. Do you know? One? Two? Or more? 2) The const proposal depends on 1) and makes it irreversible thru a syntactic change. Since 1) is questionable, 2) is questionable by inference. Moreover, 2) has a syntactic weakness of its own due to the const semantics in the C language, again identified earlier in this trail. Whether or not the unrelated MPI_INIT_ASSERTED proposal is going to help with 1) remains to be seen. Accepting 2) now will definitely close that door. Even though a compromise (like accepting 1) now with the possibility to fix it later thru MPI_INIT_ASSERTED, and not accepting 2)) may look tempting, this may not be the "right" way if you ask me. Breaking something only to fix it later, maybe, while encouraging dangerous programming practices for now, does not seem totally prudent to me. "MPI standard is changing all the time" is not the signal we want to send. Finally, that the one-sided communication does not impose a restriction on the buffer access is a different matter. It is unclear to me whether this precedent should be used to justify the proposed change. It may itself be open to review - possibly, not now. This is why I would propose to vote both proposals down when it comes to the vote that, as I've already agreed, is the correct procedure to follow. Best regards. Alexander -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Barrett, Brian W Sent: Wednesday, December 10, 2008 6:36 AM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Ok, now that I've gotten my user's point of view off my chest, back to my view as an MPI implementer. I've tried to respond to all Alexander's objections. If I missed one, I apologize. > 1) The memory remapping scenario IO brought up a couple of days ago Unless someone's done something fun with operating system and architecture design I'm not aware of, remapping has one problem that always must be considered... Send buffers are not required to be (and frequently aren't) page aligned or a multiple of page size. Therefore, completely removing the send buffer from the user's process has the problem of also taking legal addresses with it (which would violate the standard). IBM's solution is elegant in that it allows remapping without removing from the sender's process space. Sandia has a solution called SMARTMAP that is both not patented and allows single copy transfers in shared memory environments. AS> See Hubert's reply. The point of this particular argument is that the possible restriction of the proposal upon the implementation freedom may have been overlooked. My bet is that we're getting but a corner of this matter raised - see other points. Point number 2 was a procedural argument. I believe others are in a better position than I to comment on this. My understanding, however, is that a technical objection can cause the vote to fail, but is not grounds on preventing a vote (particularly a second vote). If it were, we'd never get anything done. AS> I agreed on the procedure after Erez' and Bill's clarifications. We're going to have whatever vote is due. We're preparing our position to voice it before the voting. > 3) Imagine send buffers have to pinned in the memory. To avoid doing this too > often, these registrations will normally be cached. If more than one send can > be used for a buffer or, for that matter, overlapping portions of the same > buffer, say by different threads, access to the lookup-and-pin will have to be > made atomic. This will further complicate implementation and introduce a > potentially costly mutual exclusion primitive into the critical path. The caching problem already exists. Consider a case where a large send is completed, then multiple small sends occur within that base and bound after the first is completed. This situation is perfectly legal, happens in codes in the wild, and must be dealt with by MPI implementations. If that's not enough, consider a case where the buffer is part of an active Window (which is legal, as long as the buffers in use for communication don't overlap). All these cases certainly should be handled by an MPI today. AS> Let me try to understand this. You say above that the large send is completed. If I got this correctly, the buffer is no longer engaged. Where is the problem to discuss here? We're discussing sends that try to send the same buffer while it's already engaged. Pleas clarify. > 4) I wonder what a const modifier will do for a buffer identifies by > MPI_BOTTOM and/or a derived data type, possibly with holes in it. How will > this square up with the C language sequence association rules? This sounds like an issue for the const proposal, which is different from the send buffer access proposal. I'm not sure I have enough data to form an opinion on the const proposal, but I'm fairly sure we can discuss the send buffer access proposal without considering this issue. AS> The const proposal depends on the send buffer proposal. This argument revealed yet another possible complication in the const proposal. > 5) Note also if both #45 and #46 will be introduced, there will be no way to > retract this, even with the help of the MPI_INIT_ASSERTED, should we later > decide to introduce assertion like MPI_NO_SEND_BUFFER_READ_ACCESS. The const > modifier from #46 will make that syntactically useless. If both are passed, that might be true. It could be argued the const proposal depends on the access proposal. However, it can not be rationally argued that the access proposal in any way depends upon the const proposal. The send buffer access proposal can certainly be passed and an assert added later (at whatever point the init_assert proposal is integrated into the standard) that allows MPI implementations to modify the send buffer. You raise a good point about the const proposal. But it has absolutely no bearing on the send buffer access proposal. AS> This comment was primarily directed at the const proposal, you're right. > 6) Finally, what will happen in the Fortran interface? With the > copy-in/copy-out possibly happening on the MPI subroutine boundary for array > sections? If more than one send is allowed, the application can pretty easily > exhaust any virtual memory with a couple of long enough vectors. How does that change from today? Today users send multiple buffers at the same time, and seem to cope with memory exhaustion issues just fine. So soon they might be able to remove the data copy they've had to make at the user level to work around the MPI access restriction, so there's actually less virtual memory in use. Seems like a win to me. AS> I'm not sure we're talking about the same problem here. Fortran copy-in/out is done by compiler runtime to create contiguous data out of a possibly noncontiguous array section. If more than one Send is allowed, there will possibly be as many hidden copies. If this goes on and on, the memory may get exhausted. Sending different buffers is OK. The main argument of the proposal is, as far as I got it, that the user may resend the same buffer over and over again. This buffer may be large, otherwise why trying to write an incorrect program? And a big buffer copied many time may be an extra issue. > 7) In-place compression and/or encryption of the messages. Compression in > particular can work wonders on monotonous messages, and cost less time in > total than the transmission of so many giga-zeroes, for example. Again, having > send buffer access allowed and const modifier attached will kill this huge > optimization opportunity. Too bad. While I hope you're joking about the giga-zeros, you do raise a valid concern, in that there are a number of optimizations regarding compression, encryption, and endian-swapping that may be eliminated by this proposal. On the flip side, as I argued in a previous e-mail, the user gains quite a bit in usability. We have to balance these two factors. Since users know where my office is, I tend to lean towards making their lives easier, particularly when it doesn't cause extra work for me. But I already sent an e-mail on that point... AS> I see and respect this point of view. I just beg to suggest that the balance in this case may have to be adjusted to allow more freedom of implementation, which would generally follow the spirit of many (most?) decisions MPI Forum made before. Our experience with Open MPI was that the potential for performance in other parts of the MPI (collectives, etc.) far outweighed any send-side tricks we could think of (and you haven't brought up any we didn't think of). So if we wanted to do compression or encryption, it would be done with send-side bounce buffers. Since a software pipeline would practically be required to get good performance, the bounce buffer would not have to scale with the size of the communication buffer but instead with the properties of the network pipeline. Of course, my opinion would be that it would be much simpler and much higher performance to support compression or encryption as part of the NIC as the data is streamed to the network. Otherwise, you're burning memory bandwidth doing the extra copy (even in the modify the send buffer case), and memory bandwidth is a precious resource for HPC applications. AS> We're talking about possibly huge savings here. And the proposal seems to limit the ways in which these savings can be achieved. We're engineers and researchers here, we can work around any obstacle (death and taxes included). My point is that we may have a better application to the creativity than artificially imposing limitation on the implementation in this particular place. One other point to consider. If I was a user, I'd expect that my one-sided traffic also be compressed, encrypted, or endian-swapped. The standard already requires multiple accesses be legal for one-sided communication. So you're going to have a situation where some communication can use a send-modify implementation and some can not. I'm not familiar with how Intel's MPI is architected, but Open MPI is architected such that decisions such as compression, encryption, and endian-swapping would be made at a low enough level that the code path is the same whether the message is a point-to-point send or a one-sided put. Since that's some of the most complicated code in Open MPI, I can't foresee adding a second code path just to get a (dubious) performance benefit. AS> We're talking about big savings here (2 and more times, potentially). I'd not call that dubious, to start with. Coming back to the argument that one-sided allow access to the engaged buffer, I ask myself whether that was the right decision and why it was made. It's not that I wanted to reverse that right now. It's about whether a questionable decision made once, for whatever reason, should predefine our actions since then. A precedent may be positive and negative. In this case, it's probably negative. It should not be use to justify the intention to do a wrong thing once again. It should be reviewed in the light of what is right. Brian -- Brian W. Barrett Dept. 1422: Scalable Computer Architectures Sandia National Laboratories _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. From treumann at [hidden] Wed Dec 10 12:44:47 2008 From: treumann at [hidden] (Richard Treumann) Date: Wed, 10 Dec 2008 13:44:47 -0500 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: Message-ID: Brian has stated the issue related to the send buffer access proposal very well. We have seen some arguments that focus on how hard the rule is for users, how counter intuitive it is and how often it is violated either naively or knowingly. These are important issues but not the only issues. We have seen arguments about the ways optimization options could be lost if the rule is changed. These are important issues but not the only issues. As Brian states: there is a cost benefit analysis here, not a perfect or even obvious right answer. Early in the debate I raised the question about whether we could be risking optimizations and reminded people that it really does matter. That was at a stage when I perceived a lot of passion based on user centered arguments and almost no discussion of the other side. The tone I perceived was basically "the rationale in the original standard seems to be moot so all that remains is the user centered argument". I felt the counter argument at least needed to be examined. That has happened. I think we have now reached a point where both sides of the debate have been aired. People on each side have heard and considered the counter arguments. I think most forum members would agree that we cannot be absolutely certain whether keeping the send buffer restriction would ever prove to be valuable. I still think there is some risk in removing the restriction but I also see substantial value in removing it. My take is that as a cost/benefit decision we should remove the restriction on send buffer access. I also think putting the compiler attribute "const" on a send buffer parameter should be voted down. The formal argument to a send seen by the compiler may or may not correspond to the buffer. The datatype offsets play a role too. This most obvious case is when MPI_BOTTOM is the send argument but there are other examples. MPI_Send(&(array[0]) ...) MPI_Send(array,....) MPI_Send(&var, ....) MPI_Send(MPI_BOTTOM, ......) are all valid ways of sending the content of array[10] when combined with a suitable datatype. (for the "var" example we would need a datatype that set MPI_LB at addr (var) and used ( addr(array[10]-addr(var) ) as a displacement. Weird but valid.) The complier optimizations that can come from adding "const" are probably small. The const attribute is semantically inaccurate if we consider MPI_BOTTOM to represent the entire memory array. Every subroutine call alters some portions of memory. I presume the compiler just recognizes that it has no idea what range MPI_BOTTOM represents and ignores the "const". Dick Dick Treumann - MPI Team IBM Systems & Technology Group Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 12/10/2008 12:36:05 AM: > [image removed] > > Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Barrett, Brian W > > to: > > MPI 2.2, > > 12/10/2008 12:45 AM > > Sent by: > > mpi-22-bounces_at_[hidden] > > Please respond to "MPI 2.2" > > Ok, now that I've gotten my user's point of view off my chest, back to my > view as an MPI implementer. I've tried to respond to all Alexander's > objections. If I missed one, I apologize. > > > 1) The memory remapping scenario IO brought up a couple of days ago > > Unless someone's done something fun with operating system and architecture > design I'm not aware of, remapping has one problem that always must be > considered... Send buffers are not required to be (and frequently aren't) > page aligned or a multiple of page size. Therefore, completely removing the > send buffer from the user's process has the problem of also taking legal > addresses with it (which would violate the standard). IBM's solution is > elegant in that it allows remapping without removing from the sender's > process space. Sandia has a solution called SMARTMAP that is both not > patented and allows single copy transfers in shared memory environments. > > Point number 2 was a procedural argument. I believe others are in a better > position than I to comment on this. My understanding, however, is that a > technical objection can cause the vote to fail, but is not grounds on > preventing a vote (particularly a second vote). If it were, we'd never get > anything done. > > > 3) Imagine send buffers have to pinned in the memory. To avoid > doing this too > > often, these registrations will normally be cached. If more than > one send can > > be used for a buffer or, for that matter, overlapping portions of the same > > buffer, say by different threads, access to the lookup-and-pin > will have to be > > made atomic. This will further complicate implementation and introduce a > > potentially costly mutual exclusion primitive into the critical path. > > The caching problem already exists. Consider a case where a large send is > completed, then multiple small sends occur within that base and bound after > the first is completed. This situation is perfectly legal, happens in codes > in the wild, and must be dealt with by MPI implementations. If that's not > enough, consider a case where the buffer is part of an active Window (which > is legal, as long as the buffers in use for communication don't overlap). > All these cases certainly should be handled by an MPI today. > > > 4) I wonder what a const modifier will do for a buffer identifies by > > MPI_BOTTOM and/or a derived data type, possibly with holes in it. How will > > this square up with the C language sequence association rules? > > This sounds like an issue for the const proposal, which is different from > the send buffer access proposal. I'm not sure I have enough data to form an > opinion on the const proposal, but I'm fairly sure we can discuss the send > buffer access proposal without considering this issue. > > > 5) Note also if both #45 and #46 will be introduced, there will beno way to > > retract this, even with the help of the MPI_INIT_ASSERTED, should we later > > decide to introduce assertion like MPI_NO_SEND_BUFFER_READ_ACCESS.The const > > modifier from #46 will make that syntactically useless. > > If both are passed, that might be true. It could be argued the const > proposal depends on the access proposal. However, it can not be rationally > argued that the access proposal in any way depends upon the const proposal. > > The send buffer access proposal can certainly be passed and an assert added > later (at whatever point the init_assert proposal is integrated into the > standard) that allows MPI implementations to modify the send buffer. > > You raise a good point about the const proposal. But it has absolutely no > bearing on the send buffer access proposal. > > > 6) Finally, what will happen in the Fortran interface? With the > > copy-in/copy-out possibly happening on the MPI subroutine boundaryfor array > > sections? If more than one send is allowed, the application can > pretty easily > > exhaust any virtual memory with a couple of long enough vectors. > > How does that change from today? Today users send multiple buffers at the > same time, and seem to cope with memory exhaustion issues just fine. So > soon they might be able to remove the data copy they've had to make at the > user level to work around the MPI access restriction, so there's actually > less virtual memory in use. Seems like a win to me. > > > 7) In-place compression and/or encryption of the messages. Compression in > > particular can work wonders on monotonous messages, and cost less time in > > total than the transmission of so many giga-zeroes, for example. > Again, having > > send buffer access allowed and const modifier attached will kill this huge > > optimization opportunity. Too bad. > > While I hope you're joking about the giga-zeros, you do raise a valid > concern, in that there are a number of optimizations regarding compression, > encryption, and endian-swapping that may be eliminated by this proposal. On > the flip side, as I argued in a previous e-mail, the user gains quite a bit > in usability. We have to balance these two factors. Since users know where > my office is, I tend to lean towards making their lives easier, particularly > when it doesn't cause extra work for me. But I already sent an e-mail on > that point... > > Our experience with Open MPI was that the potential for performance in other > parts of the MPI (collectives, etc.) far outweighed any send-side tricks we > could think of (and you haven't brought up any we didn't think of). So if > we wanted to do compression or encryption, it would be done with send-side > bounce buffers. Since a software pipeline would practically be required to > get good performance, the bounce buffer would not have to scale with the > size of the communication buffer but instead with the properties of the > network pipeline. Of course, my opinion would be that it would be much > simpler and much higher performance to support compression or encryption as > part of the NIC as the data is streamed to the network. Otherwise, you're > burning memory bandwidth doing the extra copy (even in the modify the send > buffer case), and memory bandwidth is a precious resource for HPC > applications. > > One other point to consider. If I was a user, I'd expect that my one-sided > traffic also be compressed, encrypted, or endian-swapped. The standard > already requires multiple accesses be legal for one-sided communication. So > you're going to have a situation where some communication can use a > send-modify implementation and some can not. I'm not familiar with how > Intel's MPI is architected, but Open MPI is architected such that decisions > such as compression, encryption, and endian-swapping would be made at a low > enough level that the code path is the same whether the message is a > point-to-point send or a one-sided put. Since that's some of the most > complicated code in Open MPI, I can't foresee adding a second code path just > to get a (dubious) performance benefit. > > > Brian > > -- > Brian W. Barrett > Dept. 1422: Scalable Computer Architectures > Sandia National Laboratories > > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From treumann at [hidden] Wed Dec 10 13:48:35 2008 From: treumann at [hidden] (Richard Treumann) Date: Wed, 10 Dec 2008 14:48:35 -0500 Subject: [Mpi-22] Empty members and MPI_TYPE_GET_CONTENTS Message-ID: There has been a first vote on this issue. I was not present for the discussion so I would like to present my argument for a different outcome on the second vote. The first vote opted for the position that MPI_TYPE_GET_CONTENTS should return information on empty type constructor inputs, ones that had replication count or block lengths arguments of zero. I do not think that is the best answer. Here is the issue as stated in the ticket: ================================ MPI-2.1 introduced an ambiguity in the interpretation of what MPI_TYPE_GET_CONTENTS is supposed to return. Specifically: MPI-2.1 introduced that user-defined datatypes could be created with zero counts for datatypes. The ambiguity originates from two sections of text: MPI-2.1: p78:18-20 Most datatype constructors have replication count or block length arguments. Allowed values are nonnegative integers. If the value is zero, no elements are generated in the type map and there is no effect on datatype bounds or extent. MPI-2.1: p107:47-48 The datatypes returned in array_of_datatypes are handles to datatype objects that are equivalent to the datatypes used in the original construction call. ================================= The alternatives provided were: say: The datatypes returned in array_of_datatypes are handles to datatype objects that are equivalent to the datatypes used in the original construction call, including any zero-count datatypes that were specified in the original datatype constructor. ay: The datatypes returned in array_of_datatypes are handles to datatype objects that are equivalent to the significant datatypes used in the original construction call. (datatypes with 0 count or 0 blocklength are not considered significant) ================================= Here are my arguments for the second option: 1) The phrasing now in MPI-2.1: p78:18-20 enumerates the relevant "contents" of a datatype object and says none of these contents are affected by a parameter that represents 0 bytes. If an input parameter does not affect the "contents" it does not seem consistent to require that MPI_TYPE_GET_CONTENTS report that input. The function is not MPI_TYPE_ECHO_ORIGINAL_ARGUMENTS. 2) The primary uses for MPI_TYPE_GET_ENVELOP and MPI_TYPE_GET_CONTENTS are to provide a way for tools and opaque libraries to learn the data layout of a datatype created elsewhere. Any input to a type constructor that did not affect type signature, offsets, bounds or extent is irrelevant to this purpose. If MPI_TYPE_GET_CONTENTS returns this irrelevant history then the user of MPI_TYPE_GET_CONTENTS must have special case code to filter out this chaff. Note: there was a suggestion in the ticket that the the "user" who created a datatype that includes 0 byte members ought to be willing to deal with them when he calls MPI_TYPE_GET_CONTENTS - the problem with this logic is that the user who created such a datatype is not the user who must filter the output of MPI_TYPE_GET_CONTENTS. Any user of MPI_TYPE_GET_CONTENTS that does not control the entire application will need to filter the output in case someone he cannot control produces datatypes with empty members. If he does not filter out this chaff, he must test all his algorithms that use the results of MPI_TYPE_GET_CONTENTS to ensure they tolerate empty members. 3) As long as MPI_TYPE_GET_ENVELOP only counts those components that became part of the contents under MPI-2.1: p78:18-20, the effect is transparent for datatypes that represent at least one byte. A totally empty datatype will return 0 for the envelope size and that ends the matter. There is no need to call MPI_TYPE_GET_CONTENTS. If MPI_TYPE_GET_ENVELOP counts empty members then the user of MPI_TYPE_GET_CONTENTS will not know a datatype represents 0 bytes unless he adds code to check each member size. ee ticket: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/2 Dick Treumann - MPI Team IBM Systems & Technology Group Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From erezh at [hidden] Wed Dec 10 14:16:54 2008 From: erezh at [hidden] (Erez Haba) Date: Wed, 10 Dec 2008 12:16:54 -0800 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: Message-ID: <6B68D01C00C9994A8E150183E62A119E7B5CB08D3C@NA-EXMSG-C105.redmond.corp.microsoft.com> Thanks Dick, I think that you have somewhat wrong perception regarding the C const keyword. The only promises that the C lang is making about passing parameter as 'const' is that the function will not change the content of ANY memory location through THAT pointer. As long as you are not casting away constness in your function you are good and following the const rules. You are allowed to change the content of that same memory through a different pointer that is not const. All the examples that you have below are 100% valid with the const keyword. Thanks, .Erez From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Wednesday, December 10, 2008 10:45 AM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Brian has stated the issue related to the send buffer access proposal very well. We have seen some arguments that focus on how hard the rule is for users, how counter intuitive it is and how often it is violated either naively or knowingly. These are important issues but not the only issues. We have seen arguments about the ways optimization options could be lost if the rule is changed. These are important issues but not the only issues. As Brian states: there is a cost benefit analysis here, not a perfect or even obvious right answer. Early in the debate I raised the question about whether we could be risking optimizations and reminded people that it really does matter. That was at a stage when I perceived a lot of passion based on user centered arguments and almost no discussion of the other side. The tone I perceived was basically "the rationale in the original standard seems to be moot so all that remains is the user centered argument". I felt the counter argument at least needed to be examined. That has happened. I think we have now reached a point where both sides of the debate have been aired. People on each side have heard and considered the counter arguments. I think most forum members would agree that we cannot be absolutely certain whether keeping the send buffer restriction would ever prove to be valuable. I still think there is some risk in removing the restriction but I also see substantial value in removing it. My take is that as a cost/benefit decision we should remove the restriction on send buffer access. I also think putting the compiler attribute "const" on a send buffer parameter should be voted down. The formal argument to a send seen by the compiler may or may not correspond to the buffer. The datatype offsets play a role too. This most obvious case is when MPI_BOTTOM is the send argument but there are other examples. MPI_Send(&(array[0]) ...) MPI_Send(array,....) MPI_Send(&var, ....) MPI_Send(MPI_BOTTOM, ......) are all valid ways of sending the content of array[10] when combined with a suitable datatype. (for the "var" example we would need a datatype that set MPI_LB at addr(var) and used ( addr(array[10]-addr(var) ) as a displacement. Weird but valid.) The complier optimizations that can come from adding "const" are probably small. The const attribute is semantically inaccurate if we consider MPI_BOTTOM to represent the entire memory array. Every subroutine call alters some portions of memory. I presume the compiler just recognizes that it has no idea what range MPI_BOTTOM represents and ignores the "const". Dick Dick Treumann - MPI Team IBM Systems & Technology Group Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 12/10/2008 12:36:05 AM: > [image removed] > > Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Barrett, Brian W > > to: > > MPI 2.2 > > 12/10/2008 12:45 AM > > Sent by: > > mpi-22-bounces_at_[hidden] > > Please respond to "MPI 2.2" > > Ok, now that I've gotten my user's point of view off my chest, back to my > view as an MPI implementer. I've tried to respond to all Alexander's > objections. If I missed one, I apologize. > > > 1) The memory remapping scenario IO brought up a couple of days ago > > Unless someone's done something fun with operating system and architecture > design I'm not aware of, remapping has one problem that always must be > considered... Send buffers are not required to be (and frequently aren't) > page aligned or a multiple of page size. Therefore, completely removing the > send buffer from the user's process has the problem of also taking legal > addresses with it (which would violate the standard). IBM's solution is > elegant in that it allows remapping without removing from the sender's > process space. Sandia has a solution called SMARTMAP that is both not > patented and allows single copy transfers in shared memory environments. > > Point number 2 was a procedural argument. I believe others are in a better > position than I to comment on this. My understanding, however, is that a > technical objection can cause the vote to fail, but is not grounds on > preventing a vote (particularly a second vote). If it were, we'd never get > anything done. > > > 3) Imagine send buffers have to pinned in the memory. To avoid > doing this too > > often, these registrations will normally be cached. If more than > one send can > > be used for a buffer or, for that matter, overlapping portions of the same > > buffer, say by different threads, access to the lookup-and-pin > will have to be > > made atomic. This will further complicate implementation and introduce a > > potentially costly mutual exclusion primitive into the critical path. > > The caching problem already exists. Consider a case where a large send is > completed, then multiple small sends occur within that base and bound after > the first is completed. This situation is perfectly legal, happens in codes > in the wild, and must be dealt with by MPI implementations. If that's not > enough, consider a case where the buffer is part of an active Window (which > is legal, as long as the buffers in use for communication don't overlap). > All these cases certainly should be handled by an MPI today. > > > 4) I wonder what a const modifier will do for a buffer identifies by > > MPI_BOTTOM and/or a derived data type, possibly with holes in it. How will > > this square up with the C language sequence association rules? > > This sounds like an issue for the const proposal, which is different from > the send buffer access proposal. I'm not sure I have enough data to form an > opinion on the const proposal, but I'm fairly sure we can discuss the send > buffer access proposal without considering this issue. > > > 5) Note also if both #45 and #46 will be introduced, there will beno way to > > retract this, even with the help of the MPI_INIT_ASSERTED, should we later > > decide to introduce assertion like MPI_NO_SEND_BUFFER_READ_ACCESS.The const > > modifier from #46 will make that syntactically useless. > > If both are passed, that might be true. It could be argued the const > proposal depends on the access proposal. However, it can not be rationally > argued that the access proposal in any way depends upon the const proposal. > > The send buffer access proposal can certainly be passed and an assert added > later (at whatever point the init_assert proposal is integrated into the > standard) that allows MPI implementations to modify the send buffer. > > You raise a good point about the const proposal. But it has absolutely no > bearing on the send buffer access proposal. > > > 6) Finally, what will happen in the Fortran interface? With the > > copy-in/copy-out possibly happening on the MPI subroutine boundaryfor array > > sections? If more than one send is allowed, the application can > pretty easily > > exhaust any virtual memory with a couple of long enough vectors. > > How does that change from today? Today users send multiple buffers at the > same time, and seem to cope with memory exhaustion issues just fine. So > soon they might be able to remove the data copy they've had to make at the > user level to work around the MPI access restriction, so there's actually > less virtual memory in use. Seems like a win to me. > > > 7) In-place compression and/or encryption of the messages. Compression in > > particular can work wonders on monotonous messages, and cost less time in > > total than the transmission of so many giga-zeroes, for example. > Again, having > > send buffer access allowed and const modifier attached will kill this huge > > optimization opportunity. Too bad. > > While I hope you're joking about the giga-zeros, you do raise a valid > concern, in that there are a number of optimizations regarding compression, > encryption, and endian-swapping that may be eliminated by this proposal. On > the flip side, as I argued in a previous e-mail, the user gains quite a bit > in usability. We have to balance these two factors. Since users know where > my office is, I tend to lean towards making their lives easier, particularly > when it doesn't cause extra work for me. But I already sent an e-mail on > that point... > > Our experience with Open MPI was that the potential for performance in other > parts of the MPI (collectives, etc.) far outweighed any send-side tricks we > could think of (and you haven't brought up any we didn't think of). So if > we wanted to do compression or encryption, it would be done with send-side > bounce buffers. Since a software pipeline would practically be required to > get good performance, the bounce buffer would not have to scale with the > size of the communication buffer but instead with the properties of the > network pipeline. Of course, my opinion would be that it would be much > simpler and much higher performance to support compression or encryption as > part of the NIC as the data is streamed to the network. Otherwise, you're > burning memory bandwidth doing the extra copy (even in the modify the send > buffer case), and memory bandwidth is a precious resource for HPC > applications. > > One other point to consider. If I was a user, I'd expect that my one-sided > traffic also be compressed, encrypted, or endian-swapped. The standard > already requires multiple accesses be legal for one-sided communication. So > you're going to have a situation where some communication can use a > send-modify implementation and some can not. I'm not familiar with how > Intel's MPI is architected, but Open MPI is architected such that decisions > such as compression, encryption, and endian-swapping would be made at a low > enough level that the code path is the same whether the message is a > point-to-point send or a one-sided put. Since that's some of the most > complicated code in Open MPI, I can't foresee adding a second code path just > to get a (dubious) performance benefit. > > > Brian > > -- > Brian W. Barrett > Dept. 1422: Scalable Computer Architectures > Sandia National Laboratories > > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 * -------------- next part -------------- An HTML attachment was scrubbed... URL: From goodell at [hidden] Wed Dec 10 14:43:26 2008 From: goodell at [hidden] (Dave Goodell) Date: Wed, 10 Dec 2008 14:43:26 -0600 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: Message-ID: On Dec 10, 2008, at 12:44 PM, Richard Treumann wrote: > I also think putting the compiler attribute "const" on a send buffer > parameter should be voted down. > > The formal argument to a send seen by the compiler may or may not > correspond to the buffer. The datatype offsets play a role too. This > most obvious case is when MPI_BOTTOM is the send argument but there > are other examples. > MPI_Send(&(array[0]) ...) > MPI_Send(array,....) > MPI_Send(&var, ....) > MPI_Send(MPI_BOTTOM, ......) > are all valid ways of sending the content of array[10] when combined > with a suitable datatype. > (for the "var" example we would need a datatype that set MPI_LB at > addr(var) and used ( addr(array[10]-addr(var) ) as a displacement. > Weird but valid.) > > The complier optimizations that can come from adding "const" are > probably small. The const attribute is semantically inaccurate if we > consider MPI_BOTTOM to represent the entire memory array. Every > subroutine call alters some portions of memory. I presume the > compiler just recognizes that it has no idea what range MPI_BOTTOM > represents and ignores the "const". > Dick, IMO the most compelling reason to add const to the the C bindings has nothing to do with compiler optimizations. This is a usability issue for compilers that enforce const-correctness. If our bindings are incorrectly missing const this will force callers to cast away any const-qualifier prior to the call to MPI_Send (and similar). This is extremely irritating to users and tends to make the code look a fair bit uglier. As const is more commonly used by programmers and enforced by compilers this will be a barrier to MPI adoption and use, particularly by library writers. Also our lack of const-ness is worse than the send buffer restriction in the sense that this is more likely to frustrate experienced programmers who are "doing the right thing" rather than MPI rookies. As Erez has already mentioned, all of your examples should be compatible with a const-ified MPI C binding. -Dave From alexander.supalov at [hidden] Wed Dec 10 18:12:50 2008 From: alexander.supalov at [hidden] (Supalov, Alexander) Date: Thu, 11 Dec 2008 00:12:50 +0000 Subject: [Mpi-22] please review - Send Buffer Access (ticket #45) In-Reply-To: Message-ID: <928CFBE8E7CB0040959E56B4EA41A77E0CB5762A@irsmsx504.ger.corp.intel.com> Hi, Thanks. A couple of closing arguments against both proposals: 8) The IMPI standard (Interoperable MPI, see http://impi.nist.gov/) requires all data to be put on the wire in the external32 representation. The most natural way to do this for a little-endian platform is to transform the data in place on the sender. The proposed changes will effectively disadvantage little-endian platforms in the area of MPI interoperability. 9) To implement external32 representation for the file I/O, one may want to do the in-place conversion on the source process to implement the external32 datarep. One may also want to map the file I/O upon pt2pt and collective communication. Again, the proposed changes will hit the little-endian platforms. Little endian is used a lot in the HPC now and will be used a lot in the future I hope. Does the fate of a couple of incorrect programs, yet to be identified, outweighs specific concerns expressed in this trail, and the potential disadvantage to the little endian HPC platforms in particular? Let the Forum decide. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Wednesday, December 10, 2008 7:45 PM To: MPI 2.2 Subject: Re: [Mpi-22] please review - Send Buffer Access (ticket #45) Brian has stated the issue related to the send buffer access proposal very well. We have seen some arguments that focus on how hard the rule is for users, how counter intuitive it is and how often it is violated either naively or knowingly. These are important issues but not the only issues. We have seen arguments about the ways optimization options could be lost if the rule is changed. These are important issues but not the only issues. As Brian states: there is a cost benefit analysis here, not a perfect or even obvious right answer. Early in the debate I raised the question about whether we could be risking optimizations and reminded people that it really does matter. That was at a stage when I perceived a lot of passion based on user centered arguments and almost no discussion of the other side. The tone I perceived was basically "the rationale in the original standard seems to be moot so all that remains is the user centered argument". I felt the counter argument at least needed to be examined. That has happened. I think we have now reached a point where both sides of the debate have been aired. People on each side have heard and considered the counter arguments. I think most forum members would agree that we cannot be absolutely certain whether keeping the send buffer restriction would ever prove to be valuable. I still think there is some risk in removing the restriction but I also see substantial value in removing it. My take is that as a cost/benefit decision we should remove the restriction on send buffer access. I also think putting the compiler attribute "const" on a send buffer parameter should be voted down. The formal argument to a send seen by the compiler may or may not correspond to the buffer. The datatype offsets play a role too. This most obvious case is when MPI_BOTTOM is the send argument but there are other examples. MPI_Send(&(array[0]) ...) MPI_Send(array,....) MPI_Send(&var, ....) MPI_Send(MPI_BOTTOM, ......) are all valid ways of sending the content of array[10] when combined with a suitable datatype. (for the "var" example we would need a datatype that set MPI_LB at addr(var) and used ( addr(array[10]-addr(var) ) as a displacement. Weird but valid.) The complier optimizations that can come from adding "const" are probably small. The const attribute is semantically inaccurate if we consider MPI_BOTTOM to represent the entire memory array. Every subroutine call alters some portions of memory. I presume the compiler just recognizes that it has no idea what range MPI_BOTTOM represents and ignores the "const". Dick Dick Treumann - MPI Team IBM Systems & Technology Group Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 12/10/2008 12:36:05 AM: > [image removed] > > Re: [Mpi-22] please review - Send Buffer Access (ticket #45) > > Barrett, Brian W > > to: > > MPI 2.2 > > 12/10/2008 12:45 AM > > Sent by: > > mpi-22-bounces_at_[hidden] > > Please respond to "MPI 2.2" > > Ok, now that I've gotten my user's point of view off my chest, back to my > view as an MPI implementer. I've tried to respond to all Alexander's > objections. If I missed one, I apologize. > > > 1) The memory remapping scenario IO brought up a couple of days ago > > Unless someone's done something fun with operating system and architecture > design I'm not aware of, remapping has one problem that always must be > considered... Send buffers are not required to be (and frequently aren't) > page aligned or a multiple of page size. Therefore, completely removing the > send buffer from the user's process has the problem of also taking legal > addresses with it (which would violate the standard). IBM's solution is > elegant in that it allows remapping without removing from the sender's > process space. Sandia has a solution called SMARTMAP that is both not > patented and allows single copy transfers in shared memory environments. > > Point number 2 was a procedural argument. I believe others are in a better > position than I to comment on this. My understanding, however, is that a > technical objection can cause the vote to fail, but is not grounds on > preventing a vote (particularly a second vote). If it were, we'd never get > anything done. > > > 3) Imagine send buffers have to pinned in the memory. To avoid > doing this too > > often, these registrations will normally be cached. If more than > one send can > > be used for a buffer or, for that matter, overlapping portions of the same > > buffer, say by different threads, access to the lookup-and-pin > will have to be > > made atomic. This will further complicate implementation and introduce a > > potentially costly mutual exclusion primitive into the critical path. > > The caching problem already exists. Consider a case where a large send is > completed, then multiple small sends occur within that base and bound after > the first is completed. This situation is perfectly legal, happens in codes > in the wild, and must be dealt with by MPI implementations. If that's not > enough, consider a case where the buffer is part of an active Window (which > is legal, as long as the buffers in use for communication don't overlap). > All these cases certainly should be handled by an MPI today. > > > 4) I wonder what a const modifier will do for a buffer identifies by > > MPI_BOTTOM and/or a derived data type, possibly with holes in it. How will > > this square up with the C language sequence association rules? > > This sounds like an issue for the const proposal, which is different from > the send buffer access proposal. I'm not sure I have enough data to form an > opinion on the const proposal, but I'm fairly sure we can discuss the send > buffer access proposal without considering this issue. > > > 5) Note also if both #45 and #46 will be introduced, there will beno way to > > retract this, even with the help of the MPI_INIT_ASSERTED, should we later > > decide to introduce assertion like MPI_NO_SEND_BUFFER_READ_ACCESS.The const > > modifier from #46 will make that syntactically useless. > > If both are passed, that might be true. It could be argued the const > proposal depends on the access proposal. However, it can not be rationally > argued that the access proposal in any way depends upon the const proposal. > > The send buffer access proposal can certainly be passed and an assert added > later (at whatever point the init_assert proposal is integrated into the > standard) that allows MPI implementations to modify the send buffer. > > You raise a good point about the const proposal. But it has absolutely no > bearing on the send buffer access proposal. > > > 6) Finally, what will happen in the Fortran interface? With the > > copy-in/copy-out possibly happening on the MPI subroutine boundaryfor array > > sections? If more than one send is allowed, the application can > pretty easily > > exhaust any virtual memory with a couple of long enough vectors. > > How does that change from today? Today users send multiple buffers at the > same time, and seem to cope with memory exhaustion issues just fine. So > soon they might be able to remove the data copy they've had to make at the > user level to work around the MPI access restriction, so there's actually > less virtual memory in use. Seems like a win to me. > > > 7) In-place compression and/or encryption of the messages. Compression in > > particular can work wonders on monotonous messages, and cost less time in > > total than the transmission of so many giga-zeroes, for example. > Again, having > > send buffer access allowed and const modifier attached will kill this huge > > optimization opportunity. Too bad. > > While I hope you're joking about the giga-zeros, you do raise a valid > concern, in that there are a number of optimizations regarding compression, > encryption, and endian-swapping that may be eliminated by this proposal. On > the flip side, as I argued in a previous e-mail, the user gains quite a bit > in usability. We have to balance these two factors. Since users know where > my office is, I tend to lean towards making their lives easier, particularly > when it doesn't cause extra work for me. But I already sent an e-mail on > that point... > > Our experience with Open MPI was that the potential for performance in other > parts of the MPI (collectives, etc.) far outweighed any send-side tricks we > could think of (and you haven't brought up any we didn't think of). So if > we wanted to do compression or encryption, it would be done with send-side > bounce buffers. Since a software pipeline would practically be required to > get good performance, the bounce buffer would not have to scale with the > size of the communication buffer but instead with the properties of the > network pipeline. Of course, my opinion would be that it would be much > simpler and much higher performance to support compression or encryption as > part of the NIC as the data is streamed to the network. Otherwise, you're > burning memory bandwidth doing the extra copy (even in the modify the send > buffer case), and memory bandwidth is a precious resource for HPC > applications. > > One other point to consider. If I was a user, I'd expect that my one-sided > traffic also be compressed, encrypted, or endian-swapped. The standard > already requires multiple accesses be legal for one-sided communication. So > you're going to have a situation where some communication can use a > send-modify implementation and some can not. I'm not familiar with how > Intel's MPI is architected, but Open MPI is architected such that decisions > such as compression, encryption, and endian-swapping would be made at a low > enough level that the code path is the same whether the message is a > point-to-point send or a one-sided put. Since that's some of the most > complicated code in Open MPI, I can't foresee adding a second code path just > to get a (dubious) performance benefit. > > > Brian > > -- > Brian W. Barrett > Dept. 1422: Scalable Computer Architectures > Sandia National Laboratories > > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. * -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsquyres at [hidden] Thu Dec 11 10:13:59 2008 From: jsquyres at [hidden] (Jeff Squyres) Date: Thu, 11 Dec 2008 11:13:59 -0500 Subject: [Mpi-22] Empty members and MPI_TYPE_GET_CONTENTS In-Reply-To: Message-ID: <29C7933F-C1DA-469F-8160-3206F1CB0084@cisco.com> FWIW, I don't particularly care whether we return the 0's or not -- I just want the ambiguity resolved so that all MPI implementations do the same thing. That being said: a) Dick makes some pretty good arguments. I can see the value of not returning the 0's. If we voted again, I'd probably vote to not return the 0's. b) I *think* the predominant attitude in the forum for the straw votes was "what the heck; might as well return the 0's since that's what was originally passed." I don't recall there being any other strong technical reason. Specifically: if we take a new straw vote and go the other way (*don't* return the 0's), I'll happily change the proposal to reflect that and we can start it over again (ie., have first reading, etc.). On Dec 10, 2008, at 2:48 PM, Richard Treumann wrote: > > All phases MPI install Test build Test run > Date range: > Hardware: > Show Hide > OS: > Show Hide > MPI name: > Show Hide > MPI version: > Show Hide > Org: > Show Hide > Local username: > Show Hide > Platform name: > Show Hide > [Reset form] [Start over] [Preferences] [Advanced] > Current time (GMT): > 2008-12-11 14:00:11 > Date range (GMT): > 2008-12-11 02:00:11 - 2008-12-11 14:00:11 > Phase(s): > MPI install, Test build, and Test run > Number of rows: > 22 > Absolute date range: > Create permalink > > # > Org > Platform name > Hardware > OS > MPI name > MPI version > MPI install > Test build > Test run > Pass > Fail > Pass > Fail > Pass > Fail > Skip > Timed > Perf > 1 > absoft > Fortran_10.2_32_Suse9.3 > ia32 > Linux > ompi-nightly-trunk > 1.4a1r20114 > 0 > 1 > 0 > 0 > 0 > 0 > 0 > 0 > 0 > 2 > absoft > Fortran_10.2_32_Suse9.3 > ia32 > Linux > ompi-nightly-v1.2 > 1.2.9rc1r20040 > 1 > 0 > 1 > 0 > 24 > 0 > 0 > 0 > 0 > 3 > absoft > Fortran9.5_OSX10.5 > ppc > Darwin > ompi-nightly-trunk > 1.4a1r20114 > 1 > 0 > 1 > 0 > 18 > 0 > 0 > 0 > 0 > 4 > absoft > Fortran9.5_OSX10.5 > ppc > Darwin > ompi-nightly-v1.2 > 1.2.9rc1r20040 > 1 > 0 > 1 > 0 > 18 > 0 > 0 > 0 > 0 > 5 > cisco > svbu-mpi > x86_64 > Linux > ompi-nightly-trunk > 1.4a1r20114 > 19 > 0 > 130 > 0 > 15380 > 74623 > 96 > 220 > 264 > 6 > cisco > svbu-mpi > x86_64 > Linux > ompi-nightly-v1.3 > 1.3rc3r20107 > 20 > 0 > 137 > 0 > 169905 > 58952 > 564 > 500 > 675 > 7 > ibm > ibm_ia32 > ia32 > Linux > ompi-nightly-trunk > 1.4a1r20114 > 1 > 0 > 5 > 0 > 195 > 2 > 0 > 0 > 13 > 8 > ibm > ibm_ia32 > ia32 > Linux > ompi-nightly-v1.2 > 1.2.9rc1r20040 > 1 > 0 > 5 > 0 > 2 > 193 > 0 > 2 > 0 > 9 > iu > IU_BigRed > ppc64 > Linux > ompi-nightly-trunk > 1.4a1r20114 > 2 > 0 > 10 > 0 > 2560 > 9 > 18 > 11 > 0 > 10 > iu > IU_BigRed > ppc64 > Linux > ompi-nightly-v1.2 > 1.2.9rc1r20040 > 2 > 0 > 8 > 0 > 2534 > 34 > 18 > 0 > 0 > 11 > iu > IU_BigRed > ppc64 > Linux > ompi-nightly-v1.3 > 1.3rc3r20107 > 2 > 0 > 10 > 0 > 2552 > 10 > 18 > 18 > 0 > 12 > iu > IU_Odin > x86_64 > Linux > ompi-nightly-trunk > 1.4a1r20114 > 10 > 0 > 48 > 0 > 7112 > 0 > 8 > 4 > 0 > 13 > iu > IU_Odin > x86_64 > Linux > ompi-nightly-v1.2 > 1.2.9rc1r20040 > 2 > 0 > 8 > 0 > 1491 > 6 > 4 > 3 > 0 > 14 > iu > IU_Odin > x86_64 > Linux > ompi-nightly-v1.3 > 1.3rc3r20107 > 10 > 0 > 48 > 0 > 7116 > 0 > 8 > 0 > 0 > 15 > iu > IU_Sif > x86_64 > Linux > ompi-nightly-trunk > 1.4a1r20114 > 5 > 0 > 18 > 0 > 4187 > 0 > 15 > 3 > 0 > 16 > iu > IU_Sif > x86_64 > Linux > ompi-nightly-v1.2 > 1.2.9rc1r20040 > 1 > 0 > 4 > 0 > 3294 > 15 > 15 > 1 > 0 > 17 > iu > IU_Sif > x86_64 > Linux > ompi-nightly-v1.3 > 1.3rc3r20107 > 5 > 0 > 18 > 0 > 4184 > 3 > 15 > 3 > 0 > 18 > sun > burl-ct-v20z-12 > x86_64 > Linux > ompi-nightly-trunk > 1.4a1r20098 > 0 > 0 > 0 > 0 > 4506 > 4 > 376 > 0 > 56 > 19 > sun > burl-ct-v20z-2 > i86pc > SunOS > ompi-nightly-trunk > 1.4a1r20098 > 0 > 0 > 15 > 0 > 5254 > 0 > 456 > 0 > 56 > 20 > sun > burl-ct-v20z-2 > i86pc > SunOS > ompi-nightly-v1.3 > 1.3rc3r20092 > 0 > 0 > 15 > 0 > 5152 > 0 > 456 > 0 > 56 > 21 > sun > burl-ct-v440-2 > sun4u > SunOS > ompi-nightly-trunk > 1.4a1r20098 > 0 > 0 > 15 > 0 > 2760 > 0 > 12 > 2 > 0 > 22 > sun > burl-ct-v440-2 > sun4u > SunOS > ompi-nightly-v1.3 > 1.3rc3r20092 > 1 > 0 > 15 > 0 > 2757 > 3 > 12 > 2 > 0 > Totals > 84 > 1 > 512 > 0 > 241001 > 133854 > 2091 > 769 > 1120 > > > Total script execution time: 32 second(s) > > Total SQL execution time: 31 second(s) > > Overall MTT contribution graph (updated nightly): mtt-contrib.pdf > -- Jeff Squyres Cisco Systems