<html><body>
<p>Hi Bronis<br>
<br>
For some time IBM MPI offered a User Space communication mode in which certain adapter resources are mapped into process space. The mapping happens at MPI_Init and once it is done, MPI communication calls can manipulate the adapter to send/recv without a need to switch into the kernel. The ability to access the adapter is actually managed by AIX on a per thread basis. That means that on return from MPI_Init, the thread that called MPI_Init can manipulate the adapter to send/recv but other threads in the process cannot (yet).<br>
<br>
If an appplication thread that did not go through MPI_Init tries to do send/recv we recognize that it is not yet registered and register it on the fly. We do this because we added thread support before the MPI Forum defined MPI_INIT_THREAD with MPI_THREAD_FUNNELED. By the time the Forum defined MPI_INIT_THREAD we were offering MPI_THREAD_MULTIPLE so stuck with that even when MPI_THREAD_FUNNELED is requested.<br>
<br>
If we had been starting from scratch we might have decided the for MPI_THREAD_FUNNELED we would not try to do registration on the fly but for MPI_THREAD_MULTIPLE we would.<br>
<br>
Dick <br>
<br>
Dick Treumann - MPI Team/TCEM <br>
IBM Systems & Technology Group<br>
Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601<br>
Tele (845) 433-7846 Fax (845) 433-8363<br>
<br>
<br>
<tt>mpi-22-bounces@cs.uiuc.edu wrote on 01/31/2008 12:57:59 PM:<br>
<br>
> <br>
> Jim and Rich:<br>
> <br>
> Then I suggest that fixing this broken semantic decision is<br>
> something that we should consider for MPI 2.2. It should not<br>
> break any existing programs and might even make some existing<br>
> ones standards conforming.<br>
> <br>
> Although I can imagine ways for the MPI implementation to detect<br>
> that the one thread is not the main thread it is not at all clear<br>
> to me how it would matter to the implementation.<br>
> <br>
> Bronis<br>
> <br>
> <br>
> On Thu, 31 Jan 2008, Cownie, James H wrote:<br>
> <br>
> > Because that's how it's always been. We're not adding a restriction with<br>
> > the change, merely clarifying the existing restriction.<br>
> ><br>
> > -- Jim<br>
> ><br>
> > James Cownie <james.h.cownie@intel.com><br>
> > SSG/DPD/PAT<br>
> > Tel: +44 117 9071438<br>
> ><br>
> > ________________________________<br>
> ><br>
> > From: mpi-21-bounces@cs.uiuc.edu [<a href="mailto:mpi-21-bounces@cs.uiuc.edu">mailto:mpi-21-bounces@cs.uiuc.edu</a>] On<br>
> > Behalf Of Richard Graham<br>
> > Sent: 31 January 2008 15:51<br>
> > To: Mailing list for discussion of MPI 2.1<br>
> > Subject: Re: [mpi-21] Ballot 4 - MPI_THREAD_FUNNELED - was Re:<br>
> > Attending, topics, etc<br>
> ><br>
> ><br>
> ><br>
> > Why restrict this to a standard specified thread (main thread), why not<br>
> > word it<br>
> > as a singe thread, and let the app decide what thread this is, based on<br>
> > what<br>
> > ever criteria it wants to use to select this thread ?<br>
> ><br>
> > Rich<br>
> ><br>
> ><br>
> > On 1/31/08 10:27 AM, "Richard Treumann" <treumann@us.ibm.com> wrote:<br>
> ><br>
> > How about::<br>
> > MPI_THREAD_FUNNELED The process may be multi-threaded, but the<br>
> > application<br>
> > must insure that only the main thread makes MPI calls.<br>
> ><br>
> ><br>
> > Dick Treumann - MPI Team/TCEM<br>
> > IBM Systems & Technology Group<br>
> > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601<br>
> > Tele (845) 433-7846 Fax (845) 433-8363<br>
> ><br>
> ><br>
> > mpi-21-bounces@cs.uiuc.edu wrote on 01/31/2008 09:44:08 AM:<br>
> ><br>
> > > A simpler change which would seem to achieve the desired clarification<br>
> > > would be :-<br>
> > ><br>
> > > MPI_THREAD_FUNNELED The process may be multi-threaded, but only the<br>
> > > main<br>
> > > thread is allowed to make MPI calls.<br>
> > ><br>
> > > (and you could add<br>
> > > If other threads make MPI calls the behavior is undefined.<br>
> > > if you want to be verbose about it).<br>
> > ><br>
> > > -- Jim<br>
> > ><br>
> > > James Cownie <james.h.cownie@intel.com><br>
> > > SSG/DPD/PAT<br>
> > > Tel: +44 117 9071438<br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > > > -----Original Message-----<br>
> > > > From: mpi-21-bounces@cs.uiuc.edu [<a href="mailto:mpi-21-bounces@cs.uiuc.edu">mailto:mpi-21-bounces@cs.uiuc.edu</a>]<br>
> > <<a href="mailto:mpi-21-bounces@cs.uiuc.edu%5d">mailto:mpi-21-bounces@cs.uiuc.edu%5d</a>><br>
> > > On<br>
> > > > Behalf Of Rolf Rabenseifner<br>
> > > > Sent: 31 January 2008 14:31<br>
> > > > To: mpi-21@cs.uiuc.edu<br>
> > > > Subject: [mpi-21] Ballot 4 - MPI_THREAD_FUNNELED - was Re:<br>
> > Attending,<br>
> > > > topics, etc<br>
> > > ><br>
> > > > This is a proposal for MPI 2.1, Ballot 4.<br>
> > > ><br>
> > > > I'm asking especially<br>
> > > > Greg Lindahl,<br>
> > > > the participants of the email-discussion in 2007, to review this<br>
> > > proposal.<br>
> > > ><br>
> > > > This is a follow up to:<br>
> > > > Which thread is the funneled thread?<br>
> > > > in <a href="http://www.cs.uiuc.edu/homes/wgropp/projects/parallel/MPI/mpi-">http://www.cs.uiuc.edu/homes/wgropp/projects/parallel/MPI/mpi-</a><br>
> > > > errata/index.html<br>
> > > > with mail discussion in<br>
> > > > <a href="http://www.cs.uiuc.edu/homes/wgropp/projects/parallel/MPI/mpi-">http://www.cs.uiuc.edu/homes/wgropp/projects/parallel/MPI/mpi-</a><br>
> > > > errata/discuss/funneled/<br>
> > > > ___________________________________<br>
> > > ><br>
> > > > Proposal:<br>
> > > > MPI-2.0 Sect. 8.7.3, MPI_Init_thread, page 196, lines 25-26 read:<br>
> > > ><br>
> > > > MPI_THREAD_FUNNELED The process may be multi-threaded, but only<br>
> > > > the main thread will make MPI calls (all MPI calls are "funneled"<br>
> > > > to the main thread).<br>
> > > ><br>
> > > > but should read:<br>
> > > ><br>
> > > > MPI_THREAD_FUNNELED The process may be multi-threaded, but only<br>
> > > > the main thread will make MPI calls (all MPI calls are "funneled"<br>
> > > > to the main thread, e.g., by using the OpenMP directive "master"<br>
> > > > in the application program).<br>
> > > > ___________________________________<br>
> > > > Rationale for this clarification from the email from Greg Lindahl:<br>
> > > > The existing document doesn't make it clear that<br>
> > > > the MPI user has to funnel the calls to the main thread;<br>
> > > > it's not the job of the MPI library. I have seen multiple<br>
> > > > MPI users confused by this issue, and when I first read<br>
> > > > this section, I was confused by it, too.<br>
> > > > ___________________________________<br>
> > > ><br>
> > > ><br>
> > > > Best regards<br>
> > > > Rolf<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > Dr. Rolf Rabenseifner . . . . . . . . . .. email<br>
> > rabenseifner@hlrs.de<br>
> > > > High Performance Computing Center (HLRS) . phone<br>
> > ++49(0)711/685-65530<br>
> > > > University of Stuttgart . . . . . . . . .. fax ++49(0)711 /<br>
> > 685-65832<br>
> > > > Head of Dpmt Parallel Computing . . .<br>
> > www.hlrs.de/people/rabenseifner<br>
> > > > Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)<br>
> > > > _______________________________________________<br>
> > > > mpi-21 mailing list<br>
> > > > mpi-21@cs.uiuc.edu<br>
> > > > <a href="http://lists.cs.uiuc.edu/mailman/listinfo/mpi-21">http://lists.cs.uiuc.edu/mailman/listinfo/mpi-21</a><br>
> > > ---------------------------------------------------------------------<br>
> > > Intel Corporation (UK) Limited<br>
> > > Registered No. 1134945 (England)<br>
> > > Registered Office: Pipers Way, Swindon SN3 1RJ<br>
> > > VAT No: 860 2173 47<br>
> > ><br>
> > > This e-mail and any attachments may contain confidential material for<br>
> > > the sole use of the intended recipient(s). Any review or distribution<br>
> > > by others is strictly prohibited. If you are not the intended<br>
> > > recipient, please contact the sender and delete all copies.<br>
> > ><br>
> > ><br>
> > > _______________________________________________<br>
> > > mpi-21 mailing list<br>
> > > mpi-21@cs.uiuc.edu<br>
> > > <a href="http://lists.cs.uiuc.edu/mailman/listinfo/mpi-21">http://lists.cs.uiuc.edu/mailman/listinfo/mpi-21</a><br>
> ><br>
> > ________________________________<br>
> ><br>
> > _______________________________________________<br>
> > mpi-21 mailing list<br>
> > mpi-21@cs.uiuc.edu<br>
> > <a href="http://lists.cs.uiuc.edu/mailman/listinfo/mpi-21">http://lists.cs.uiuc.edu/mailman/listinfo/mpi-21</a><br>
> ><br>
> ><br>
> ><br>
> > ---------------------------------------------------------------------<br>
> > Intel Corporation (UK) Limited<br>
> > Registered No. 1134945 (England)<br>
> > Registered Office: Pipers Way, Swindon SN3 1RJ<br>
> > VAT No: 860 2173 47<br>
> ><br>
> > This e-mail and any attachments may contain confidential material for<br>
> > the sole use of the intended recipient(s). Any review or distribution<br>
> > by others is strictly prohibited. If you are not the intended<br>
> > recipient, please contact the sender and delete all copies.<br>
> ><br>
> _______________________________________________<br>
> mpi-22 mailing list<br>
> mpi-22@cs.uiuc.edu<br>
> <a href="http://lists.cs.uiuc.edu/mailman/listinfo/mpi-22">http://lists.cs.uiuc.edu/mailman/listinfo/mpi-22</a><br>
</tt></body></html>