[Mpi3-hybridpm] Rajeev's and Pavan's questions

[Snir Marc]

> ---------------------------------
>
> Message: 1
> Date: Thu, 13 Aug 2009 12:37:06 -0500
> From: "Rajeev Thakur" <thakur at mcs.anl.gov>
> Subject: Re: [Mpi3-hybridpm] MPI + threads / MPI +OpenMP
> To: <mpi3-hybridpm at lists.mpi-forum.org>
> Message-ID: <882ADD2B55184BDC81DB70D0CED7D168 at mcs.anl.gov>
> Content-Type: text/plain;	charset="us-ascii"
>
> Some questions just for clarification:
>
> * What does MPI_Comm_size of MPI_COMM_TWORLD return -- the total  
> number
> of end points on all processes?

Yes -- total number of endpoints
>
> * For collective operations on MPI_COMM_TWORLD, how many
> processes/threads must call them -- exactly one for each end point? If
> each process has 11 threads and 5 end points say, the user has to  
> ensure
> that exactly 5 threads on each process call the collective, one for  
> each
> end point?
>
Yes, a collective communication requires one invocation for each  
endpoint that is member of the communicator being used. Of course, you  
are free to use a communicator with only one endpoint at each process,  
if this is what you want.

>
>
>
>
>
> Sorry about the delay in joining in on this discussion. This is a very
> good proposal. A couple of questions:
>
> 1. I assume the motivation for defining MPI_ENDPOINTS is to allow the
> implementation to tell the user how many physical connection resources
> it can use (sockets or some other network connection endpoints).
> However, it is possible for multiple MPI endpoints to share the same
> physical network endpoints. This is obviously inefficient since it'll
> bring back all the locking overheads for shared resources, but  
> probably
> not impossible.
>
> My question is -- do we want to explicitly restrict users from  
> creating
> more end points than the available physical resources? Or is it
> beneficial to allow users have any number of MPI endpoints, but have
> MPI_ENDPOINTS specify how many it can "efficiently" support?

Page 6 says "The [MPI_ENDPOINT_CREATE] call is erroneous if  
num_endpoints > MPI_ENDPOINTS". The implementation has a defined  
maximum number of endpoints it supports for each process, and it is  
erroneous to try to create more. If one uses a hybrid model, there is  
no strong advantage of having more than one endpoint per process, but  
for the possible performance benefit. Thus, I do not expect vendors to  
support an arbitrarily large number of endpoints per process, but only  
such an (platform and implementation dependent) number that is  
reasonable, performance wise. Vendors could, for example, allow for  
more endpoints when there is only one process per node, and fewer  
endpoints per process, when there are multiple processes per node.

>
> 2. I didn't quite get the motivation for this statement on page 6: "At
> each process, the endpoint with rank 0 in MPI_COMM_PROCESS is also an
> endpoint in MPI_COMM_WORLD". Obviously, we want *some* endpoint to  
> be a
> part of COMM_WORLD, but can't the user figure this out at runtime?  
> Or is
> this just for simplicity (which is a good reason, but I just wasn't  
> sure)?

For simplicity -- the "ur-endpoint" has rank zero within  
MPI_COMM_PROCESS

>
> Thanks,
>
>  -- Pavan
>
> -- 
> Pavan Balaji
> http://www.mcs.anl.gov/~balaji
>
>
> ------------------------------
>
> _______________________________________________
> Mpi3-hybridpm mailing list
> Mpi3-hybridpm at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm
>
>
> End of Mpi3-hybridpm Digest, Vol 5, Issue 9
> *******************************************

Marc Snir
Michael Faiman and Saburo Muroga Professor
Department of Computer Science, University of Illinois at Urbana  
Champaign
4323 Siebel Center, 201 N Goodwin, IL 61801
Tel (217) 244 6568
Web http://www.cs.uiuc.edu/homes/snir