[Mpi3-hybridpm] Reminder for telecon tomorrow

[Pavan Balaji]

Hi Joe,

On 02/17/2010 11:57 AM, Joe Ratterman wrote:
> If you have 8 endpoints doing communication in parallel, they would each
> not need to be able saturate the network; it may be most efficient to
> allow them greater communication parallelism at the software level.
> 
> Allocating as many endpoints as allowed without sacrificing FIFO space
> would not, to the best of my understanding, result in performance issues.

I'm referring to this statement in the previous email:

"If we were to allow users to create up to 8 endpoints per node, it
would only be possible to allocate 4 FIFOs for each endpoint.  Doing
this in all cases would slow the single-threaded case, which will
probably be more common for quite some time."

If creating many FIFOs does not have any performance or resource
consumption issues, then why not create as many as you can at
initialization time, and later figure out how many FIFOs are allocated
to each endpoint?

The reason this is important is by doing this, we can allow MPI
applications to do regular communication after the first phase of the
initialization on MPI_COMM_WORLD. Only communication on
MPI_COMM_ENDPOINTS would be restricted to be performed after the second
phase of the initialization.

For further clarification on the issue I'm raising, consider the
following example:

==========================================================
MPI_Init_endpoints(&argc, &argv);

/* Can communicate on COMM_WORLD now */
MPI_Barrier(MPI_COMM_WORLD);

MPI_Create_endpoints(MPI_MAX_ENDPOINTS, endpoints[]);

for (i = 1 ; i < MPI_MAX_ENDPOINTS; i++)
	pthread_create(...);

MPI_Attach_endpoint(PTHREAD_ID, endpoint[PTHREAD_ID]);

/* Can communicate on COMM_ENDPOINTS now */
MPI_Barrier(MPI_COMM_ENDPOINTS);
==========================================================

To allow such behavior, you can initialize all FIFOs at
MPI_Init_endpoints() time and allow the barrier on COMM_WORLD to
proceed. Later, after MPI_Create_endpoints() and MPI_Attach_endpoint(),
you can allocate the FIFOs to different endpoints.

Am I misunderstanding something here?

Thanks,

 -- Pavan

-- 
Pavan Balaji
http://www.mcs.anl.gov/~balaji