[Mpi3-hybridpm] Endpoints Proposal

[Jeff Hammond]

On Tue, Mar 19, 2013 at 5:23 PM, Sur, Sayantan <sayantan.sur at intel.com> wrote:
>> On 03/19/2013 12:44 PM US Central Time, Sur, Sayantan wrote:
>> > For example, do we want to create independent endpoints for each
>> > thread? What would be the motivation of doing that? One thought is
>> > that it could help existing MPI+OpenMP codes to be ported in a
>> > conceptually similar manner (just that now each rank is really an
>> > endpoint). Now each endpoint can inject and receive messages to remote
>> > ranks/endpoints independently. However, there is memory cost to pay in
>> > this model. On a system with N nodes, P cores per node memory cost per
>> > node is O(NP^2).
>>
>> The motivation is not to create one endpoint per thread.  The motivation is to
>> give a model where the number of endpoints is not required to always be 1
>> per process.  Yes, in the extreme case, the user can create as many
>> endpoints as processes, but that's just an extreme usage case.
>> If we did limit how many endpoints can be created per address space, what
>> would that limit be?
>>
>
> OK. I understand that you are just exposing the model and giving as much flexibility as you possibly can.
>
> What I am trying to find out is what problem is the endpoint approach trying to solve? It has major implications on the standard as it touches many chapters.
>
> Just as an example: Suppose there is an MPI+OpenMP app that runs on 16 cores with 1 MPI rank and 16 threads. On certain platform you find out if there are two endpoints you get better network utilization. In this case, can you not just run 2 MPI ranks with 8 threads each? How is this not achieve the same effect as your endpoint proposal?

Most apps run best with MPI only until they run out of memory.  Your
argument can and often does lead back to MPI-only if applied
inductively.  It's really not an intellectually stimulating example to
discuss.

Jeff

-- 
Jeff Hammond
Argonne Leadership Computing Facility
University of Chicago Computation Institute
jhammond at alcf.anl.gov / (630) 252-5381
http://www.linkedin.com/in/jeffhammond
https://wiki.alcf.anl.gov/parts/index.php/User:Jhammond