[mpi3-coll] Neighborhood collectives round 2: reductions

[Torsten Hoefler]

On Sat, Dec 08, 2012 at 12:49:35PM -0800, Jed Brown wrote:
>    On Sat, Dec 8, 2012 at 11:12 AM, Torsten Hoefler <[1]htor at illinois.edu>
>    wrote:
> 
>      Hi all,
> 
>      We discussed the neighborhood reductions at the last Forum and the straw
>      vote if we should include them in the next revision was:
> 
>       - yes: 18
>       - no: 0
>       - abstain: 3
> 
>      I addressed all the issues in the draft that were brought up at the
>      Forum. The new draft is now at [2]http://www.unixer.de/sec/topol.pdf .
>      Look
>      for ticket "XXX".
> 
>    The other interfaces for MPI_Ineighbor_reduce do not contain the "I" (page
>    40, line 25 and 30); MPI_Ineighbor_reducev 
>     does not have the "v" (page 41, line 26 and 31).
Thanks! Those are fixed. http://www.unixer.de/sec/topol.pdf

>      One open question remains: would a single send buffer for
>      neighbor_reduce suffice or do we need one buffer per destination
>      process? The second case could always be done with neighbor_reducev
>      (with small additional costs). This questions is more to the potential
>      users (Jed etc.).
> 
>    My understanding of Neighbor_reducev is that I send different sizes to
>    each neighbor, but receive the same size from every neighbor (reducing
>    into exactly one buffer). 
That is correct. 

>    That isn't really a useful operation in domain
>    decomposed methods. The useful operation is that I reduce different sized
>    received buffers into different (but sometimes overlapping) parts of my
>    local buffer.
Well, the issue is that MPI reduction operations cannot be applied to
vectors of different length because this would require the definition of
an identity element. While this is easy for the predefined ops (e.g.,
"MIN_INT" for int max and "0" for sum, etc.), this would require much
more changes for user-defined operations (another call to specify
these). The current interface proposal would require you to fill in
those elements where needed. Or am I misunderstanding or missing something?

>    The single send buffer (with different datatype arguments) imposes no
>    restriction and is natural.
Thanks!

Best,
  Torsten

-- 
### qreharg rug ebs fv crryF ------------- http://www.unixer.de/ -----
Torsten Hoefler           | Assistant Professor
Dept. of Computer Science | ETH Zürich
Universitätsstrasse 6     | Zurich-8092, Switzerland
CAB E 64.1                | Phone: +41 76 309 79 29