[Mpi3-hybridpm] Reminder for telecon tomorrow

[Joe Ratterman]

I was also talking about this.

BGP has a DMA-driven 3D torus network.  The DMA operates by inserting
message descriptors into injection FIFOs, of which there are a finite number
built into the HW.  The DMA will process the head (and only the head)
element in each FIFO all together.  Because large messages will block
subsequent ones, you must use at least 6 FIFOs to achieve the maximum
bandwidth on the 6 outbound links.  There can also be small message latency
problem if a large message is blocking small messages to other nodes.  The
point is, using more FIFOs is generally better.

Let's assume that there are 32 FIFOs per node, and that using 6 per process
or endpoint is considered optimal (both are actually higher in actuality,
but it isn't too important).  If we were to allow users to create up to 8
endpoints per node, it would only be possible to allocate 4 FIFOs for each
endpoint.  Doing this in all cases would slow the single-threaded case,
which will probably be more common for quite some time.


Thanks,
Joe Ratterman
jratt at us.ibm.com


On Wed, Feb 17, 2010 at 3:17 AM, Pavan Balaji <balaji at mcs.anl.gov> wrote:

>
> I wanted to follow up on some of the discussion in the telecon yesterday.
>
> Someone from IBM (Brian?) pointed out that how they initialize their
> communication hardware on BG/P depends on how many endpoints need to be
> created. Can you elaborate on this? Does initializing the hardware with
> more endpoints have a performance impact, or use more memory? If not,
> can't this be initialized with the maximum number of endpoints (this
> information will be available at MPI_Init time)?
>
> Not allowing any communication calls between the two phases of the split
> initialization might be restrictive for users used to the MPI-2.2 kind
> of semantics.
>
> Thanks,
>
>  -- Pavan
>
> On 02/16/2010 09:23 PM, Pavan Balaji wrote:
> > Folks,
> >
> > The minutes for today's telecon are available here:
> >
> https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MPI3Hybrid/notes-2010-02-16
> >
> > Doug: can you upload your helper threads document (current version) to
> > the wiki and add a link on the notes page? Also, it might be useful to
> > version it, so we can follow any changes to it more easily.
> >
> > Thanks,
> >
> >  -- Pavan
> >
> > On 02/15/2010 02:23 PM, Pavan Balaji wrote:
> >> Folks,
> >>
> >> This is a reminder for the telecon tomorrow. The primary agenda item is
> >> continue discussion on Marc's proposal.
> >>
> >> Doug: do you have more notes/details on the helper threads stuff? If
> >> yes, we should try to make some progress on that as well.
> >>
> >> I'm also hoping to talk about small items that were discussed earlier,
> >> but we never decided one way or the other (depending on time available).
> >>
> >> Telecon information is available on the hybrid wiki.
> >>
> >> Thanks,
> >>
> >>  -- Pavan
> >>
> >
>
> --
> Pavan Balaji
> http://www.mcs.anl.gov/~balaji
> _______________________________________________
> Mpi3-hybridpm mailing list
> Mpi3-hybridpm at lists.mpi-forum.org
> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-hybridpm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-hybridpm/attachments/20100217/f730b875/attachment-0001.html>