[Mpi3-hybridpm] New draft of the EI chapter uploaded

[Douglas Miller]

The problem is that none of these INFO options represent the mode I was
discussing in our last meeting. These are different interpretations of how
helper threads will be used and implemented.

I am not at all comfortable with this concept of adding synchronization to
all MPI calls. In fact, I'm not at all sure what it means to set balanced
to true and then go into computation. How do computation threads get
synchronized in those cases?  It sounds the same as balanced=false.


_______________________________________________
Douglas Miller                  BlueGene Messaging Development
IBM Corp., Rochester, MN USA                     Bldg 030-2 A410
dougmill at us.ibm.com               Douglas Miller/Rochester/IBM


                                                                           
             Pavan Balaji                                                  
             <balaji at mcs.anl.g                                             
             ov>                                                        To 
                                       mpi3-hybridpm at lists.mpi-forum.org   
             02/08/2011 05:38                                           cc 
             PM                        Douglas Miller/Rochester/IBM at IBMUS  
                                                                   Subject 
                                       Re: [Mpi3-hybridpm] New draft of    
                                       the EI chapter uploaded             
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           




Hi Doug,

On 02/07/2011 07:31 AM, Douglas Miller wrote:
> *balanced *- if set to true, the MPI implementation may assume that all
> threads in the team will actively participate by making MPI calls (or
> calling MPI_HELPER_TEAM_LEAVE). If false, the implementation must allow
> for some threads spending the bulk of their time performing computation
> or other non-MPI activities.

With respect to the first point, threads don't need to "actively
participate" from the user's perspective. The user just needs to be
aware that all MPI calls might synchronize all or some of the threads.

For the second point, I think this is incorrect. The threads spending
the bulk of their time in computation is fine even in "balanced" mode.
If the user's code is willing to accept the "synchronization overhead"
during the MPI call, that's fine. For example, we already discussed that
the following is valid:

MPI_Helper_team_join()
compute(random_time)
pthread_barrier();
if (thread == 0) {
   MPI_SOMETHING();
}
else {
   MPI_SOMETHING_ELSE();
}
MPI_Helper_team_leave()

But the following might be valid too (pthread_barrier is gone):

MPI_Helper_team_join()
compute(random_time)
if (thread == 0) {
   MPI_SOMETHING();
}
else {
   MPI_SOMETHING_ELSE();
}
MPI_Helper_team_leave()

In the second example, if the MPI implementation internally decides to
synchronize the threads, that's fine, since the application would have
done that anyway. This should be left to the user. The only thing that
will be invalid is:

MPI_Helper_team_join()
compute(random_time)
if (thread == 0) {
   MPI_SOMETHING();
}
pthread_barrier();
MPI_Helper_team_leave()

This is invalid because the MPI implementation is expecting to
synchronize with other threads during the MPI_SOMETHING call, but it
can't as all other threads are blocked in pthread_barrier().

> ...also, wouldn't setting 'singular' to true also imply/force 'balanced'
> to true? It almost seems like we have a three-state value rather than
> two booleans. From strict to weak: (singular), (!singular,balanced),
> (!singular,!balanced). I'm not clear just how much difference there is
> between singular (true) and balanced (true). Maybe we need only
'balanced'?

Yes, 'singular' is a superset of 'balanced'. I'll point that out. But as
we discussed in the telecon, 'singular' can give more benefit for
implementations as they know that other MPI calls will not be scheduled,
so it's easier to optimize.

  -- Pavan

--
Pavan Balaji
http://www.mcs.anl.gov/~balaji
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-hybridpm/attachments/20110209/432c67b6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-hybridpm/attachments/20110209/432c67b6/attachment-0003.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic25286.gif
Type: image/gif
Size: 1255 bytes
Desc: not available
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-hybridpm/attachments/20110209/432c67b6/attachment-0004.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ecblank.gif
Type: image/gif
Size: 45 bytes
Desc: not available
URL: <http://lists.mpi-forum.org/pipermail/mpiwg-hybridpm/attachments/20110209/432c67b6/attachment-0005.gif>