[Mpi3-subsetting] MPI_ANY_SOURCE

[Richard Graham]

On 2/29/08 11:30 AM, "Supalov, Alexander" <alexander.supalov_at_[hidden]>
wrote:

> I see. Sorry for explaining the obvious. I guess the progress engine may
> take a hit every time there are either an MPI_ANY_SOURCE Recv or (thanks
> to Rich) multiple paths between the processes. Hence, all transfers are
> potentially affected.

With any_source the message can come from anyone, so the cost really depends
on the mpi's queuing strategy, so the actual cost is very implementation
specific.  What ever the cost is, there are more potential sources, so at
100k there are 100k potential sources.  The queuing could always have the
unexpected messages cached in a single queue, but then all matching would be
more expensive, vs. more of a hierarchical queue structure ....  For
expected messages there can also be an increase in matching costs, but again
this is implementation specific.

The other cost is that matching really has to be done at the destination -
just a practical need - try to cancel 100k posted receives, after one match
has been made, and make sure that only one proc has done the match.

> 
> Cancellation is a sticky matter. Some fabrics won't let you do this, so
> a cancel will always misfire.

Is this the case on the receive side ?  The cancellation that Richard is
mentioning is a receive side cancellation.  I don't remember a network with
this limitation, but I could very well be wrong on this one - I suppose it
can also depend on how you do the matching.

Rich

> 
> -----Original Message-----
> From: mpi3-subsetting-bounces_at_[hidden]
> [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of
> Richard Barrett
> Sent: Friday, February 29, 2008 5:02 PM
> To: mpi3-subsetting_at_[hidden]
> Subject: [Mpi3-subsetting] MPI_ANY_SOURCE
> 
> 
> 
>>> Now, to one of your questions. An MPI_ANY_SOURCE
> 
> Although I appreciate the discussion, my intent (uh-oh!) in bring this
> up to
> let you know I "accept" the problem, yet ask for the capability anyway,
> but
> in a manner that keeps it from presenting problems everywhere. Or maybe
> I'm
> under-estimating what I was once told: the use of MPI_ANY_SOURCE
> anywhere
> means it is a problem everywhere, ie in _every_ function involved in
> transmitting data?
> 
> If that is the case, but I still _really_ wanted to use -lmpi_subset, I
> could do this: suppose a pe knows it will receive data from m pes. It
> could
> post numpe non-blocking receives, complete m, discover who they're from,
> then cancel the rest. Now I'm thinking I've created a bigger problem:
> when
> running acros numpes=100k cores, but m is say 10. True?
> 
> Barring some sort of workaround, excluding codes that "need"
> MPI_ANY_SOURCE
> seems to meaningfully reduce the number of codes that could use
> -lmpi_subset.
> 
>> 3. I (basically) understand the adverse performance effects of
> allowing
> promiscuous receives (MPI_ANY_SOURCE). However, this is a powerful
> capability
> for many codes, and used only in moderation, eg for setting up
> communication
> requirements (such as communication partners in unstructured,
> semi-structured,
> and dynamic mesh computations). In this case the sender knows its
> partner, but
> the receiver does not. A reduction(sum) is used to let each process know
> the
> number of communication partners from which it will receive data, the
> process
> posts that many promiscuous receives, which when satisfied lets it from
> then on
> specify the sender. So would it be possible to include this capability
> in a
> separate function, say the blocking send/recv, but not allow it in the
> non-blocking version?
> 
> Richard