[Mpi3-tools] draft MPIT hierarchy query function text

Sun Mar 7 10:43:45 CST 2010

Hi Dave,

On Mar 5, 2010, at 10:20 AM, William Gropp wrote:

> I'm in favor of this approach - the text based interface is most  
> useful to end users, not to the tool middleware that is most likely  
> to need this functionality.

I agree, the general idea that you have brought up to replace the  
string description
with a more explicit set of functions to query is certainly good and I  
think we should
go with it.

As for the concrete functions - your proposal basically turns the  
existing structure
around and decouples it from the actual performance variables; it  
becomes a
separate aspect of the interface and a tool writer would have to query  
all of this
separately. I am not sure, yet, what this really means for new tools  
and how much
more complicated this would make it. The interface would, however, be  
much
cleaner, which IMHO makes up for some extra work that has to be put  
into using it.

A few questions/comments (not just for Dave):
- MPI_TAXA is find with me, CLASSIFICATION is probably a bit too long,  
and there
was opposition at the Atlanta meeting against using group due the name  
clash
with MPI groups.
- Isn't the ALL_SETS redundant? We could just have a global set  
(MPI_SETWORLD)
and could use that to iterate over all sets.
- Returning all sets in a query operation might be a bit confusing and  
make it
difficult to extract the full hierarchy information. Should we just  
include the next
level down in the hierarchy tree? Same for querying up starting at the
variables? For sets this could also be a routine to get the parent  
node, which
wouldn't need an iterator.
- General, representing this as a forest of trees and variable can be  
a leaf
of any number of trees might be easier to understand than an arbitrary  
DAG.
- All sets are identified by character strings, which is the most  
natural, but could
also be cumbersome for repeated queries and the representation of the  
variable
hierarchy in a tree. It might be easier to represent sets with numbers  
or other
identifiers at some sort and at the end just provide one function that  
extracts
the name and description that a tool can call on demand when it actually
prints any of the text.
- When I originally thought about this, I wanted to overlay an  
additional structure
over the set functionality that indicates an second, independent  
neighbor or
before/after relationship. The idea is to allow tools to understand  
the order
of events in the MPI implementation, e.g., if you have two timers that  
are
used during a receive, this could tell you which one is used before  
the  other.
Also, it would allow the interface to express full event chains, as  
they were
prescribed in PERUSE, but without demanding them. In the original  
string-based
interface, this would have been too complicated, but here we could just
get another function that allows us to query preceding and subsequent
events. Would this be useful?
- For all iterators we need "has changed" functiosns to see if  
something has
changed, e.g., by activating new variables.
- Since this is a new and independent functions, should we also open  
this uo
to coinfiguration variables?

Martin

>
> Bill
>
> On Mar 5, 2010, at 11:07 AM, Dave Goodell wrote:
>
>> Feedback on this proposal would be appreciated.  Should I proceed or
>> will I be wasting my time?
>>
>> -Dave
>>
>> On Mar 2, 2010, at 6:09 PM, Dave Goodell wrote:
>>
>>> Enclosed is alternative text for the MPIT draft.  This text replaces
>>> the "Set and Hierarchy Information" \subsubsection in 1.3.5 in the
>>> current draft PDF (md5sum: 4fa7d0f3a25cabce8c2f5ef95bb41599).  My
>>> motivation for this alternative interface is to avoid pushing the
>>> burden of text parsing onto all tools that which to utilize
>>> hierarchy information.  As I mentioned at the last concall, I
>>> believe that few or no tools will want to work with the text format
>>> directly and will instead always access this information through a
>>> functional interface.
>>>
>>> The text below has many "...MORE TEXT HERE..." points indicating
>>> where the text should be expanded in a fashion consistent with the
>>> rest of the MPIT text.  I'm happy to finish writing this text but I
>>> wanted to get the rough shape of the proposal out for comment before
>>> I take the time to fill in the non-essential details.  If I've
>>> omitted too much in a particular area, please ask and I'll clarify
>>> what I had in mind.
>>>
>>> Discussion items:
>>>
>>> * function names: did we decide that iterative functions should be
>>> named _ITER or similar?
>>> * function names: most of these are listed as starting with
>>> MPIT_TAXA (as in taxonomy) in order to avoid verb/noun confusion
>>> with the word "SET".  I am open to alternative names, such as
>>> MPIT_TAXONOMY, MPIT_HIERARCHY, MPIT_CLASSIFICATION, MPIT_GROUP.
>>> (the first three are long, the last has potential for confusion with
>>> MPI_Group)  Maybe MPIT_VARGROUP?
>>> * iteration order: does the topological order requirement help
>>> client code or is it just additional burden for MPI implementors?
>>> * I've eliminated the "pretty names" for sets.  I'm assuming that
>>> the uniquely identifying name is sufficient and that anything that
>>> needs to be spelled out more explicitly for humans should go into
>>> the description information.  If anyone disagrees with this, I can
>>> add the pretty names back in.
>>> * I think it also makes sense to move the rewritten \subsubsection
>>> to after the "Query and Handle Functions" \subsubsection.  It is
>>> easier to talk about grouping performance variables once the reader
>>> knows a bit more about what performance variables actually are.
>>>
>>> -Dave
>>>
>>> ------8<------
>>> Performance Variable Taxonomic Information
>>>
>>> MPI implementations can optionally provide information that
>>> describes the relationship of performance variables to each other.
>>> For this, an MPI implementation can define names that represent sets
>>> of variables and then associate each performance variable with zero
>>> or more sets.  Sets may contain zero or more performance variables
>>> and zero or more other sets.  Sets may not contain themselves either
>>> directly or indirectly.  Said another way, these sets and
>>> performance variables form a directed acyclic graph (DAG).  This
>>> information is accessible via several interrogative routines.
>>>
>>> MPIT_TAXA_QUERY_ALL_SETS(iterator, name, namelen)
>>>
>>> INOUT iterator
>>> OUT   name
>>> OUT   namelen
>>>
>>> Iterate over all sets in topological order.  A unique identifying
>>> name for the set is returned in "name" and "namelen" is set to the
>>> number of characters written.  The value of "namelen" cannot be
>>> larger than MPIT_MAX_SET_NAME-1.
>>> ...MORE TEXT HERE...
>>>
>>> MPIT_TAXA_QUERY_VARIABLE_SETS(iterator, varname, name, namelen)
>>>
>>> INOUT iterator
>>> IN    varname
>>> OUT   name
>>> OUT   namelen
>>>
>>> Iterate over all sets that contain the performance variable
>>> identified by "varname".
>>> ...MORE TEXT HERE...
>>>
>>> MPIT_TAXA_QUERY_SET_SETS(iterator, setname, name, namelen)
>>>
>>> INOUT iterator
>>> IN    setname
>>> OUT   name
>>> OUT   namelen
>>>
>>> Iterate over all sets that contained in the set identified by
>>> "setname".
>>> ...MORE TEXT HERE...
>>>
>>> MPIT_TAXA_QUERY_SET_VARIABLES(iterator, setname, name, namelen)
>>>
>>> Iterate over all variables contained in the set identified by
>>> "setname".
>>> ...MORE TEXT HERE...
>>>
>>> MPIT_TAXA_SET_DESCRIBE(name, desc, desclen)
>>>
>>> IN    name
>>> OUT   desc
>>> OUT   desclen
>>>
>>> Retrieve the description for the set identified by "name".
>>> ...MORE TEXT HERE...
>>> ------8<------
>>>
>>> Also, delete the "sets" and "setslen" parameters to
>>> MPIT_PERFORMANCE_INFO.  Delete the paragraphs contained in lines
>>> 7-16 of page 19 ("The argument sets ... setslen must be set to
>>> zero.").
>>>
>>> _______________________________________________
>>> Mpi3-tools mailing list
>>> Mpi3-tools at lists.mpi-forum.org
>>> http://*lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools
>>
>> _______________________________________________
>> Mpi3-tools mailing list
>> Mpi3-tools at lists.mpi-forum.org
>> http://*lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools
>
> William Gropp
> Deputy Director for Research
> Institute for Advanced Computing Applications and Technologies
> Paul and Cynthia Saylor Professor of Computer Science
> University of Illinois Urbana-Champaign
>
>
>
>
> _______________________________________________
> Mpi3-tools mailing list
> Mpi3-tools at lists.mpi-forum.org
> http://*lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools
>

________________________________________________________________________
Martin Schulz, schulzm at llnl.gov, http://people.llnl.gov/schulzm
CASC @ Lawrence Livermore National Laboratory, Livermore, USA