[Mpi3-tools] lightweight event log
Martin Schulz
schulzm at llnl.gov
Thu Jul 2 12:24:58 CDT 2009
Hi Marc-Andre,
On Jul 2, 2009, at 1:58 AM, Marc-Andre Hermanns wrote:
> Hi Bill,
>
>> The idea is that the implementation may be instrumented with the
>> ability to record timestamped events. Like the "variable" interface,
>> the idea here is to provide a way to expose whatever the
>> implementation is prepared to make available, such as information
>> that
>> the developers and tuners of that implementation have found valuable.
>> So in answer to your questoins:
>>
>> 1) what will be logged? Probably timestamps, some indication of
>> location, and some context (like message length). I would not expect
>> all routine parameters to be logged, though an implementation could
>> choose to do thiat
>
> Sometimes tools need all parameter information, though. Especially for
> higher level analysis, one might need more than the above information.
The idea is to log low-level events below the MPI call interface.
Hence, we
can't assume a fixed list of parameters anymore. Each implementation can
have its own set of information here. What Bill is suggesting is
basically
only a way for a tool to gather some context around the event and we
will try to suggest some commonly existing parameters, but as with the
performance variables, we don't want to demand any particular structure.
Note, though, that you (as a tool writer) can always combine this
logging
interface with a PMPI logger and can grab any cleanly defined logging
information from there as well.
>
>> 2) it is lightweight for two reasons: (1) the data logged may be as
>> small and simple as the implementation chooses and (2) the data may
>> be
>> logged in an internal, time and space efficient format. A separate
>> set of routines are provided to interpret the logged data. For
>> example, the timestamps may be stored in an internal format (such as
>> the bits from a clock register) and converted to seconds only when
>> the
>> user evaluates the logged data.
>
> Combining your answers to point 1 and 2, I envision to pre-construct
> memory-layouts (like or even with MPI datatypes) to the relevant
> information, so that a tool can basically first tell the
> implementation
> what it want to be logged, then define handles for the data layout,
> and
I am not sure if it's worth having the user pick what should be
recorded and
what not, since this will be very limited anyway.
> finally, retrieve the desired information through the defined
> handles at
> the location where they are needed.
>
> I admit I didn't evaluate the implementation overhead for something
> like
> that yet, and I am aware that flexibility usually comes with a
> runtime cost.
>
>> 3) A performance tool can examine the log files but (as part of the
>> light-weight part) there is no option to invoke a method or routine
>> at
>> every point a log record could be created.
>>
>> "Users" here means users with some degree of sophistication.
>> Basically, if you can use PAPI and really understand what the various
>> counters mean, then you should be able to use this interface.
>
> I am uncertain whether this logging facility would/should be
> complementing or competing with existing tracing tools.
>
> Some MPI implementations already have tracing facilities. To my
> knowledge mpich2 has MPE and OpenMPI has vampirtrace. Is this new
> logging facility to be provided by these or is it envisioned to be a
> new
> infrastructure. Would this not create competition between measurement
> infrastructures within MPI implementations?
No, it is certainly not intended to be an alternative to existing MPI
tracers.
This logging facility would run at a much lower level and only record
selected MPI internal events, mostly for debugging or very low-level
performance analysis in conjunction with a high-level tracer such
as VT.
>
> Martin mentioned a ring buffer to be queried? What happens when the
> buffer is full? If some logging information is lost, what can the user
> derive from an incomplete log?
The user will be able to size the buffer and query whether there was an
overflow, but if there is too much data, it will be lost. The user is
responsible for querying this on time. Anything else has the danger
of significantly increasing the critical path.
Martin
>
> Best regards,
> Marc-Andre
>
>> On Jun 29, 2009, at 7:42 AM, Marc-Andre Hermanns wrote:
>>
>>> Hi Martin,
>>>
>>> I am interested in how this logging facility is meant. Do you have a
>>> more detailed description of it somewhere?
>>>
>>> - What do you anticipate to be logged (despite of timestamps)?
>>> - What will make it lightweight?
>>> - Is there any way that a performance tool might hook into?
>>>
>>> "[...] Users can then read and evaluate this event log at a later
>>> time.
>>> [...]"
>>>
>>> Who are these users? Application developers or tool developers?
>>>
>>> Best regards,
>>> Marc-Andre
>>>
>>> Martin Schulz wrote:
>>>> Hi all,
>>>>
>>>> As discussed during the last meeting, our next MPI 3 tools WG
>>>> TelCon
>>>> is on 6/29
>>>> at the usual time (8am PDT, 11am EDT, 5pm MSZ). The dial-in numbers
>>>> are as
>>>> usual:
>>>>
>>>> * Toll Free Dial In Number for Offsite Participants:
>>>> 866-914-3976
>>>> * International Access/Caller Paid Dial In Number: 925-424-8105
>>>> * Dial In Number for LLNL Onsite Participants: 4-8105
>>>> * Access Security Code: 860609 #
>>>>
>>>> So far, the agenda is to talk about a proposal for a general
>>>> performance information
>>>> interface. More information on this proposal is at:
>>>>
>>>> https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MPI3Tools/pimpi
>>>>
>>>> If anybody wants to bring up another agenda item, please let me
>>>> know.
>>>>
>>>> Martin
>>>>
>>>> _______________________________________________________________________
>>>> Martin Schulz, schulzm at llnl.gov, http:// people.llnl.gov/schulz6
>>>> CASC @ Lawrence Livermore National Laboratory, Livermore, USA
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Mpi3-tools mailing list
>>>> Mpi3-tools at lists.mpi-forum.org
>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools
>>>
>>> --
>>> Marc-Andre Hermanns
>>> Juelich Supercomputing Centre
>>> Institute for Advanced Simulation
>>> Forschungszentrum Juelich GmbH
>>> D-52425 Juelich
>>> Germany
>>>
>>> Phone : +49-2461-61-2054
>>> Fax : +49-2461-61-6656
>>> eMail : m.a.hermanns at fz-juelich.de
>>> WWW : http://www.fz-juelich.de/jsc/
>>>
>>> JSC is the coordinator of the
>>> John von Neumann Institute for Computing
>>> and member of the
>>> Gauss Centre for Supercomputing
>>>
>>> Sitz der Gesellschaft: Juelich
>>> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
>>> Vorsitzende des Aufsichtsrats: MinDir'in Baerbel Brumme-Bothe
>>> Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),
>>> Dr. Ulrich Krafft (stellv. Vorsitzender),
>>> Prof. Dr. Harald Bolt,
>>> Prof. Dr. Sebastian M. Schmidt
>>> <smime.p7s><ATT00001.txt>
>>
>> William Gropp
>> Deputy Director for Research
>> Institute for Advanced Computing Applications and Technologies
>> Paul and Cynthia Saylor Professor of Computer Science
>> University of Illinois Urbana-Champaign
>>
>>
>>
>>
>> _______________________________________________
>> Mpi3-tools mailing list
>> Mpi3-tools at lists.mpi-forum.org
>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools
>
>
> --
> Marc-Andre Hermanns
> Juelich Supercomputing Centre
> Institute for Advanced Simulation
> Forschungszentrum Juelich GmbH
> D-52425 Juelich
> Germany
>
> Phone : +49-2461-61-2054
> Fax : +49-2461-61-6656
> eMail : m.a.hermanns at fz-juelich.de
> WWW : http://www.fz-juelich.de/jsc/
>
> JSC is the coordinator of the
> John von Neumann Institute for Computing
> and member of the
> Gauss Centre for Supercomputing
>
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzende des Aufsichtsrats: MinDir'in Baerbel Brumme-Bothe
> Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),
> Dr. Ulrich Krafft (stellv. Vorsitzender),
> Prof. Dr. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
> _______________________________________________
> Mpi3-tools mailing list
> Mpi3-tools at lists.mpi-forum.org
> http://*lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools
_______________________________________________________________________
Martin Schulz, schulzm at llnl.gov, http:// people.llnl.gov/schulz6
CASC @ Lawrence Livermore National Laboratory, Livermore, USA
More information about the mpiwg-tools
mailing list