timemory 3.3.0
Modular C++ Toolkit for Performance Analysis and Logging. Profiling API and Tools for C, C++, CUDA, Fortran, and Python. The C++ template API is essentially a framework to creating tools: it is designed to provide a unifying interface for recording various performance measurements alongside data logging and interfaces to other tools.
|
timemory on GitHub (Source code)
timemory General Documentation (ReadTheDocs)
timemory Source Code Documentation (Doxygen)
timemory Testing Dashboard (CDash)
GitHub | git clone https://github.com/NERSC/timemory.git |
PyPi | pip install timemory |
Spack | spack install timemory |
conda-forge | conda install -c conda-forge timemory |
The goal of timemory is to create an open-source performance measurement and analyis package with modular and reusable components which can be used to adapt to any existing C/C++ performance measurement and analysis API and is arbitrarily extendable by users within their application. Timemory is not just another profiling tool, it is a profling toolkit which streamlines building custom profiling tools through modularity and then utilizes the toolkit to provides several pre-built tools.
In other words, timemory provides many pre-built tools, libraries, and interfaces but, due to it's modularity, codes can re-use only individual pieces – such as the classes for measuring different timing intervals, memory usage, and hardware counters – without the timemory "runtime management".
Timemory uses a standard CMake installation. Several installation examples can be found in the Wiki. See the installation documentation for detailed information on the CMake options.
The full documentation is available at timemory.readthedocs.io. Detailed source documentation is provided in the doygen section of the full documentation. Tutorials are available in the github.com/NERSC/timemory-tutorials.
The primary objective of the timemory is the development of a common framework for binding together software monitoring code (i.e. performance analysis, debugging, logging) into a compact and highly-efficient interface.
Timemory arose out of the need for a universal adapator kit for the various APIs provided several existing tools and a straight-forward and intuitive method for creating new tools. Timemory makes it possible to bundle together deterministic performance measurements, statistical performance measurements (i.e. sampling), debug messages, data logging, and data validation into the same interface for custom application-specific software monitoring interfaces, easily building tools like time
, netstat
, instrumentation profilers, sampling profilers, and writing implementations for MPI-P, MPI-T, OMPT, KokkosP, etc. Furthermore, timemory can forward its markers to several third-party profilers such as LIKWID, Caliper, TAU, gperftools, Perfetto, VTune, Allinea-MAP, CrayPAT, Nsight-Systems, Nsight-Compute, and NVProf.
Timemory provides a front-end C/C++/Fortran API and Python API which allows arbitrary selection of 50+ different components from timers to hardware counters to interfaces with third-party tools. This is all built generically from the toolkit API with type-safe bundles of tools such as: component_tuple<wall_clock, papi_vector, nvtx_marker, user_bundle>
where wall_clock
is a wall-clock timer, papi_vector
is a handle for hardware counters, nvxt_marker
creates notations in the NVIDIA CUDA profilers, and user_bundle
is a generic component which downstream users can insert more components into at runtime.
Performance measurement components written with timemory are arbitrarily scalable up to any number of threads and processes and fully support intermixing different measurements at different locations within the program – this uniquely enables timemory to be deployed to collect performance data at scale in HPC because highly detailed collection can occur at specific locations within the program where ubiquitous collection would simulatenously degrade performance significantly and require a prohibitive amount of memory.
Timemory can be used as a backend to bundle instrumentation and sampling tools together, support serialization to JSON/XML, and provide statistics among other uses. It can also be utilized as a front-end to invoke custom instrumentation and sampling tools. Timemory uses the abstract term "component" for a structure which encapsulates some performance analysis operation. The structure might encapsulate function calls to another tool, record timestamps for timing, log values provided by the application, provide a operator for replacing a function in the code dynamically, audit the incoming arguments and/or outgoing return value from function, or just provide stubs which can be overloaded by the linker.
The native output format of timemory is JSON and text; other output formats such as XML are also supported. The text format is intended to be human readable. The JSON data is intended for analysis and comes in two flavors: hierarchical and flat. Basic plotting capabilities are available via timemory-plotting
but users are highly encouraged to use hatchet for analyzing the heirarchical JSON data in pandas dataframes. Hatchet supports filtering, unions, addition, subtractions, output to dot
and flamegraph formats, and an interactive Jupyter notebook. At present, timemory supports 45+ metric types for analysis in Hatchet.
There are 4 primary categories in timemory: components, operations, bundlers, and storage. Components provide the specifics of how to perform a particular behavior, operations provide the scaffold for requesting that a component perform an operation in complex scenarios, bundlers group components into a single generic handle, and storage manages data collection over the lifetime of the application. When all four categories are combined, timemory effectively resembles a standard performance analysis tool which passively collects data and provides reports and analysis at the termination of the application. Timemory, however, makes it very easy to subtract storage from the equation and, in doing so, transforms timemory into a toolkit for customized data collection.
tim::component::wall_clock
: a simple wall-clock timertim::component::vtune_profiler
: a simple component which turns the VTune Profiler on and off (when VTune is actively profiling application)tim::component::data_tracker_integer
: associates an integer values with a label as the application executes (e.g. number of loop iterations used somewhere)tim::component::papi_vector
: uses the PAPI library to collect hardware-counters valuestim::component::user_bundle
: encapsulates an array of components which the user can dynamically manipulate during runtimetim::operation::start<wall_clock>
will attempt to call the start()
member function on a wall_clock
component instancewall_clock
have a store(int)
function? store()
?)tim::operation::start
: instruct a component to start collectiontim::operation::sample
: instruct a component to take individual measurementtim::operation::derive
: extra data from other components if it is availableauto_tuple
starts all components when constructed and stops all components when destructed whereas component_tuple
requires an explicit startcomponent_tuple
allocates all components on the stack and components are "always on" whereas component_list
allocates components on the heap and thus components can be activated/deactivated at runtimelightweight_tuple
does not implicitly perform any expensive actions, such as call-stack tracking in "Storage"NOTE:
tim::lightweight_tuple
is the recommended bundle for those seeking to use timemory as a toolkit for implementing custom tools and interfaces
timemory-avail
is provided as a stand-alone Python classtime
command-line tool that includes additional information on memory usage, context switches, and hardware countersnvidia-smi
timemory-python-profiler
from timemory.profiler import Profile
timemory-python-trace
from timemory.trace import Trace
timemory-python-line-profiler
from timemory.line_profiler import LineProfiler
Various macros are defined for C in source/timemory/compat/timemory_c.h and source/timemory/variadic/macros.hpp. Numerous samples of their usage can be found in the examples.
Timemory originated as a very simple tool for recording timing and memory measurements (hence the name) in C, C++, and Python and only supported three modes prior to the 3.0.0 release: a fixed set of timers, a pair of memory measurements, and the combination of the two. Prior to the 3.0.0 release, timemory was almost completely rewritten from scratch with the sole exceptions of some C/C++ macro, e.g. TIMEMORY_AUTO_TIMER
, and some Python decorators and context-manager, e.g. timemory.util.auto_timer
, whose behavior were able to be fully replicated in the new release. Thus, while it may appear that timemory is a mature project at v3.0+, it is essentially still in it's first major release.
To reference timemory in a publication, please cite the following paper:
For more information, refer to the documentation.