TheRock/rocm-systems/projects/rocprofiler-systems/external/timemory/docs/tools.md

Tools

.. toctree::
   :glob:
   :maxdepth: 1

   tools/timemory-timem/README
   tools/timemory-avail/README
   tools/timemory-run/README
   tools/timemory-stubs/README
   tools/timemory-jump/README
   tools/timemory-mpip/README
   tools/timemory-ncclp/README
   tools/timemory-mallocp/README
   tools/timemory-ompt/README
   tools/timemory-nvml/README
   tools/timemory-compiler-instrument/README
   tools/kokkos-connector/README

This section covers the executables and libraries that are distributed as part of the library.

• Executables
  • timem
    • Drop-in replacement for time (UNIX)
    • For MPI applications, use timem-mpi
  • timemory-avail
    • Use this executable to query available components, available settings, and available hardware counters
  • timemory-run
    • Use this executable (Linux-only) for dynamic instrumentation
  • timemory-nvml
    • Data collection similar to nvidia-smi
• Libraries
  • timemory-stubs
    • Provides timemory instrumentation stubs for dynamic library preloading
  • timemory-jump
    • Provides timemory instrumentation via dlopen and dlsym
  • timemory-mpip
    • Provide MPI profiling via GOTCHA
  • timemory-ncclp
    • Provide NCCL profiling via GOTCHA
  • timemory-mallocp
    • Records amount of memory allocated and freed on the CPU and GPU via GOTCHA wrappers around malloc, free, cudaMalloc, etc.
  • timemory-ompt
    • Provide OpenMP profiling via OMPT (OpenMP Tools)
  • timemory-compiler-instrument
    • Automatically instrument C and C++ source code via -finstrument-functions compiler flag
  • Kokkos Connectors
    • Libraries for Kokkos profiling

Profiling with timemory

Instrumenting Existing Binary

Timemory includes the timemory-run as a full profiler for Linux systems. This executable supports dynamic instrumentation (instrumenting at the target applicaiton's runtime), attaching to a running process, and binary re-writing (creating a new instrumented binary). The instrumented applications support flat-profiling, call-stack profiling, and timeline profiling and can be configured to use any of the components timemory provides or, with a little work, can also be used to instrument custom components defined by the user. It is highly recommended for custom tools targetting specific functions to use the combination of GOTCHA and the dynamic instrumentation. Using the GOTCHA extensions for profiling specific functions enables creating components which replace the function or audit the incoming arguments and return values for the functions and the dynamic instrumentation makes it easy to inject using the GOTCHA wrappers into an executable or library.

Instrumentation Binary at Compile-Time

Timemory includes support for compile-time instrumentation via the timemory-compiler-instrument library. This form of instrumentation is available on UNIX systems and compilers which support the -finstrument-functions compiler flag. It is generally recommended that the compiler instrumentation be propagated all the way to the compilation of the executable (as opposed to only the libraries called by the executable) since finalization is typically triggered when the executable returns from main. Failure to insert instrumentation around main may or may not result in segmentation faults.