TheRock/rocm-systems/projects/rocprofiler-systems/external/timemory/examples/ex-gpu

ex-gpu

Build

See examples. Additionally requires -DTIMEMORY_USE_CUDA=ON or -DTIMEMORY_USE_HIP=ON option enabled in cmake.

ex_gpu_event

This example executes a set of kernels using either CUDA or HIP and measures the kernel execution time via either cudaEvent_t or hipEvent_t. Aliases to the different backends are defined as follow:

#if defined(TIMEMORY_USE_CUDA)
using gpu_marker = tim::component::nvtx_marker;
using gpu_event  = tim::component::cuda_event;
#else
using gpu_marker = tim::component::roctx_marker;
using gpu_event  = tim::component::hip_event;
#endif

Kernel Instrumentation

This example contains two example implementations of using timemory to instrument GPU kernels. In the each example, the kernels have an extra parameter of type device_data which is defined within the gpu_device_timer component. Device timing is initialized via calling gpu_device_timer::start(device::gpu{}, size_t threads_per_block). As usual, you can pass these parameters through the component bundle start member function. The gpu_device_timer start member function allocates memory on the GPU for the device_data object to update. Only when stop() is called on gpu_device_timer (or it's bundle) will the data be copied back over to the host and updated.

Overview

namespace comp = tim::component;

using bundle_t = tim::component_tuple<comp::wall_clock, comp::gpu_device_timer>;

__global__
void bar(...,
         comp::gpu_device_timer::device_data _timer)
{
    _timer.start();
    // ... etc. ...
    _timer.stop();
}

// "n" is the number of blocks. Threads per block is 1024 here but could also
// be variable
void foo(size_t n)
{
    bundle_t _obj{ __FUNCTION__ };

    _obj.start(tim::device::gpu{}, 1024);

    // get_reference<T>() is used here to guarantee that gpu_device_timer is available
    bar<<<n, 1024>>>(...,
                     _obj.get_reference<comp::gpu_device_timer>().get_device_data());

    _obj.stop();
}

device_data uses clock64() on the device to collect timestamps. In both of the kernel instrumentation examples, it's definition is similar to the following:

struct device_data
{
    __device__ void start()
    {
        if(!m_data)
            return;
        m_buff = clock64();
    }

    __device__ void stop()
    {
        if(!m_data)
            return;
        auto _idx = (blockDim.x * blockDim.y * blockIdx.z) +
                    (blockDim.x * blockIdx.y) +
                    threadIdx.x;
        atomicAdd(&(m_data[_idx]), static_cast<int>(clock64() - m_buff));
    }

    long long int m_buff = 0;
    unsigned int* m_data = nullptr;
};

ex_kernel_instrument

This is one theoretical implementation of using timemory to instrument GPU kernels. In the gpu_device_timer component implementation, the data is directly stored in the gpu_device_timer instance.

ex_kernel_instrument_v2

This is another theoretical implementation of using timemory to instrument GPU kernels. The primary difference between this instance and ex_kernel_instrument is that the data is not stored directly in gpu_device_timer but instead, it uses the data_tracker to improve the data storage and output capabilities.

ex_kernel_instrument_v3

This is the same as ex_kernel_instrument_v2 but does not required modifying the kernel parameters.

Version 3 (ex_kernel_instrument_v3) is the recommended style.