derek.holloway
7502bf64a4
CodeQL / Analyze (actions) (push) Canceled after 0s
CodeQL / Analyze (python) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Multi-Arch CI / setup (push) Canceled after 0s
Publish build_manylinux_x86_64 images / publish_build_manylinux_x86_64 (push) Canceled after 0s
Setup (Multi-Arch) / setup (push) Canceled after 0s
Multi-Arch CI / Linux::${{ fromJSON(needs.setup.outputs.linux_build_config || '{}').build_variant_label || 'skip' }} (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish no_rocm_image_ubi10 image / publish_no_rocm_image_ubi10 (push) Canceled after 0s
Multi-Arch CI / Windows::${{ fromJSON(needs.setup.outputs.windows_build_config || '{}').build_variant_label || 'skip' }} (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Multi-Arch CI / Manifest Diff (push) Canceled after 0s
TheRock Manifest Diff Report / Generate Manifest Diff Report (push) Canceled after 0s
Multi-Arch CI / CI Summary (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04 (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04_ocl_rt images / publish_no_rocm_image_ubuntu24_04_ocl_rt (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_rocgdb (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04_rocgdb images / publish_no_rocm_image_ubuntu24_04_rocgdb (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_ocl_rt (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Unit Tests / Unit Tests :: ubuntu-24.04 (push) Canceled after 0s
Unit Tests / Unit Tests :: windows-2022 (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_media (push) Canceled after 0s
Unit Tests / Unit Tests Summary (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_openmpi (push) Canceled after 0s
SDMA Test
Overview
This example benchmarks SDMA (System DMA) engine performance by measuring bandwidth for Host-to-Device (H2D), Device-to-Device (D2D), and Device-to-Host (D2H) memory transfers. It runs configurable iterations of async copy operations using pinned host memory and HIP streams, reports per-transfer-type bandwidth in GB/s, and verifies data integrity via memcmp. An infinite-loop mode allows sustained profiling of SDMA utilization over extended periods.
Source Files
sdma-test.cpp- Implements transfer benchmarking withhipMemcpyAsync, CLI argument parsing with size suffix support (K/M/G), bandwidth calculation, and signal handling for graceful shutdown in infinite mode.
Prerequisites
- CMake 3.25+
- HIP runtime and
hipcccompiler
Building
Standalone build:
cmake -B <build_dir> -S <project_root>/examples/sdma-test -DCMAKE_PREFIX_PATH=/opt/rocm
cmake --build <build_dir>
As part of the examples suite:
cmake -B <build_dir> -S <project_root>/examples/ -DCMAKE_PREFIX_PATH=/opt/rocm
cmake --build <build_dir> --target sdma-test
Running
# Default: 512 MB, 10 iterations, 10 copies/iteration, device 0
./sdma-test
# Custom: 1 GB transfers, 5 iterations
./sdma-test -s 1024 -n 5
# Infinite mode (Ctrl+C to stop)
./sdma-test -s 256 -n 0
Flags:
| Flag | Description | Default |
|---|---|---|
-s, --size |
Transfer size in MB (supports K/M/G suffixes) | 512 |
-n, --iterations |
Number of iterations (0 = infinite) | 10 |
-c, --copies |
Number of copies per iteration | 10 |
-d, --device |
GPU device ID | 0 |
Profiling with rocprofiler-systems
rocprof-sys-run -- ./sdma-test -s 256 -n 5
Recommended Configuration
| Variable | Value | Purpose |
|---|---|---|
ROCPROFSYS_ROCM_DOMAINS |
hip_runtime_api,kernel_dispatch,memory_copy |
Trace HIP API and memory copy operations |
ROCPROFSYS_ROCM_EVENTS |
SQ_WAVES,GRBM_COUNT |
Sample GPU hardware counters |
ROCPROFSYS_AMD_SMI_METRICS |
sdma_usage |
Collect SDMA engine usage via AMD-SMI |
ROCPROFSYS_TRACE |
true |
Generate Perfetto trace for timeline analysis |
To capture SDMA-specific metrics:
rocprof-sys-run \
-e ROCPROFSYS_ROCM_DOMAINS=hip_runtime_api,memory_copy \
-e ROCPROFSYS_AMD_SMI_METRICS=sdma_usage \
-e ROCPROFSYS_TRACE=true \
-- ./sdma-test -s 512 -n 5