derek.holloway
7502bf64a4
CodeQL / Analyze (actions) (push) Canceled after 0s
CodeQL / Analyze (python) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Multi-Arch CI / setup (push) Canceled after 0s
Publish build_manylinux_x86_64 images / publish_build_manylinux_x86_64 (push) Canceled after 0s
Setup (Multi-Arch) / setup (push) Canceled after 0s
Multi-Arch CI / Linux::${{ fromJSON(needs.setup.outputs.linux_build_config || '{}').build_variant_label || 'skip' }} (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish no_rocm_image_ubi10 image / publish_no_rocm_image_ubi10 (push) Canceled after 0s
Multi-Arch CI / Windows::${{ fromJSON(needs.setup.outputs.windows_build_config || '{}').build_variant_label || 'skip' }} (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Multi-Arch CI / Manifest Diff (push) Canceled after 0s
TheRock Manifest Diff Report / Generate Manifest Diff Report (push) Canceled after 0s
Multi-Arch CI / CI Summary (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04 (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04_ocl_rt images / publish_no_rocm_image_ubuntu24_04_ocl_rt (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_rocgdb (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04_rocgdb images / publish_no_rocm_image_ubuntu24_04_rocgdb (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_ocl_rt (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Unit Tests / Unit Tests :: ubuntu-24.04 (push) Canceled after 0s
Unit Tests / Unit Tests :: windows-2022 (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_media (push) Canceled after 0s
Unit Tests / Unit Tests Summary (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_openmpi (push) Canceled after 0s
73 lines
3.4 KiB
Markdown
73 lines
3.4 KiB
Markdown
# Cotenant benchmarking
|
|
|
|
Run `hipblaslt-bench` (or any command) while a fixed number of CUs are occupied
|
|
by a background "cotenant" kernel, to measure GEMM performance under CU
|
|
contention.
|
|
|
|
Linux only: the kernel uses POSIX APIs (`unistd.h`, `pause()`, `usleep()`,
|
|
`getpid()`) and is not built or installed on Windows.
|
|
|
|
```bash
|
|
hipblaslt-cotenant --cus 64 -- hipblaslt-bench -m 4096 -n 4096 -k 4096
|
|
```
|
|
|
|
All options must come **before** `--`; everything after it is the command to run
|
|
(the kernel binary is found automatically, so no path is needed).
|
|
|
|
## Usage
|
|
|
|
`hipblaslt-cotenant` and its kernel `hipblaslt-cotenant-kernel` install into
|
|
`bin/` next to `hipblaslt-bench`:
|
|
|
|
- **Installed:** `hipblaslt-cotenant --cus 64 -- hipblaslt-bench ...` (on `PATH`).
|
|
- **Build tree:** `<build>/clients/hipblaslt-cotenant --cus 64 -- ...`.
|
|
- **Source checkout:** `clients/scripts/cotenant/hipblaslt-cotenant --cus 64 -- ...`
|
|
builds the kernel on first use into `~/.cache/hipblaslt/cotenant`. Point
|
|
`--binary` at a prebuilt kernel to skip that.
|
|
|
|
`hipblaslt-bench` itself needs the ROCm runtime libraries on the loader path; if
|
|
it fails to start with a `libomp.so` error, export
|
|
`LD_LIBRARY_PATH=/opt/rocm/lib:/opt/rocm/llvm/lib`.
|
|
|
|
## How it works
|
|
|
|
`hipblaslt-cotenant-kernel` is a persistent, compute-free kernel. It reserves the
|
|
**entire** per-CU LDS as dynamic shared memory for a single block, sized from a
|
|
runtime device query (no per-architecture constants). This does two things at
|
|
once: only one cotenant block fits per CU (so a grid of `N` workgroups occupies
|
|
exactly `N` CUs), and **zero LDS remains** for anything else — so the hardware
|
|
cannot co-schedule a benchmarked-kernel workgroup onto a cotenant-occupied CU
|
|
either. `N` is therefore an exact count of CUs *removed* from the benchmarked
|
|
command, not merely "one competing block per CU".
|
|
|
|
Reserving only *half* the LDS still pins the cotenant to one block per CU, but
|
|
leaves ~half the LDS free — enough for a low-LDS kernel (e.g. a shallow-`K` GEMM)
|
|
to co-reside on a cotenant CU and run unaffected, silently under-reporting
|
|
contention. Full-LDS reservation closes that gap; on gfx942/gfx950 it is
|
|
permitted because a single block may take all of the per-CU LDS.
|
|
|
|
Written for mi300 and mi350 architectures; it might not work
|
|
correctly on other targets.
|
|
|
|
To confirm the kernel is actually executing (not just that a GPU context
|
|
exists), each workgroup increments a system-scope atomic counter in host-pinned
|
|
memory at entry; the host waits until all `N` have reported, then logs `READY`.
|
|
This is what the launcher waits for, so the command starts against full
|
|
residency without polling driver internals or guessing a settle time.
|
|
|
|
`hipblaslt-cotenant`:
|
|
|
|
1. builds the kernel on first use (arch auto-detected via `rocminfo`, override
|
|
with `--arch`; compiler defaults to `hipcc`, override with `HIPCC=...`),
|
|
2. launches it on `--cus` CUs and waits for its `READY` marker,
|
|
3. runs the command after `--` under that contention,
|
|
4. kills the cotenant when the command exits or the script is interrupted.
|
|
|
|
Pass `--cus 0` to run the command with no cotenant at all — the uncontended
|
|
baseline. Otherwise `--cus` must be at least 1 and less than the device CU count
|
|
(reported by `rocminfo`); occupying every CU would leave none for the benchmark,
|
|
so that is rejected.
|
|
|
|
Useful flags: `--device N` (sets `HIP_VISIBLE_DEVICES`), `--wait` (max seconds to
|
|
wait for `READY`), `--grace` (extra settle time after residency, default 0).
|