TheRock/docs/rfcs/RFC0010-Test-Scripts-Migration.md

author, created, modified, status, discussion

author	created	modified	status	discussion
Joseph Macaranas (jayhawk-commits)	2026-03-11	2026-03-18	in-progress	https://github.com/ROCm/TheRock/discussions/3964

Migrate Test Executable Scripts to Owning Project Repositories

This RFC proposes moving the test runner scripts in build_tools/github_actions/test_executable_scripts/ out of TheRock and into the rocm-systems and rocm-libraries repositories where the components they test actually live.

Motivation

TheRock currently hosts ~35 pytest runner scripts that exercise binaries from components whose source code resides in rocm-systems and rocm-libraries. This creates a structural mismatch:

• Discoverability: Developers working in rocm-libraries or rocm-systems must know to look in TheRock to find or modify their component's test runner.
• Ownership: Test scripts should be owned and reviewed by the same team/codeowners as the source they test. As-is, component teams must submit PRs to a different repo for test changes.
• Co-evolution: When a project's binary interface or test binary changes, the test script needs to change too. Keeping them in the same repo enables this as a single atomic commit.
• Scalability: As more components are added, test_executable_scripts/ grows without clear ownership boundaries.

Placement Options

Before migration begins, we need to agree on a canonical location for scripts within the owning repos. Three options are considered:

Option A — Co-located in each project subdirectory

Scripts live alongside the project source under a standard subdirectory:

rocm-systems/projects/amdsmi/test/therock/test_amdsmi.py
rocm-libraries/projects/hipblas/test/therock/test_hipblas.py

Pros: Tightest coupling to source; easy to find from within a project. Cons: No single location to discover all TheRock-level integration test runners across a repo; some projects already have a tests/ directory with different conventions.

Option B — Centralized test/therock/ per repo

All TheRock integration test runners for a given repo live under a single top-level directory:

rocm-systems/test/therock/test_amdsmi.py
rocm-systems/test/therock/test_rccl.py
rocm-libraries/test/therock/test_hipblas.py

Pros: Single location to discover and run all TheRock integration tests within a given repo. Consistent, predictable path for CI to reference. Cons: Slight separation from per-project source.

Option C — Reorganize within TheRock

Keep all scripts in TheRock but move them to a more intentional home such as tests/integration/, making the intent explicit without distributing them.

Pros: No cross-repo coordination required. Cons: Does not resolve the ownership or discoverability problems; component teams still submit test PRs to the wrong repo.

Chosen Approach

Initial migration: Option B — all scripts land in test/therock/ at the repo root of rocm-systems and rocm-libraries. This minimises the number of codeowner approvals required (one CODEOWNERS entry per repo rather than one per project subdirectory) and provides a single predictable location for CI to reference.

Long-term goal: Option A — once scripts are stable and owned by the right teams inside each super-repo, they will be moved to sit alongside test_categories.yaml in each project's own test subdirectory. That second move is a follow-on RFC.

Migration Plan

Step 1: Inventory and map all scripts ✅

Produce a complete mapping of every script to its owning repo and target path. See the Step 1 Inventory section below.

Step 2: Copy scripts into owning repos ✅

Manually copy scripts from TheRock at commit fc47892f1 into test/therock/ in each owning repo, opening a PR per repo to seed the canonical location.

• rocm-systems: PR #4190
• rocm-libraries: PR #5563

Note: the scripts are not usable from their new locations until Step 3 is complete and the updated submodule refs have been picked up by TheRock (Step 4).

Step 3: Finalize scripts in owning repos ✅

For each script, refine the seeded copy in the owning repo:

• Path resolution: Replace the hardcoded SCRIPT_DIR.parent.parent.parent fallback with Path(os.environ.get("THEROCK_DIR") or SCRIPT_DIR.parent.parent.parent).resolve() so scripts work both when the super-repo is a TheRock submodule and when run standalone with THEROCK_DIR pointing to a checked-out TheRock installation. Done in rocm-systems PR #4197 and rocm-libraries PR #5572.
• github_actions_api import: Scripts that import helpers via sys.path automatically benefit from the corrected THEROCK_DIR. test_rccl.py required an explicit header reorder so THEROCK_DIR is defined before the sys.path insertion that depends on it.
• CODEOWNERS: Add entries for test/therock/ in each repo.
• README: Add a note explaining how to run the scripts locally.

Step 4: Update CI workflow references

Blocked: This step requires the submodule refs for rocm-systems and rocm-libraries to be bumped in TheRock, incorporating the changes from Steps 2 and 3. Only once those updated refs are present in TheRock can the CI callers be safely switched to the new script locations. This will be done in a separate PR.

Update all pytest invocations that reference build_tools/github_actions/test_executable_scripts/test_X.py to point to the new canonical locations. Known callers:

• TheRock/build_tools/github_actions/fetch_test_configurations.py
• rocm-systems/.github/workflows/therock-rccl-test-packages-single-node.yml
• Any additional callers discovered during the Step 1 inventory.

Step 5: Validate end-to-end

Run a full CI pass with updated workflow references to confirm all tests execute correctly from their new locations before removing the old copies.

Step 6: Remove old scripts from TheRock

Once CI is green with all workflows pointing to canonical locations, delete TheRock/build_tools/github_actions/test_executable_scripts/ (or the subset of migrated files if any are intentionally retained).

Step 1 Inventory

The tables below map every file in test_executable_scripts/ to its owning repository and target path. Placement follows Option B — all scripts for a given repo land in a single test/therock/ directory at the repo root.

Naming convention

Scripts keep their existing test_<component>.py names. In the flat test/therock/ layout every script shares the same directory, so the component name in the filename is necessary for disambiguation — there is no redundancy.

Scripts are flagged with the github_actions_api helpers they import; see TheRock#3968 for the planned resolution of that dependency once scripts have landed in their owning repos.

fetch_test_configurations.py callers — indicates whether and how a script is invoked from CI (python, pytest, or commented-out). Scripts not listed there are currently manual-run only.

rocm-systems (8 scripts)

Script	Project	CI caller	github_actions_utils import	Target path
test_amdsmi.py	projects/amdsmi	manual only	—	test/therock/test_amdsmi.py
test_aqlprofile.py	projects/aqlprofile	python	—	test/therock/test_aqlprofile.py
test_hiptests.py	projects/hip-tests	python	is_asan	test/therock/test_hiptests.py
test_rccl.py	projects/rccl	pytest	get_visible_gpu_count	test/therock/test_rccl.py
test_rocprofiler_compute.py	projects/rocprofiler-compute	python	—	test/therock/test_rocprofiler_compute.py
test_rocprofiler_systems.py	projects/rocprofiler-systems	python	—	test/therock/test_rocprofiler_systems.py
test_rocr-debug-agent.py	projects/rocr-debug-agent	python	—	test/therock/test_rocr-debug-agent.py
test_rocrtst.py	projects/rocr-runtime	python	—	test/therock/test_rocrtst.py

rocm-libraries (22 scripts + hipdnn_install_tests/)

Script	Project	CI caller	github_actions_utils import	Target path
test_hipblas.py	projects/hipblas	python	is_asan	test/therock/test_hipblas.py
test_hipblaslt.py	projects/hipblaslt	python	is_asan	test/therock/test_hipblaslt.py
test_hipcub.py	projects/hipcub	python	—	test/therock/test_hipcub.py
test_hipdnn.py	projects/hipdnn	python	—	test/therock/test_hipdnn.py
test_hipdnn_install.py	projects/hipdnn	python	—	test/therock/test_hipdnn_install.py
test_hipdnn_samples.py	projects/hipdnn	python	—	test/therock/test_hipdnn_samples.py
test_hipfft.py	projects/hipfft	python	—	test/therock/test_hipfft.py
test_hiprand.py	projects/hiprand	python	—	test/therock/test_hiprand.py
test_hipsolver.py	projects/hipsolver	python	—	test/therock/test_hipsolver.py
test_hipsparse.py	projects/hipsparse	python	—	test/therock/test_hipsparse.py
test_hipsparselt.py	projects/hipsparselt	python	—	test/therock/test_hipsparselt.py
test_miopen.py	projects/miopen	manual only¹	—	test/therock/test_miopen.py
test_rocblas.py	projects/rocblas	python	is_asan	test/therock/test_rocblas.py
test_rocfft.py	projects/rocfft	python	—	test/therock/test_rocfft.py
test_rocprim.py	projects/rocprim	python	—	test/therock/test_rocprim.py
test_rocrand.py	projects/rocrand	python	—	test/therock/test_rocrand.py
test_rocroller.py	shared/rocroller	python	—	test/therock/test_rocroller.py
test_rocsolver.py	projects/rocsolver	python	—	test/therock/test_rocsolver.py
test_rocsparse.py	projects/rocsparse	python	—	test/therock/test_rocsparse.py
test_rocthrust.py	projects/rocthrust	python	—	test/therock/test_rocthrust.py
test_rocwmma.py	projects/rocwmma	python	—	test/therock/test_rocwmma.py
hipdnn_install_tests/	projects/hipdnn	(used by test_hipdnn_install.py)	—	test/therock/hipdnn_install_tests/

¹ The CI miopen job uses test_runner.py (a generic multi-component runner that stays in TheRock), not test_miopen.py directly. test_miopen.py is currently manual-run only.

Standalone submodules — out of scope for initial migration

These scripts test components that live outside both rocm-systems and rocm-libraries. Their destination repos and migration timelines need separate discussion.

Script	Source set	Owning repo (tentative)	CI caller	github_actions_utils import
test_rocgdb.py	debug-tools	ROCm/ROCgdb	python	—
test_libhipcxx_hipcc.py	math-libs	ROCm/libhipcxx	python	—
test_libhipcxx_hiprtc.py	math-libs	ROCm/libhipcxx	python	—

iree-libs / fusilli plugins — out of scope for initial migration

These scripts test Fusilli provider plugins whose build artifacts come from the iree-libs source set. Their natural home is a Fusilli or IREE repo.

Script	Artifact	CI caller	github_actions_utils import
test_fusilliprovider.py	fusilli_plugin_test_infra	commented out	—
test_hipblasltprovider.py	hipblaslt_plugin	python	—
test_miopenprovider.py	miopen_plugin	python	—

Stays in TheRock

Script	Reason
test_sanity.py	Tests TheRock-level ROCm installation sanity, not a specific component.
test_runner.py	Generic multi-component runner driven by TEST_COMPONENT env var; currently used as the CI entry point for the miopen job.

CI caller summary

The primary CI entry point for all scripts is build_tools/github_actions/fetch_test_configurations.py, which constructs test_script strings that reference paths relative to the repo root. That file and test_component.yml will be the main callers to update in Step 4.

The rocm-systems workflow therock-rccl-test-packages-single-node.yml also directly invokes test_rccl.py and will need a separate update.

github_actions_api dependency summary

Five scripts have a hard import dependency on helpers from TheRock/build_tools/github_actions/github_actions_api.py (renamed from github_actions_utils.py in TheRock @ 76ad42a63):

Helper	Used by
is_asan	test_hipblas.py, test_hipblaslt.py, test_hiptests.py, test_rocblas.py
get_visible_gpu_count	test_rccl.py

These scripts resolve the import via sys.path constructed from THEROCK_DIR, so the dependency is satisfied as long as THEROCK_DIR points to a valid TheRock checkout. See TheRock#3968 for longer-term cleanup (vendoring, packaging, or local runnability improvements).

Open Questions

• github_actions_api dependency: Resolved for Steps 2–3 — scripts use THEROCK_DIR to locate and import from TheRock's build_tools/github_actions/. Longer-term resolution (vendoring, packaging) tracked in TheRock#3968.
• hipdnn_install_tests/ subdirectory: Resolved — migrated alongside the three test_hipdnn_*.py scripts in rocm-libraries PR #5563.
• Step 4 dependency: CI caller updates in TheRock are blocked until the rocm-systems and rocm-libraries submodule refs (carrying Steps 2 & 3) are bumped in TheRock. That update and the CI caller changes will be a single PR.