TheRock/dockerfiles/README.md

# Dockerfiles for TheRock

This directory is home to the Dockerfiles we use as part of ROCm development, as
well as supporting scripts.

## Available Dockerfiles

### `build_manylinux_*.Dockerfile`

| Source .Dockerfile                                                        | Published package                                                             |
| ------------------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| [`build_manylinux_x86_x64.Dockerfile`](build_manylinux_x86_64.Dockerfile) | https://github.com/ROCm/TheRock/pkgs/container/therock_build_manylinux_x86_64 |

These Dockerfiles are used to build ROCm, PyTorch, and other packages for
release across a wide variety of Linux distributions. They are derived from
[manylinux](https://github.com/pypa/manylinux) Docker images which are based on
older RHEL (Red Hat Enterprise Linux) derivatives, provide a minimal set of
development libraries, and support building binaries compatible with most Linux
distributions that use glibc 2.39 or greater.

Requirements for these files:

- Tools like gcc, CMake, ninja, etc. should generally be set to the minimum
  versions of those tools that we support.
- These Dockerfiles deliberately exclude development _packages_ for most
  libraries because their presence would create hidden dependencies on specific
  library versions that may not be binary-compatible across distributions.
  Instead, TheRock vendors all necessary dependencies and builds them from
  source with specific modifications to ensure they cannot conflict with system
  libraries.
  - Certain development _tools_ like texinfo and DVC are acceptable to install
    in the base build images, so long as they don't leak non-portable code into
    binaries.
  - See also
    [gist - "TheRock Dependency Auditing and Vendoring Guide"](https://gist.github.com/stellaraccident/866acb0cae35e9f59c0c67d2fe3773c1).
    <!-- TODO: Merge that gist into this repository somewhere -->

### `no_rocm_image_*.Dockerfile`

| Source .Dockerfile                                                             | Published package                                                        |
| ------------------------------------------------------------------------------ | ------------------------------------------------------------------------ |
| [`no_rocm_image_ubuntu24_04.Dockerfile`](no_rocm_image_ubuntu24_04.Dockerfile) | https://github.com/ROCm/TheRock/pkgs/container/no_rocm_image_ubuntu24_04 |

These Dockerfiles are used to test ROCm, PyTorch, and other packages. They
are derived from standard OS images (e.g. `FROM ubuntu:24.04`) and contain a
minimal extra set of development tools and environment settings necessary to
run project tests on GPU test machines. These images do NOT include a ROCm
install, as they are used from test workflows that begin by installing ROCm
(or other) packages produced by CI/CD workflows.

Requirements for these files:

- No system install of ROCm - workflows using these images are expected to
  install ROCm after starting a container.
- Minimal other system dependencies, especially development packages, for many
  of the same reasons as for the "build" Dockerfiles above. Test environments
  should be similar to user/production environments, so any extra dependencies
  that are added to these test Dockerfiles may hide real issues.
  - Some extra tools like `lit` for LLVM testing are acceptable and should be
    evaluated on a case-by-case basis.

This test dockerfile is used by various CI/CD workflows, such as:

- [`.github/workflows/test_component.yml`](/.github/workflows/test_component.yml)
- [`.github/workflows/test_pytorch_wheels.yml`](/.github/workflows/test_pytorch_wheels.yml)

To run this test dockerfile yourself on a machine with a compatible GPU:

```bash
sudo docker run -it \
  --device=/dev/kfd --device=/dev/dri \
  --ipc=host --group-add=video --group-add=render --group-add=110 \
  ghcr.io/rocm/no_rocm_image_ubuntu24_04@latest
```

> [!TIP]
> We recommend creating a Python venv to install Python packages while using
> this image:
>
> ```bash
> sudo apt update && sudo apt install python3-venv -y
> python3 -m venv .venv && source .venv/bin/activate
>
> # Then install packages. For example, with nightly rocm packages for gfx94X:
> pip install --index-url=https://rocm.nightlies.amd.com/v2/gfx94X-dcgpu 'rocm[libraries,devel]'
> ```

| Source .Dockerfile                                                                           | Published package                                                               |
| -------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
| [`no_rocm_image_ubuntu24_04_rocgdb.Dockerfile`](no_rocm_image_ubuntu24_04_rocgdb.Dockerfile) | https://github.com/ROCm/TheRock/pkgs/container/no_rocm_image_ubuntu24_04_rocgdb |

Extended version of no_rocm_image_ubuntu24_04.Dockerfile, containing additional
packages and tools required for validation of rocgdb.
This includes dejagnu, make, gcc, g++ and gfortran.

| Source .Dockerfile                                                                           | Published package                                                               |
| -------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
| [`no_rocm_image_ubuntu24_04_ocl_rt.Dockerfile`](no_rocm_image_ubuntu24_04_ocl_rt.Dockerfile) | https://github.com/ROCm/TheRock/pkgs/container/no_rocm_image_ubuntu24_04_ocl_rt |

Used in ocltst execution. It installs OCL ICD package required
by ocltst

| Source .Dockerfile                                                                             | Published package                                                                |
| ---------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
| [`no_rocm_image_ubuntu24_04_openmpi.Dockerfile`](no_rocm_image_ubuntu24_04_openmpi.Dockerfile) | https://github.com/ROCm/TheRock/pkgs/container/no_rocm_image_ubuntu24_04_openmpi |

Extended version of no_rocm_image_ubuntu24_04.Dockerfile, containing a system
OpenMPI install (`libopenmpi-dev`, `openmpi-bin`) for the rocprofv3 `mpi-ranks`
integration tests. Like the other `no_rocm_image_*` images this is a test-only
image (not a ROCm build image): the rocprofiler-sdk tests are configured and
compiled in-container at test time, so `find_package(MPI)` needs the OpenMPI
headers and `mpiexec`. OpenMPI is intentionally not bundled in TheRock artifacts
(to avoid forcing a specific MPI onto users), so it is provided at the system
level here for the test environment.

### `rocm_runtime.Dockerfile`

| Source .Dockerfile                                   | Published package |
| ---------------------------------------------------- | ----------------- |
| [`rocm_runtime.Dockerfile`](rocm_runtime.Dockerfile) | N/A               |

This Dockerfile builds ROCm runtime images from TheRock prebuilt tarballs for
multiple Linux distributions using a single Dockerfile via the `BASE_IMAGE`
build argument.

This is a user-space ROCm runtime image. It does NOT include kernel drivers.
The host must provide compatible AMDGPU/ROCm kernel components and device
access (e.g., `--device=/dev/kfd --device=/dev/dri`).

See the header comments in [`rocm_runtime.Dockerfile`](rocm_runtime.Dockerfile)
for supported base images, build arguments, and build/run examples.

The installation method is selected via the `INSTALL_METHOD` build argument:

- `tarball` (default): Downloads and extracts a prebuilt ROCm tarball.
- `packages`: Configures the AMD package repository and installs ROCm via
  apt/dnf/tdnf/zypper.

By default each image targets a single GPU family (e.g.,
`AMDGPU_FAMILY=gfx110X-all`). To build one image that supports all GPU
families, set `AMDGPU_FAMILY=multi-arch` (works with both `INSTALL_METHOD`
options):

- With `INSTALL_METHOD=packages`: pulls the all-GPU meta-package from AMD's
  `packages-multi-arch/` repository.
- With `INSTALL_METHOD=tarball`: downloads AMD's bundled `multiarch`
  tarball from the `tarball-multi-arch/` path (contains per-family `.kpack/`
  files for every supported GPU target).

Supporting scripts:

- [`install_rocm_deps.sh`](install_rocm_deps.sh): Auto-detects the distribution
  and installs ROCm runtime dependencies using the appropriate package manager
  (apt, dnf, tdnf, or zypper).

- [`install_rocm_packages.sh`](install_rocm_packages.sh): Configures the AMD
  deb/rpm package repository (nightlies, prereleases, or stable) and installs
  ROCm via the system package manager. Auto-detects the distribution to select
  the correct repository URL and package manager. Can also be used standalone
  on a Linux host:

  ```bash
  # One-liner installation via packages
  curl -sSL https://raw.githubusercontent.com/ROCm/TheRock/main/dockerfiles/install_rocm_packages.sh | \
    sudo bash -s -- <VERSION> <AMDGPU_FAMILY> [RELEASE_TYPE]

  # Example: Install ROCm 7.13.0a20260322 for gfx110x (nightly)
  curl -sSL https://raw.githubusercontent.com/ROCm/TheRock/main/dockerfiles/install_rocm_packages.sh | \
    sudo bash -s -- 7.13.0a20260322 gfx110x nightlies

  # Example: Install ROCm 7.13.0a20260322 with multi-arch (all GPU families)
  curl -sSL https://raw.githubusercontent.com/ROCm/TheRock/main/dockerfiles/install_rocm_packages.sh | \
    sudo bash -s -- 7.13.0a20260322 multi-arch nightlies
  ```

- [`install_rocm_tarball.sh`](install_rocm_tarball.sh): Downloads ROCm tarball
  from `rocm.{nightlies|prereleases|devreleases}.amd.com` or `repo.amd.com` (for
  stable releases), extracts to `/opt/rocm-{VERSION}` with `/opt/rocm` symlink.
  Can also be used standalone on a Linux host (prints environment variable
  setup instructions after installation):

  ```bash
  # One-liner installation
  curl -sSL https://raw.githubusercontent.com/ROCm/TheRock/main/dockerfiles/install_rocm_tarball.sh | \
    sudo bash -s -- <VERSION> <AMDGPU_FAMILY> [RELEASE_TYPE]

  # Example: Install ROCm 7.11.0a20251211 for gfx110X
  curl -sSL https://raw.githubusercontent.com/ROCm/TheRock/main/dockerfiles/install_rocm_tarball.sh | \
    sudo bash -s -- 7.11.0a20260108 gfx94X-dcgpu nightlies
  ```

## Using published images

Images for these Dockerfiles are published as
[GitHub Packages](https://docs.github.com/en/packages) to the
[GitHub Container Registry](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry)
(`ghcr.io`). The full list of published packages can be viewed at
https://github.com/orgs/ROCm/packages?repo_name=TheRock.

Stable tags for these packages are typically `latest` and `main`, while other
tags may be used for test packages.

## Working on the Dockerfiles themselves

### Testing and debugging

<!-- TODO: document how to build a docker image locally -->

<!-- TODO: document how to run a docker image locally (mount options, entrypoints, etc.) -->

### Automated image publishing

Images are automatically built and published to GitHub Packages (`ghcr.io`) by
GitHub Actions after pushes to the associated `.Dockerfile` on the `main` or
`stage/docker/**` branches. These actions can also be triggered manually from
other branches
[using workflow_dispatch](https://docs.github.com/en/actions/how-tos/manage-workflow-runs/manually-run-a-workflow).

The common
[`.github/workflows/publish_dockerfile.yml`](/.github/workflows/publish_dockerfile.yml)
workflow is used by other `publish_*.yml` workflows:

| Workflow file                                                                                                         | Workflow run history                                                                                                                               |
| --------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`.github/workflows/publish_build_manylinux_x86_64.yml`](/.github/workflows/publish_build_manylinux_x86_64.yml)       | [actions/workflows/publish_build_manylinux_x86_64.yml](https://github.com/ROCm/TheRock/actions/workflows/publish_build_manylinux_x86_64.yml)       |
| [`.github/workflows/publish_no_rocm_image_ubuntu24_04.yml`](/.github/workflows/publish_no_rocm_image_ubuntu24_04.yml) | [actions/workflows/publish_no_rocm_image_ubuntu24_04.yml](https://github.com/ROCm/TheRock/actions/workflows/publish_no_rocm_image_ubuntu24_04.yml) |

Tags for built docker images are set based on the branch name pattern:

| Branch name pattern   | Tag pattern    |
| --------------------- | -------------- |
| `main`                | `latest`       |
| `stage/docker/SUFFIX` | `stage-SUFFIX` |
| `OTHER_NAME`          | `OTHER_NAME`   |

### Updating images used by GitHub Actions workflows

The general sequence for updating an image is this:

1. Make any changes to the relevant Dockerfiles and/or scripts
1. Build and test images locally
   (see [Testing and debugging](#testing-and-debugging))
1. Build and test images on CI
   1. Push the Dockerfile changes to a shared branch and build test packages
      (see [Automated image publishing](#automated-image-publishing))

   1. Edit sha256 pins across the project to use the staging packages, like so:

      ```diff
          container:
      -     image: ghcr.io/rocm/therock_build_manylinux_x86_64@sha256:4af52d56d91ef6ef8b7d0a13c6115af1ab2c9bf4a8a85d9267b489ecb737ed25
      +     image: ghcr.io/rocm/therock_build_manylinux_x86_64@sha256:6e8242d347af7e0c43c82d5031a3ac67b669f24898ea8dc2f1d5b7e4798b66bd
      ```

   1. Trigger test CI/CD jobs (e.g. by creating a draft PR with those changes)
1. Create a pull request containing the Dockerfile changes and go through code
   review
   - For example: https://github.com/ROCm/TheRock/pull/2365
1. Wait for new packages to be published
1. Create a second pull request editting the sha256 pins across the project to
   use the new packages (from the `latest` tag)
   - For example: https://github.com/ROCm/TheRock/pull/2390