TheRock/docs/development/s3_buckets.md

S3 Buckets

TheRock uses Amazon S3 buckets to store CI build outputs (artifacts, logs, python packages, etc.) and release artifacts. This page lists all buckets and explains the authentication needed to upload to them.

Table of contents

• Authentication
• Bucket inventory
  • CI buckets: therock-ci-artifacts, therock-ci-artifacts-external
  • Release buckets: therock-{dev,nightly,prerelease,release}-{artifacts,packages,python,tarball}
  • Build system buckets: rocm-third-party-deps
  • Cache buckets: therock-pytorch-sccache-*
  • Legacy buckets: therock-artifacts, therock-artifacts-external

Authentication

Most buckets have public read access for use by developers as well as CI/CD systems.

To write to most buckets, assuming an IAM role via aws-actions/configure-aws-credentials using OIDC is needed. This requires id-token: write in the job's permissions block. The full ARN pattern is arn:aws:iam::692859939525:role/therock-{ci,dev,nightly,prerelease}.

Use the configure_aws_artifacts_credentials composite action to set up credentials. It determines the correct IAM role and bucket from the repository, event type, and optional release_type input by using build_tools/_therock_utils/s3_buckets.py:

jobs:
  build:
    runs-on: azure-linux-scale-rocm
    permissions:
      id-token: write
    # Linux containers only — mount runner baseline credentials
    env:
      AWS_SHARED_CREDENTIALS_FILE: /home/awsconfig/credentials.ini

    steps:
      # ... build steps ...

      # Credentials are short-lived — assume the role close to when it's needed.
      - name: Configure AWS Credentials
        uses: ./.github/actions/configure_aws_artifacts_credentials

      # ... upload steps that use the credentials ...

Platform-specific details:

• Linux containers mount runner credentials via AWS_SHARED_CREDENTIALS_FILE: /home/awsconfig/credentials.ini in the job's env block. These baseline credentials allow uploading to therock-ci-artifacts-external without OIDC.
• Windows jobs must pass special-characters-workaround: true to aws-actions/configure-aws-credentials. This retries credential fetching until the secret access key contains no special characters, which some Windows environments cannot tolerate. (The configure_aws_artifacts_credentials composite action mentioned above handles this automatically)

External repos and forks:

• External repos (e.g., rocm-libraries) use OIDC with the therock-ci-external role to upload to therock-ci-artifacts-external.
• Fork PRs cannot use OIDC (no trust relationship). They fall back to runner base credentials.

Bucket inventory

CI buckets

Our CI runners come with baseline credentials that allow uploading to therock-ci-artifacts-external without any extra setup. Workflows in downstream repos like rocm-libraries, rocm-systems, and llvm-project upload to this bucket and do not need aws-actions/configure-aws-credentials.

Bucket	Contents	IAM role
therock-ci-artifacts	Build outputs for ROCm/TheRock	therock-ci
therock-ci-artifacts-external	Build outputs for forks and other repos	therock-ci-external, or runner base credentials

Release buckets

Each release type (dev, nightly, prerelease, release) has a matching set of buckets.

The dev, nightly, and prerelease types are accessed via the therock-{release_type} IAM role while stable release buckets are manually promoted from prereleases via IAM user policies (see how_to_do_release.md).

Python, tarball, and native package buckets are fronted by CloudFront CDNs. Developer-facing documentation and manual installs should use the CDN URLs; CI may read the backing S3 buckets directly to avoid CloudFront data-transfer charges.

Bucket	Contents	IAM role	CDN
therock-dev-artifacts	Build outputs	therock-dev	—
therock-dev-packages	Native packages	therock-dev	rocm.devreleases.amd.com/packages-multi-arch/deb/, …/rpm/
therock-dev-python	Python packages	therock-dev	rocm.devreleases.amd.com/whl-multi-arch/
therock-dev-tarball	ROCm tarballs	therock-dev	rocm.devreleases.amd.com/tarball-multi-arch/
therock-nightly-artifacts	Build outputs	therock-nightly	—
therock-nightly-packages	Native packages	therock-nightly	rocm.nightlies.amd.com/packages-multi-arch/deb/, …/rpm/
therock-nightly-python	Python packages	therock-nightly	rocm.nightlies.amd.com/whl-multi-arch/
therock-nightly-tarball	ROCm tarballs	therock-nightly	rocm.nightlies.amd.com/tarball-multi-arch/
therock-prerelease-artifacts	Build outputs	therock-prerelease	—
therock-prerelease-packages	Native packages	therock-prerelease	rocm.prereleases.amd.com/packages-multi-arch/
therock-prerelease-python	Python packages	therock-prerelease	rocm.prereleases.amd.com/whl-multi-arch/
therock-prerelease-tarball	ROCm tarballs	therock-prerelease	rocm.prereleases.amd.com/tarball-multi-arch/
therock-release-artifacts	Build outputs	—	—
therock-release-packages	Native packages	—	repo.amd.com/rocm/packages-multi-arch/
therock-release-python	Python packages	—	repo.amd.com/rocm/whl-multi-arch/
therock-release-tarball	ROCm tarballs	—	repo.amd.com/rocm/tarball-multi-arch/

Build system buckets

We mirror third-party dependency files into S3 for use by the build system.

Bucket	Contents	Details
rocm-third-party-deps	Mirrors for third_party/ subprojects	See "Updating a third-party mirror" in git_chores.md

Cache buckets

Unlike the other buckets on this page, the PyTorch sccache buckets live in a separate AWS account (324352301041) in region us-east-1. The IAM role names are identical to the artifact-pipeline roles but resolve to a different account ID; the corresponding role ARNs are arn:aws:iam::324352301041:role/therock-{ci,dev,nightly,prerelease}. OIDC trust for therock-{dev,nightly} also covers repo:ROCm/rockrel:* so reusable workflow invocations from rockrel can assume the role.

Bucket	Contents	IAM role
therock-pytorch-sccache-ci	PyTorch CI sccache	therock-ci
therock-pytorch-sccache-dev	PyTorch dev sccache	therock-dev
therock-pytorch-sccache-nightly	PyTorch nightly sccache	therock-nightly
therock-pytorch-sccache-prerelease	PyTorch prerelease sccache	therock-prerelease

Legacy buckets

CI runs before 2025-11-11 (see TheRock#2046) used different bucket names. These are no longer written to but still contain historical data. We may remove these once we implement a retention policy for artifacts.

Legacy bucket	Replaced by	IAM role
therock-artifacts	therock-ci-artifacts	therock-artifacts
therock-artifacts-external	therock-ci-artifacts-external	therock-artifacts-external