TheRock/rocm-systems/emulation/rocjitsu/README.md

# rocjitsu: ROCm™ Just-in-Time Suite

[![rocjitsu CI](https://github.com/ROCm/rocm-systems/actions/workflows/rocjitsu-ci.yml/badge.svg)](https://github.com/ROCm/rocm-systems/actions/workflows/rocjitsu-ci.yml)

An emulation toolkit for running AMD GPU applications on simulated or
real hardware. Supports three execution strategies:

- **Simulation** — Full ISA emulation on simulated GPU hardware via
  LD_PRELOAD interposition. No physical GPU or kernel module required.
- **Dynamic Binary Translation (DBT)** — Cross-ISA translation enabling
  applications compiled for one GPU architecture to run on another.
- **Dynamic Binary Instrumentation (DBI)** — Runtime instrumentation
  of GPU kernels for profiling, tracing, and analysis.

## Supported architectures

| Architecture | GFX Target | ISA Family | Simulation | DBT | DBI |
|---|---|---|---|---|---|
| CDNA1™ | gfx908 | GFX9 | Experimental | Experimental | Planned |
| CDNA2™ | gfx90a | GFX9 | Experimental | Experimental | Planned |
| CDNA3™ | gfx94x | GFX9 | Experimental | Experimental | Planned |
| CDNA4™ | gfx950 | GFX9 | Beta | Experimental | Planned |
| RDNA1™ | gfx1010 | GFX10 | Experimental | Experimental | Planned |
| RDNA2™ | gfx1030 | GFX10 | Experimental | Experimental | Planned |
| RDNA3™ | gfx1100 | GFX11 | Experimental | Experimental | Planned |
| RDNA3.5™ | gfx1150 | GFX11 | Experimental | Experimental | Planned |
| RDNA4™ | gfx1200 | GFX12 | Experimental | Experimental | Planned |
| gfx1250      | gfx1250 | GFX12 | Experimental | Experimental | Planned |
| RISC-V | RV64I | RV | Experimental | — | — |

<!-- \NPI new GPU: add a row to the supported-architectures table above. -->

## Project layout

```
lib/
  simdojo/              Simulation engine (PDES framework)
  rocjitsu/
    include/rocjitsu/   Public C API
    src/rocjitsu/
      vm/amdgpu/        GPU hardware model (CP, CU, caches, memory)
      isa/arch/amdgpu/  ISA execution (10 targets, mostly autogenerated)
      kmd/linux/        KFD driver emulation + LD_PRELOAD interposer
      code/             Code object loader, basic block analysis
      code/dbt/         Dynamic binary translator
      code/patch/       Code object patcher, spill manager
      analysis/         Register liveness and def-use analysis
      config/           JSON/FlatBuffers configuration
  util/                 Shared utilities
  python/amdisa/        ISA codegen pipeline
tools/rocjitsu/         CLI (local, daemon, attach modes)
configs/                GPU topology JSON files
schemas/                FlatBuffers schemas
tests/                  Test suite
docs/                   Design documents and guides
```

## Building

```bash
cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build
```

See [docs/building.md](docs/building.md) for CMake options, sanitizers,
and formatting setup.

## Running tests

```bash
ctest --test-dir build
```

HIP kernel tests and RCCL daemon tests require an ROCm installation.
They are disabled automatically when `hipcc` or `libhsa-runtime64` is
not found.

## Running HIP workloads

```bash
# Local mode (in-process simulation)
rocjitsu --config configs/gfx950_cdna4_kmd.json -- ./my_hip_app

# Daemon mode (separate daemon process)
rocjitsu --daemon --config configs/gfx950_cdna4_kmd.json -- ./my_hip_app
```

See [docs/rocjitsu-cli.md](docs/rocjitsu-cli.md) for all CLI modes.

## VGPR observation contract

Execution plugins observe architectural instruction-level VGPR reads and
writes. Callback lane and byte masks describe the architectural register
effect, including masked and sub-dword operations. Internal storage operations
used to preserve unaffected register state are not reported as additional
instruction effects.

Asynchronous memory operations use a separate lifecycle. The race detector
records register dependencies when the operation is issued. Later completion
updates register storage without reporting the same instruction effect again.
See
[docs/plugins.md](docs/plugins.md) for the plugin contract.

## Running PyTorch

```bash
rocjitsu --daemon --config configs/gfx950_cdna4_kmd.json -- \
  python3 -c "import torch; x = torch.randn(4, 4, device='cuda'); print(x @ x)"
```

See [docs/building.md](docs/building.md) for container setup with PyTorch.

## Documentation

### Getting started

| Document | Description |
|---|---|
| [Building](docs/building.md) | Build options, sanitizers, container setup |
| [Configuration](docs/configuration.md) | JSON config format and topology |
| [CLI & Transport](docs/rocjitsu-cli.md) | CLI modes, daemon RPC protocol |
| [Race Detector](docs/race-detector.md) | Race detection tutorial and internals |

### Design

| Document | Description |
|---|---|
| [Architecture](docs/architecture.md) | Component overview and layer map |
| [Plugins](docs/plugins.md) | Execution plugin system and sink API |
| [VM Design](docs/vm-design.md) | GPU hardware model (CP, CU, caches, KMD) |
| [Simdojo Engine](docs/simdojo.md) | PDES simulation framework |
| [DBT Design](docs/dbt-design.md) | Binary translator architecture |
| [DBI Design](docs/dbi-design.md) | Binary instrumentation (in progress) |
| [Codegen](docs/codegen.md) | ISA codegen pipeline and regen commands |
| [ISA Target Providers](docs/isa-target-providers.md) | Static target registration and per-component subsets |

### Reference

| Document | Description |
|---|---|
| [Utilities](lib/util/README.md) | Shared utility library (`lib/util/`) |
| [rj_dbt_translate](docs/rj_dbt_translate.md) | Standalone DBT translation tool |
| [CHANGELOG](CHANGELOG.md) | Release history |

## Contributing

Read [CONTRIBUTING.md](CONTRIBUTING.md) before submitting changes. It
covers where new code should live, which existing libraries to use and
extend, the codegen workflow, and code style conventions.

## License

[MIT License](LICENSE.md) &mdash; Copyright (c) 2025-2026 Advanced Micro Devices, Inc.