TheRock/rocm-systems/emulation/rocjitsu/docs/plugins.md

rocjitsu Plugins

Execution plugins that hook into rocjitsu's simulation model. Each plugin implements the ExecutionPlugin interface and receives callbacks for wavefront dispatches, memory instructions, register reads, barriers, etc.

Plugins

Plugin	Location	Description
RaceDetectorPlugin	race_detector/	Hooks memory instructions, register reads, barriers, and s_waitcnt to detect data races. Reports violations with disassembly traces. See race-detector.md.
KernelLoggingPlugin	logging/	Logs kernel dispatches and detects MMA instruction usage.

The race detector plugin contains both the core detection algorithm (race_detector/core/) and the rocjitsu adapter (race_detector/plugin.h).

Kernel Logging Plugin

The logging plugin records kernel dispatch metadata and detects MMA (matrix multiply-accumulate) instruction usage:

• Kernel dispatches: entry PC, grid dimensions, workgroup dimensions, register counts, and kernel name (when available from the code object).
• MMA detection: reports the first MFMA or WMMA instruction seen in each dispatch.

Enabling plugins

Plugins are compiled into standalone shared objects named librocjitsu_plugin_<name>.so and discovered at runtime through the standard dynamic-linker search path (librocjitsu_plugin_*.so are installed next to the interposer, and the launcher adds that directory to LD_LIBRARY_PATH).

A plugin is enabled by listing it in the plugins section of the rocjitsu config file. The key is the plugin name (the <name> in librocjitsu_plugin_<name>.so) and the value is a JSON object with the plugin's configuration:

{
  "plugins": {
    "race": {},
    "logging": {}
  }
}

The bundled plugins are race (RaceDetectorPlugin) and logging (KernelLoggingPlugin).

Enabling plugins from the mirage CLI

When launching a workload through mirage, plugins can be selected on the command line with --plugin <name> instead of editing a config file. mirage injects each selected plugin into the rocjitsu config it synthesises for the run (and, for containerised profiles, bind-mounts the plugin's .so next to the interposer). The flag is repeatable and merges with any plugins the profile already enables:

# Enable the race detector and the kernel logger for a single run.
mirage run --plugin race --plugin logging -- ./my_app

# Same, when starting a session.
mirage session start --profile mi350x --plugin race

Each --plugin enables the plugin with its schema defaults. Plugins that take required arguments, or runs that need custom sink settings, are configured through a profile or an explicit --config <file>.

Plugin ABI

The plugin boundary is a C-shaped ABI. Each plugin .so exports three extern "C" functions:

• const PluginMetadata *rocjitsu_plugin_metadata() — returns a pointer to static metadata: abi version, name, contact, version, and a config_schema JSON string.
• PluginHandle rocjitsu_plugin_create(const char *config_json) — constructs the plugin from its resolved JSON configuration string and returns an opaque handle.
• void rocjitsu_plugin_destroy(PluginHandle handle) — destroys an instance previously returned by rocjitsu_plugin_create.

Allocation and deallocation stay on the plugin side of the boundary: the host destroys each instance through the plugin's own rocjitsu_plugin_destroy export. Use the ROCJITSU_DEFINE_PLUGIN macro from plugin_abi.h to emit all three functions. The host validates the reported abi against the loader's expected version before use.

Config schema

The config_schema string describes the accepted config keys. Each key maps to an object with a type (string, number, or boolean), an optional description, and an optional default. Keys without a default are required. Example:

{
  "argname": { "type": "string", "description": "does something important", "default": "defaultvalue" },
  "requiredarg": { "type": "number" }
}

The loader merges defaults, validates types, checks for required keys, and passes the resolved JSON object to rocjitsu_plugin_create.

Plugin output

Plugins write diagnostic output (race reports, profiling data, kernel logs) through a configurable sink system rather than directly to stderr. This makes output testable and redirectable.

Sink configuration

Sinks are configured from an optional top-level sinks object in the rocjitsu config (the same file that lists the plugins). There are no sink-related environment variables.

Key	Default	Description
types	["stderr"]	Array of sink types: stderr, stdout, file
dir	(none)	Directory for file sinks. Required when file is in types

When file is in types, each plugin writes to <dir>/<plugin_name>.log. Plugin names are fixed: race for RaceDetectorPlugin, logging for KernelLoggingPlugin.

Profiled execution

Set the top-level "profiled": true key to wrap the plugins in a profiled execution group, which emits per-hook timing data (HOOK_PROFILE lines) to the configured sinks. With the default sink, timing data goes to stderr; stdout sends it to stdout, and file sinks write it to <dir>/profile.log.

Profiled execution requires the simulation engine to use "num_threads": 1. Multithreaded configurations are rejected because the profiling counters are not synchronized.

Examples

Interactive use — output goes to stderr (the default):

{ "plugins": { "race": {} } }

rocjitsu --config my_config.json -- ./my_app

Save race reports to files (for test harnesses):

{
  "plugins": { "race": {} },
  "sinks": { "types": ["file"], "dir": "/tmp/output" }
}

rocjitsu --config my_config.json -- ./my_app
# Race reports are in /tmp/output/race.log

Send output to both stderr and a file simultaneously:

{
  "plugins": { "race": {} },
  "sinks": { "types": ["stderr", "file"], "dir": "/tmp/output" }
}

Note: plugins can also be selected on the mirage command line with mirage run --plugin <name> (see "Enabling plugins from the mirage CLI" above). Sink selection is still driven entirely by the config file shown here.

Writing a plugin that uses sinks

Plugins inherit a sink from ExecutionPlugin. Use sink().write(msg) for all output instead of fprintf(stderr, ...) or std::cerr:

class MyPlugin : public ExecutionPlugin {
public:
  explicit MyPlugin(const char *config_json) : ExecutionPlugin("myplugin") {
    (void)config_json;
  }

  void onAmdgpuDispatchPacketProcessed(const KernelDispatchInfo &info) override {
    sink().write(std::format("[myplugin] dispatch {}\n", info.dispatch_id));
  }
};

The sink is assigned by the ExecutionPluginGroup when the plugin is added. If no group configures a sink, the default is stderr.

How it works

The ExecutionPlugin interface (execution_plugin.h) defines hooks that the compute unit and command processor call during execution. Multiple plugins can be active simultaneously via ExecutionPluginGroup.

VGPR observation precision

onAmdgpuWriteVgprLanes observes instruction-level VGPR destinations rather than VM/runtime storage writes. Reads and writes carry the architectural lane and byte masks for masked, sub-dword, and multi-register operands. Internal storage operations used to preserve unaffected register state do not produce additional architectural callbacks.

Asynchronous memory operations are modeled separately. The race detector records their register dependencies when they are issued. A later completion updates storage without emitting the same instruction-level write again. Synchronization retires the corresponding outstanding operations.

Adding a new plugin

1. Implement ExecutionPlugin in a new subdirectory. The plugin class must be constructible from const char *config_json.
2. Add a plugin_export.cpp that calls ROCJITSU_DEFINE_PLUGIN(MyPlugin, "myname", contact, version, schema).
3. In CMakeLists.txt, add the object library and a rj_add_plugin_so(myname <object_lib> <export_src>) call so it builds librocjitsu_plugin_myname.so.
4. Use sink().write() for all output — never write to stderr directly.
5. Enable it by adding "myname": { ... } to the plugins section of the config file.