TheRock/rocm-libraries/projects/hipdnn/docs/HowTo.md

hipDNN How-To Guide

This guide provides practical information for both using hipDNN components and extending the framework with new functionality.

Table of Contents

• Consuming hipDNN
  • Using the Frontend
  • Using the Backend
  • Using the SDKs
  • CMake Integration
  • Logging Setup
  • Working with Schemas
  • Configuring Engine Knobs
• Extending hipDNN
  • Adding a New Plugin
  • Adding a New Operation
  • Descriptor Code Generator
  • Development Workflow

---

Consuming hipDNN

Tip

For a minimal end-to-end example of using hipDNN in a CMake project, see the Consumer Quick Start.

This section covers how to use the various components of hipDNN in your applications.

Using the Frontend

The hipDNN frontend provides a C++ header-only API for building and executing operation graphs. For detailed architecture and design information, see the Frontend section in the Design Guide.

File Structure

• Library includes: frontend/include/
• Unit tests: frontend/tests/
• Samples: samples/

Using the Backend

The hipDNN backend is a shared library that provides the core C API for graph execution and plugin management. For comprehensive details about the backend architecture, descriptor types, and workflow, see the Backend section in the Design Guide.

File Structure

• Public includes: backend/include/
• Public API tests: tests/backend/

Using the SDKs

hipDNN provides three header-only C++ SDK libraries for plugin development and testing. For complete SDK functionality and roadmap, see the SDKs section in the Design Guide.

Key Components

• FlatBuffers SDK: Schema files and generated data objects: flatbuffers_sdk/schemas/
• Plugin SDK: Plugin interface definitions: plugin_sdk/include/hipdnn_plugin_sdk/EnginePluginApi.h
• Test SDK: Test utilities and CPU reference implementations: test_sdk/include/hipdnn_test_sdk/
• Logging: data_sdk/include/hipdnn_data_sdk/logging/Logger.hpp

CMake Integration

hipDNN components can be easily integrated into your CMake projects using the installed package files.

Note

Enable PIC/PIE to ensure compatibility with the plugin loader system (dlopen). This prevents potential Thread Local Storage (TLS) allocation issues (such as static TLS exhaustion) between the executable and dynamically loaded backend plugins.

set(CMAKE_POSITION_INDEPENDENT_CODE ON)

Frontend Integration

find_package(hipdnn_frontend REQUIRED)
target_link_libraries(your_target PRIVATE hipdnn_frontend)

Backend Integration

find_package(hipdnn_backend REQUIRED)
target_link_libraries(your_target PRIVATE hipdnn_backend)

Data SDK Integration

find_package(hipdnn_data_sdk REQUIRED)
target_link_libraries(your_plugin PRIVATE hipdnn_data_sdk)

Plugin SDK Integration

find_package(hipdnn_plugin_sdk REQUIRED)
target_link_libraries(your_plugin PRIVATE hipdnn_plugin_sdk)

Test SDK Integration

find_package(hipdnn_test_sdk REQUIRED)
target_link_libraries(your_test PRIVATE hipdnn_test_sdk)

Using AMD Half or BFloat16 Types

If you use AMD half or bfloat16 types (via the Data SDK's UtilsFp16.hpp or UtilsBfp16.hpp), you need:

find_package(hip REQUIRED)
enable_language(HIP)
target_link_libraries(your_target hip::host hip::device)

Note

📝 If CMake cannot find the packages after installation, ensure your CMAKE_PREFIX_PATH includes the install location. By default on Linux systems, hipDNN CMake files are installed to /opt/rocm/lib/cmake.

Working with Schemas

hipDNN uses FlatBuffers for schema-based data objects to describe graphs and operations.

Key Concepts

• Graphs and operations are defined using .fbs schema files
• Attributes marked as long types in graphs are foreign keys to the uid in tensor_attributes
• Schema files are located in flatbuffers_sdk/schemas/

Configuring Engine Knobs

hipDNN engines support runtime configuration through knobs - configurable parameters that control engine behavior, performance tuning, and feature selection.

Tip

For comprehensive knobs documentation including all available knobs, constraints, validation, and advanced usage, see the Knobs Documentation.

---

Extending hipDNN

This section covers how to extend hipDNN with new functionality.

Adding a New Plugin

Plugins extend hipDNN to support new or additional implementations of kernel engines, benchmarking, and heuristics. The Plugin SDK provides interfaces and utilities to simplify plugin development:

• Engine interfaces: IEngine, IPlanBuilder, IPlan templates for building plugin components
• Engine management: EngineManager template for managing multiple engines
• Knob utilities: KnobFactory, KnobSettingFactory, and GlobalKnobDefines for implementing runtime-configurable knobs

For comprehensive guidance on plugin development, including architecture details, implementation steps, and examples, see the Plugin Development Guide.

Adding a New Operation

Adding a new operation touches every layer of the stack — FlatBuffers schema, backend descriptor and enums, frontend attributes/packer/unpacker/node/Graph API, JSON utilities, optional Python bindings, and tests at four levels. The hipdnn-codegen agent skill (backed by tools/DescriptorGenerator/) automates the mechanical parts.

Tip

See the Adding a New Operation Guide for the full contributor walkthrough — cuDNN naming parity rules, the recommended workflow, layer-by-layer reference, testing matrix, and PR checklist.

Plugin Integration

Once the operation lands across the layers above, refer to the Plugin Development Guide to implement execution in target plugins.

Descriptor Code Generator

The Descriptor Code Generator (tools/DescriptorGenerator/) is a Python tool that generates the C++ boilerplate required to land a new operation type in hipDNN. Each operation in hipDNN requires a consistent set of files — descriptors, packers, unpackers, attributes, nodes, enum entries, factory wiring, and tests — and the generator produces all of them from a single YAML configuration.

What It Generates

From one YAML config, the tool produces:

• Backend descriptor (.hpp/.cpp) with setAttribute/getAttribute, finalize, buildNode, fromNode, and toString
• Frontend packer and unpacker for lowering and lifting between frontend graph nodes and backend descriptors
• Frontend attributes class and node class with graph method
• Unit tests for the descriptor, graph building, and fromNode round-trips
• Integration tests for lowering and lifting round-trips
• Fragment files with enum entries, factory cases, CMake entries, and string utility switch cases for insertion into existing shared files

When to Use It

Scenario	Generator Mode
Brand new operation (nothing exists yet)	--mode full
Adding backend only (frontend exists or will be added later)	--mode backend
Adding frontend only (backend descriptor already exists)	--mode frontend

Existing Configurations

The tool ships with validated configs for all current operations:

Config	Operation
convolution_fwd.yaml	Convolution forward (reference config)
convolution_bwd.yaml	Convolution backward data
convolution_wrw.yaml	Convolution backward weights
matmul.yaml	Matrix multiplication
pointwise.yaml	Pointwise operations
batchnorm.yaml	Batch normalization (training forward)
batchnorm_backward.yaml	Batch normalization backward
batchnorm_inference.yaml	Batch normalization inference
batchnorm_inference_variance_ext.yaml	Batch normalization inference (variance extension)
sdpa.yaml	Scaled dot-product attention

These serve as references when creating a config for a new operation. Use convolution_fwd.yaml as the primary reference — it exercises all config features.

Quick Start

cd projects/hipdnn/tools/DescriptorGenerator

# One-time setup
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt

# Preview what would be generated (dry run)
.venv/bin/python generate.py \
    --config configs/your_operation.yaml \
    --output-dir /tmp/preview \
    --mode full \
    --dry-run

# Generate into the project tree
.venv/bin/python generate.py \
    --config configs/your_operation.yaml \
    --output-dir ../../ \
    --mode full

For full documentation on YAML config format, field types, mode enum definitions, and post-generation integration steps, see the Descriptor Code Generator README.

---

Development Workflow

Typical Development Flow

1. For New Operations (using the code generator):

   FBS Schema → YAML Config → Code Generator → Place Files & Fragments → Plugin Implementation → Tests
   

2. For New Operations (manual):

   Data SDK Schema → Backend Descriptor → Frontend Classes → Plugin Implementation → Tests
   

3. For Existing Operations in New Plugins:

   Plugin Implementation → Integration Tests
   

Building and Testing

1. Rebuild hipDNN: After changing hipDNN, you will need to rebuild. See the quick start steps in the build guide, or rebuild the specific targets.

2. Test Your Implementation:

   • Unit tests for individual components
   • Integration tests for new and untested end-to-end functionality

Important Considerations

• Backward Compatibility: Ensure schema changes don't break existing operations
• Plugin Discovery: For example, engine plugins are loaded from hipdnn_plugins/engines/ relative to the backend library
• Error Handling: Implement proper error reporting through the plugin API
• Performance: Optimization is critical for facilitating plugin adoption

CI Maintenance

• TheRock CI: Uses a pinned Git commit hash from the TheRock repository. therock_ci.yml and several other workflows need their hash updated at a frequent cadence for CI to build hipDNN with recent deps.
• ROCm Version (hipdnn-clang-tidy.yml): Uses a fixed ROCm release version from TheRock artifacts (e.g., 7.11.0a20260112). Update on-demand by changing the --release arg in the install_rocm_from_artifacts.py call. It will need to be bumped when new dependency APIs are required that are absent from past ROCm releases.

Debugging Tips

• Enable logging with environment variables (see Environment Configuration)
• Use integration tests to verify operation behavior
• Check plugin loading with HIPDNN_LOG_LEVEL=info
• For plugin issues, check the default plugin path or use custom paths with hipdnnSetEnginePluginPaths_ext

⚠️ Troubleshooting

Segmentation Faults during Graph Execution Plan Build

If you are seeing segfaults when building execution plans for graphs, this might be caused by Thread Local Storage (TLS) allocation issues (such as static TLS exhaustion) between the executable and dynamically loaded backend plugins.

To resolve this, enable PIC/PIE to ensure compatibility with the plugin loader system (dlopen). This setting instructs CMake to emit position-independent code (e.g., via -fPIC or -fPIE), which is necessary for creating shared libraries or executables that load plugins dynamically.

set(CMAKE_POSITION_INDEPENDENT_CODE ON)