TheRock/rocm-libraries/projects/hipdnn/docs/Knobs.md

hipDNN Engine Configuration Knobs

This document provides comprehensive information about hipDNN's engine configuration knobs system.

Table of Contents

• Overview
• What are Knobs?
• Knob Types
• Naming Conventions
• Using Knobs
  • Querying Available Knobs
  • Setting Knob Values
  • Using Default Values
• Standard Global Knobs
• Provider-Specific Knobs
• API Reference
• Best Practices

---

Overview

Engine configuration knobs provide a flexible mechanism for controlling runtime behavior of hipDNN engines. They allow you to tune performance, configure algorithmic choices, and adjust memory usage without recompiling code.

What are Knobs?

Knobs are runtime-configurable parameters that affect engine behavior. Each knob has:

• Unique identifier: A string-based ID (e.g., "global.benchmarking")
• Type: Integer (int64), Float (double), or String
• Default value: The value used when not explicitly set
• Constraints: Valid ranges or allowed values
• Description: Human-readable explanation of purpose

Knobs enable you to:

• Enable or disable features (e.g., benchmarking mode)
• Tune performance parameters (e.g., tile sizes, workspace limits)
• Select algorithmic variants (e.g., solver selection)
• Control memory usage (e.g., workspace size limits)

Knob Types

hipDNN supports two categories of knobs:

Global Knobs

Global knobs are standard knobs available across all or most engines. They are defined in the global.* namespace and provide common functionality.

Characteristics:

• Namespace prefix: global.*
• Consistent behavior across engines
• Defined by the hipDNN specification
• Cannot be registered by custom plugins

Examples:

• global.benchmarking - Enable/disable solver benchmarking
• global.workspace_size_limit - Maximum workspace memory

Custom Knobs

Custom knobs are engine-specific or plugin-specific parameters. Plugin developers can register custom knobs to expose their own tuning parameters.

Characteristics:

• Namespace prefix: <plugin>.* or <plugin>.<operation>.*
• Engine-specific behavior
• Defined by plugin developers
• Extend hipDNN functionality for specific use cases

Examples:

• miopen.conv.tile_size - Convolution tile size for MIOpen
• rocblas.gemm.algo - GEMM algorithm selection for ROCm BLAS

Naming Conventions

Knobs follow a hierarchical naming scheme to avoid conflicts and improve organization:

<namespace>.<category>.<knob_name>

Global namespace (reserved):

global.benchmarking
global.workspace_size_limit
global.deterministic

Plugin-specific namespace:

miopen.conv.tile_size
rocblas.gemm.transpose_algorithm
custom_plugin.matmul.block_size

Important

The global.* namespace is reserved for standard knobs. Custom plugins cannot register knobs in this namespace.

---

Using Knobs

Querying Available Knobs

Before setting knobs, you typically want to discover what knobs an engine supports and their constraints.

C++ Frontend API

#include <hipdnn_frontend.hpp>

using namespace hipdnn_frontend;

// After building the graph
Graph graph;
// ... setup and build graph ...

// Get available knobs for an engine
std::vector<Knob> knobs;
auto error = graph.get_knobs_for_engine(engineId, knobs);

if (error.is_good()) {
    for (const auto& knob : knobs) {
        std::cout << "Knob ID: " << knob.knobId() << "\n";
        std::cout << "Description: " << knob.description() << "\n";
        std::cout << "Type: " << static_cast<int>(knob.valueType()) << "\n";

        // Access default value (it's a variant)
        const auto& defaultVal = knob.defaultValue();
        if (std::holds_alternative<int64_t>(defaultVal)) {
            std::cout << "Default: " << std::get<int64_t>(defaultVal) << "\n";
        } else if (std::holds_alternative<double>(defaultVal)) {
            std::cout << "Default: " << std::get<double>(defaultVal) << "\n";
        } else if (std::holds_alternative<std::string>(defaultVal)) {
            std::cout << "Default: " << std::get<std::string>(defaultVal) << "\n";
        }

        // Check constraints
        const IConstraint* constraint = knob.constraint();
        if (constraint) {
            std::cout << "Constraint: " << constraint->toString() << "\n";
        }

        std::cout << "---\n";
    }
} else {
    std::cerr << "Error getting knobs: " << error.get_message() << "\n";
}

Using Knob Lookup Map

For easier access by knob ID, use the lookup method:

std::unordered_map<std::string, Knob> knobMap;
auto error = graph.get_knob_lookup_for_engine(engineId, knobMap);

if (error.is_good()) {
    auto it = knobMap.find("global.benchmarking");
    if (it != knobMap.end()) {
        const Knob& benchmarkingKnob = it->second;
        // Use the knob...
    }
}

Setting Knob Values

Once you know what knobs are available, you can set them when creating an execution plan.

Basic Example

#include <hipdnn_frontend.hpp>

using namespace hipdnn_frontend;

Graph graph;
// ... setup and build graph ...

// Create knob settings
std::vector<KnobSetting> settings;

// Set integer knob
settings.emplace_back("global.benchmarking", 1);

// Set int64 knob
settings.emplace_back("global.workspace_size_limit", 1024000LL);

// Set float knob
settings.emplace_back("some.float_knob", 0.5);

// Set string knob
settings.emplace_back("some.string_knob", std::string("value"));

// Create execution plan with these settings
auto error = graph.create_execution_plan_ext(engineId, settings);

if (error.is_good()) {
    std::cout << "Execution plan created successfully with custom knob settings\n";
} else {
    std::cerr << "Error: " << error.get_message() << "\n";
}

Type-Safe Knob Setting

// KnobSetting constructor is type-safe
KnobSetting intSetting("test.knob", 42);                    // int64_t
KnobSetting floatSetting("test.knob", 3.14);                // double
KnobSetting stringSetting("test.knob", std::string("val")); // string

// You can also update values later
intSetting.setValue(100);

Using Default Values

If you don't specify a knob setting, the engine will use the default value defined by the knob. To explicitly use defaults for all knobs:

// Option 1: Don't specify any settings (simplest)
auto error = graph.create_execution_plan_ext(engineId, {});

// Option 2: Specify only the knobs you want to customize
std::vector<KnobSetting> settings;
settings.emplace_back("global.benchmarking", 1);  // Only customize this one
auto error = graph.create_execution_plan_ext(engineId, settings);

Note

Validation: All knob settings are validated against their constraints when creating an execution plan. Invalid values will result in an error with a descriptive message.

Note

Unknown Knobs: If you specify a knob that doesn't exist for the engine, hipDNN will log a warning but continue. This allows forward compatibility when new knobs are added.

Note

Deprecated Knobs: If a knob is marked as deprecated, hipDNN will log a warning when you use it, but the knob will still function normally.

---

Standard Global Knobs

The following global knobs are available in hipDNN:

Knob Name	Type	Default	Description
global.benchmarking	int64	0 (disabled)	Enable benchmarking mode for kernel selection. When enabled, engines may run multiple kernel variants and select the fastest. First run may be slower due to benchmarking overhead.

Note

Additional global knobs may be available depending on the engine. Use get_knobs_for_engine() to discover all available knobs for a specific engine.

---

Provider-Specific Knobs

Different engine providers may expose their own custom knobs. Refer to the provider-specific documentation for details:

• MIOpen Provider Knobs - Configuration knobs for the MIOpen GPU acceleration plugin

Tip

When developing with multiple providers, use get_knobs_for_engine() to programmatically discover available knobs rather than hard-coding knob names.

---

API Reference

Knob Class

Describes metadata for an available knob.

Key Methods:

• const std::string& knobId() - Get the knob identifier
• const std::string& description() - Get human-readable description
• bool isDeprecated() - Check if knob is deprecated
• KnobValueType valueType() - Get value type (INT64, FLOAT64, or STRING)
• const KnobValueVariant& defaultValue() - Get default value as variant
• const IConstraint* constraint() - Get constraint validator
• Error validate(const KnobSetting& setting) - Validate a setting

KnobSetting Class

Represents a knob value setting to apply.

Constructors:

KnobSetting(std::string knobId, KnobValueVariant value);
template <typename T> KnobSetting(std::string knobId, const T& value);

Key Methods:

• const std::string& knobId() - Get knob identifier
• const KnobValueVariant& value() - Get the value
• template <typename T> void setValue(const T& value) - Update value

Graph Methods

Querying Knobs:

Error get_knobs_for_engine(int64_t engineId, std::vector<Knob>& knobs) const;
Error get_knob_lookup_for_engine(int64_t engineId, std::unordered_map<std::string, Knob>& knobs) const;

Setting Knobs:

Error create_execution_plan_ext(int64_t engineId, const std::vector<KnobSetting>& settings);

Constraint Classes

Base Interface:

class IConstraint {
    virtual Error validateKnobSetting(const KnobSetting& setting) const = 0;
    virtual std::string toString() const = 0;
};

Implementations:

• IntConstraint(int64_t minValue, int64_t maxValue, int64_t step, std::unordered_set<int64_t> validValues)
• FloatConstraint(double minValue, double maxValue)
• StringConstraint(int32_t maxLength, std::unordered_set<std::string> validValues)
• EmptyConstraint() - No constraints

Type Definitions

using KnobValueVariant = std::variant<int64_t, double, std::string>;
typedef std::string KnobType_t;

enum class KnobValueType {
    NOT_SET = 0,
    INT64 = 1,
    FLOAT64 = 2,
    STRING = 3,
};

---

Best Practices

For Users

1. Query before setting: Always call get_knobs_for_engine() to understand available knobs and their constraints before setting values.

2. Validate constraints: Check the constraint object to ensure your values are valid before creating execution plans.

3. Use default values when possible: Only customize knobs when you have a specific performance or behavior requirement.

4. Handle errors gracefully: Always check the error return value when setting knobs or creating execution plans.

5. Be aware of deprecated knobs: Watch for deprecation warnings and update your code to use recommended alternatives.

6. Profile before tuning: Measure performance impact when changing knob values to ensure improvements.

For Plugin Developers

Plugin developers can expose custom knobs using the Plugin SDK utilities:

• KnobFactory - Helper class to create knob definitions
• IPlanBuilder::getCustomKnobs() - Interface method for exposing knobs
• GlobalKnobDefines - Constants for standard global knob names

For comprehensive guidance on exposing knobs in your plugin, see the Plugin Development Guide.

---

Related Documentation

• HowTo Guide - Configuring Engine Knobs - Quick start guide for using knobs
• Plugin Development Guide - Providing Knobs - How to expose knobs in plugins
• RFC 0004 - Engine Configuration Knobs Design - Detailed design document
• MIOpen Provider Knobs - MIOpen-specific knobs

---

Examples

For complete working examples, see:

• Knobs Usage Sample - Comprehensive example demonstrating knob discovery and configuration
• Frontend Tests - Unit tests showing knob API usage