TheRock/rocm-systems/experimental/perf-dkms/docs/design.md

DKMS Perf Interface Skeleton Module - Design Document

Overview

This document describes the design and implementation of a DKMS-compatible kernel module that implements a skeleton of the Linux perf interface. The module provides a minimal but complete PMU (Performance Monitoring Unit) driver that demonstrates proper integration with the Linux perf subsystem.

Table of Contents

1. Architecture Overview
2. File Structure
3. Core Components
4. Implementation Details
5. Usage Instructions
6. Testing Strategy
7. Extension Points
8. Troubleshooting

Architecture Overview

Design Goals

• Educational: Provide a clear, well-documented example of PMU driver implementation
• Minimal but Complete: Include all required components for a working PMU driver
• DKMS Compatible: Easy installation across different kernel versions
• Foundation: Serve as a base for real GPU performance monitoring implementations

Key Components

┌─────────────────────────────────────────────────────────────┐
│                    User Space                               │
├─────────────────────────────────────────────────────────────┤
│  perf tools  │  sysfs interface  │  /proc/modules          │
├─────────────────────────────────────────────────────────────┤
│                    Kernel Space                             │
├─────────────────────────────────────────────────────────────┤
│              Linux Perf Subsystem                          │
├─────────────────────────────────────────────────────────────┤
│                   PMU Stub Driver                          │
│  ┌─────────────┐ ┌──────────────┐ ┌─────────────────────┐  │
│  │   PMU Core  │ │ Event Handler│ │  Timer & Counters   │  │
│  └─────────────┘ └──────────────┘ └─────────────────────┘  │
└─────────────────────────────────────────────────────────────┘

File Structure

perf-dkms/
├── CMakeLists.txt            # Root build configuration
├── dkms.conf                 # DKMS configuration
├── src/
│   ├── CMakeLists.txt       # Kbuild wrapper
│   ├── amdgpu_pmu.h         # Main header file
│   ├── pmu_main.c           # Core PMU implementation
│   ├── pmu_events.c         # Event handling utilities
│   ├── pmu_dimension.h      # Dimension encoding/validation
│   ├── aql_perf.c/h         # AQL session & GPU discovery
│   ├── aql_packet_ops.c     # PM4 packet operations
│   ├── aql_pmu_integration.c # PMU-AQL bridge
│   ├── aql_error_recovery.c # Error handling & recovery
│   ├── aql_queue_manager.c/h # AQL queue lifecycle
│   ├── kfd_ioctl_bridge.c/h # KFD ioctl bridge (vm_mmap)
│   └── aql_c/               # Pure C library (kernel/userspace)
│       ├── aql_queue.c/h    # Ring buffer & IB pool
│       ├── pm4_packets.c    # Low-level PM4 primitives
│       ├── packet_generation.c # PM4 sequence generation
│       ├── counter_registry.c # Counter definitions
│       ├── arch_creator.c   # Architecture factory
│       ├── gfx12_creator.c  # GFX12 implementation
│       ├── gfx9_creator.c   # GFX9 implementation
│       ├── tests/           # Validation tests
│       └── tools/           # Packet tools
├── test/
│   ├── load_test.sh         # Basic module load/unload test
│   └── perf_test.sh         # Perf integration test
└── docs/
    └── design.md            # This document

Core Components

1. PMU Structure (struct amdgpu_pmu)

The main data structure that represents our PMU driver:

struct amdgpu_pmu {
    struct pmu pmu;                          /* Base PMU structure */
    struct device *dev;                      /* Device for sysfs */

    /* Event management */
    spinlock_t lock;                         /* Protects event_list */
    struct amdgpu_pmu_event events[AMDGPU_PMU_MAX_EVENTS];
    DECLARE_BITMAP(used_mask, AMDGPU_PMU_MAX_EVENTS);
    int num_events;

    /* Timer for simulating counter updates */
    struct hrtimer timer;
    ktime_t timer_period;

    /* Statistics and counters */
    atomic64_t total_events;
    atomic64_t total_samples;
    atomic64_t counter_cycles;
    atomic64_t counter_instructions;
    atomic64_t counter_cache_misses;
    atomic64_t counter_bandwidth;
};

2. Event Types

Four simulated performance events:

• cycles: Simulated CPU cycles
• instructions: Simulated instructions retired
• cache-misses: Simulated cache misses
• bandwidth: Simulated memory bandwidth

3. PMU Callbacks

Required callbacks for perf integration:

• event_init(): Validate and initialize events
• add(): Schedule event on PMU
• del(): Remove event from PMU
• start(): Start event counting
• stop(): Stop event counting
• read(): Read current counter values

Implementation Details

Module Initialization

1. Memory Allocation: Allocate struct amdgpu_pmu instance
2. Lock Initialization: Set up spinlocks for thread safety
3. Timer Setup: Initialize high-resolution timer for counter updates
4. PMU Registration: Register with Linux perf subsystem via perf_pmu_register()
5. Sysfs Creation: Create sysfs entries for event discovery

Event Management

Event Creation Flow

perf_event_open() syscall
        ↓
amdgpu_pmu_event_init()
        ↓
Event validation
        ↓
amdgpu_pmu_add()
        ↓
Assign counter slot
        ↓
amdgpu_pmu_start()
        ↓
Begin counting

Counter Simulation

• High-Resolution Timer: Updates counters every 50-100ms
• Realistic Increments: Different increment rates per event type
• Thread-Safe Updates: Atomic operations for counter updates
• Overflow Handling: Proper 64-bit counter management

DKMS Integration

Configuration (dkms.conf)

PACKAGE_NAME="perf-pmu-stub"
PACKAGE_VERSION="1.0.0"
BUILT_MODULE_NAME[0]="pmu_stub"
BUILT_MODULE_LOCATION[0]="src"
DEST_MODULE_LOCATION[0]="/kernel/drivers/perf"
MAKE[0]="make -C ${kernel_source_dir} M=${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build/src modules"
AUTOINSTALL="yes"

Build Process

# Add module to DKMS
sudo dkms add -m perf-pmu-stub -v 1.0.0

# Build for current kernel
sudo dkms build -m perf-pmu-stub -v 1.0.0

# Install module
sudo dkms install -m perf-pmu-stub -v 1.0.0

Sysfs Interface

The module creates the following sysfs structure:

/sys/bus/event_source/devices/amdgpu_pmu/
├── events/
│   ├── cycles
│   ├── instructions
│   ├── cache-misses
│   └── bandwidth
├── format/
│   └── config
└── type

Usage Instructions

Building and Installing

1. Clone/Download the module source
2. Build using make:

   cd perf-dkms
   cmake -B build && cmake --build build
   

3. Install via DKMS:

   make dkms-install
   

Loading the Module

# Manual load
sudo modprobe amdgpu_pmu

# Check if loaded
lsmod | grep amdgpu_pmu

# Check kernel messages
dmesg | tail -10

Using with Perf Tools

List Available Events

perf list | grep amdgpu_pmu

Expected output:

pmu_stub/bandwidth/                        [Kernel PMU event]
pmu_stub/cache-misses/                     [Kernel PMU event]
pmu_stub/cycles/                           [Kernel PMU event]
pmu_stub/instructions/                     [Kernel PMU event]

Count Events

# Single event
perf stat -e amdgpu_pmu/cycles/ sleep 5

# Multiple events
perf stat -e amdgpu_pmu/cycles/,pmu_stub/instructions/ sleep 5

# System-wide monitoring
perf stat -a -e amdgpu_pmu/cache-misses/ sleep 10

Example output:

Performance counter stats for 'sleep 5':

         5,000      pmu_stub/cycles/

       5.002381063 seconds time elapsed

Module Parameters

• debug_enable: Enable debug output (default: false)
• timer_period_ms: Timer period in milliseconds (default: 100)

# Load with debug enabled
sudo modprobe amdgpu_pmu debug_enable=true timer_period_ms=50

Testing Strategy

Automated Tests

1. Basic Load Test (test/load_test.sh)

• Module load/unload functionality
• Sysfs interface validation
• Parameter checking
• Error handling

2. Perf Integration Test (test/perf_test.sh)

• Event discovery via perf list
• Event counting via perf stat
• Multiple event handling
• Invalid event rejection

Manual Testing

Verify Module Loading

sudo ./test/load_test.sh

Verify Perf Integration

sudo ./test/perf_test.sh

Debug Information

# Module information
modinfo amdgpu_pmu

# Module parameters
cat /sys/module/amdgpu_pmu/parameters/*

# Event configuration
cat /sys/bus/event_source/devices/amdgpu_pmu/events/*

Extension Points

This skeleton can be extended for real hardware monitoring:

1. Hardware Integration

Replace simulated counters with real hardware register access:

// Replace timer-based simulation
static u64 read_hardware_counter(int counter_id)
{
    // Read from actual hardware registers
    return readq(hw_counter_base + counter_id * 8);
}

2. GPU-Specific Events

Add GPU-specific performance events:

enum gpu_events {
    GPU_SHADER_BUSY = 0x10,
    GPU_MEMORY_BANDWIDTH = 0x11,
    GPU_TEXTURE_CACHE_MISS = 0x12,
    GPU_COMPUTE_UTILIZATION = 0x13,
};

3. Multi-Instance Support

Support multiple GPU devices:

struct gpu_pmu_instance {
    struct amdgpu_pmu base;
    int gpu_id;
    void __iomem *registers;
    struct pci_dev *pdev;
};

4. Advanced Features

• Sampling Support: Implement interrupt-based sampling
• Context Switching: Track per-process GPU usage
• Power Monitoring: Integrate with GPU power management
• Trace Events: Add ftrace integration

Troubleshooting

Common Issues

Module Load Fails

Symptoms: insmod returns error, no sysfs entries created

Debugging:

# Check kernel messages
dmesg | tail -20

# Verify kernel headers
ls /lib/modules/$(uname -r)/build

# Check module dependencies
modprobe --dry-run pmu_stub

Solutions:

• Install kernel headers: sudo apt install linux-headers-$(uname -r)
• Check DKMS build logs: /var/lib/dkms/perf-pmu-stub/1.0.0/build/make.log

Events Not Visible in Perf

Symptoms: perf list doesn't show PMU events

Debugging:

# Check PMU registration
ls /sys/bus/event_source/devices/

# Verify events directory
ls /sys/bus/event_source/devices/amdgpu_pmu/events/

# Check perf capabilities
perf --version

Solutions:

• Ensure module is loaded: lsmod | grep amdgpu_pmu
• Check sysfs permissions
• Verify perf tools version compatibility

Counters Not Incrementing

Symptoms: perf stat shows zero counts

Debugging:

# Check timer status
cat /proc/interrupts | grep timer

# Enable debug output
echo 'module amdgpu_pmu +p' > /sys/kernel/debug/dynamic_debug/control

# Monitor kernel messages
dmesg -w

Solutions:

• Verify timer initialization in module load
• Check for deadlocks in timer handler
• Increase timer frequency (decrease timer_period_ms)

Debug Features

Dynamic Debug

Enable runtime debugging:

# Enable all debug messages
echo 'module amdgpu_pmu +p' > /sys/kernel/debug/dynamic_debug/control

# Enable specific function
echo 'func amdgpu_pmu_timer_handler +p' > /sys/kernel/debug/dynamic_debug/control

Module Statistics

Check module statistics:

# Via sysfs (if implemented)
cat /sys/module/amdgpu_pmu/stats/*

# Via kernel messages
dmesg | grep "pmu_stub.*statistics"

Security Considerations

Kernel Module Security

• Root Privileges: Module loading requires root access
• Module Signing: DKMS automatically signs modules for Secure Boot
• Input Validation: All user inputs are validated
• Memory Safety: Proper bounds checking and memory management

User Permissions

• perf Access: Standard perf permissions apply
• Capability Requirements: CAP_PERFMON for system-wide monitoring
• Process Isolation: Events isolated per process by default

Performance Impact

Overhead Analysis

• Timer Frequency: 10-100Hz (configurable)
• Memory Usage: ~4KB per module instance
• CPU Impact: <0.1% on modern systems
• Lock Contention: Minimal due to per-CPU design

Optimization Opportunities

• Per-CPU Counters: Reduce lock contention
• Hardware Timestamps: Use TSC for better accuracy
• Batch Updates: Group counter updates
• Lazy Allocation: Allocate resources on demand

This design document provides a comprehensive overview of the PMU stub implementation, serving as both documentation and a guide for future enhancements targeting real GPU performance monitoring hardware.

Dimension-Aware Performance Monitoring

Overview

The dimension-aware performance monitoring feature enables users to monitor specific hardware dimensions (XCC, SE, SA, WGP, CU) instead of aggregated GPU-wide counters. This provides fine-grained performance visibility into different parts of the GPU hardware hierarchy.

Hardware Dimension Hierarchy

AMD GPUs are organized in a hierarchical structure:

GPU
 └── XCC (eXtended Compute Core) [0..N]
      └── SE (Shader Engine) [0..3 typically]
           └── SA (Shader Array) [0..1 typically]
                └── WGP (Work Group Processor) [0..3 typically]
                     └── CU (Compute Unit) [0..63 typically]

For GFX12 architecture:

• XCC: 1 instance (index 0)
• SE: 4 instances (index 0-3)
• SA: 2 instances per SE (index 0-1)
• WGP: 4 instances per SA (index 0-3)
• CU: 64 total (distributed across WGPs)

config1 Bit Field Layout

Dimension specifications are encoded in the config1 field of perf_event_attr:

Bits   Field         Description
-----  ------------  -----------------------------------------
0-7    xcc           XCC index (0-255)
8-15   se            Shader Engine index (0-255)
16-23  sa            Shader Array index (0-255)
24-31  wgp           Work Group Processor index (0-255)
32-39  cu            Compute Unit index (0-255)
40     aggregate     Aggregate across dimensions (future)
41-63  reserved      Reserved for future use

Default Behavior

When dimensions are not specified (config1=0), the driver operates in broadcast mode: it reads all hardware instances and aggregates (sums) the results.

When specific dimensions are provided, unspecified sub-dimensions default to 0. For example:

• se=2 is equivalent to se=2,sa=0,wgp=0,cu=0 - targets SE 2, SA 0, WGP 0
• se=2,sa=1 is equivalent to se=2,sa=1,wgp=0,cu=0 - targets SE 2, SA 1, WGP 0

This allows hierarchical targeting of specific hardware units without requiring users to specify all levels of the hierarchy.

Named Parameter Syntax

Users can specify dimensions using named parameters that perf translates to config1:

# Monitor specific shader engine
perf stat -e amdgpu_pmu/sq_waves,se=2/ -a sleep 1

# Monitor specific shader engine and shader array
perf stat -e amdgpu_pmu/sq_waves,se=2,sa=1/ -a sleep 1

# Monitor full hierarchy path
perf stat -e amdgpu_pmu/sq_waves,se=2,sa=1,wgp=3/ -a sleep 1

# Raw config1 encoding (SE=2 is bit 8-15, so 0x0200)
perf stat -e amdgpu_pmu/sq_waves,config1=0x0200/ -a sleep 1

Counter Dimension Support

Not all counters support all dimensions. Counter support is defined in the counter registry:

Counter Type	Supported Dimensions	Example Counters
Global	None (DIM_NONE)	grbm_count, grbm_busy
Per-SE	SE, SA, WGP	sq_waves, sq_insts_valu
Per-SA	SE, SA	gl2c_hit, gl2c_miss
Per-WGP	SE, SA, WGP	ta_busy, ta_total_wavefronts

The driver validates that requested dimensions are supported by the counter before allowing measurement.

Implementation Architecture

1. Format Attribute Exposure

The driver exposes format attributes via sysfs that perf uses to parse parameters:

/sys/bus/event_source/devices/amdgpu_pmu/format/
├── config         (config:0-63)     - Counter ID
├── config1        (config1:0-63)    - Raw dimension encoding
├── xcc            (config1:0-7)     - XCC index
├── se             (config1:8-15)    - SE index
├── sa             (config1:16-23)   - SA index
├── wgp            (config1:24-31)   - WGP index
├── cu             (config1:32-39)   - CU index
└── aggregate      (config1:40)      - Aggregate flag

When a user specifies se=2, perf reads the se format file, sees it maps to config1:8-15, and encodes the value 2 into those bits of event->attr.config1.

2. Dimension Extraction and Validation

The pmu_dimension.h header provides helper functions:

/* Extract dimensions from config1 field */
static inline void pmu_extract_dimensions(u64 config1,
                                          struct pmu_dimension_coords *dims)
{
    dims->xcc = (config1 >> PMU_DIM_XCC_SHIFT) & PMU_DIM_XCC_MASK;
    dims->se = (config1 >> PMU_DIM_SE_SHIFT) & PMU_DIM_SE_MASK;
    dims->sa = (config1 >> PMU_DIM_SA_SHIFT) & PMU_DIM_SA_MASK;
    dims->wgp = (config1 >> PMU_DIM_WGP_SHIFT) & PMU_DIM_WGP_MASK;
    dims->cu = (config1 >> PMU_DIM_CU_SHIFT) & PMU_DIM_CU_MASK;
    dims->aggregate = (config1 >> PMU_DIM_AGGREGATE_SHIFT) & 1;

    /* Mark as valid if any dimension or flag is non-zero */
    dims->valid = (config1 != 0);
}

/* Validate dimensions against hardware limits */
static inline bool pmu_validate_dimensions(
    const struct pmu_dimension_coords *dims,
    const struct pmu_dimension_limits *limits)
{
    if (!dims->valid)
        return true; /* No dimensions specified is valid */
    
    return dims->xcc <= limits->max_xcc &&
           dims->se <= limits->max_se &&
           dims->sa <= limits->max_sa &&
           dims->wgp <= limits->max_wgp &&
           dims->cu <= limits->max_cu;
}

During event initialization (amdgpu_pmu_event_init):

1. Extract dimensions from event->attr.config1
2. Validate against hardware limits from GPU architecture
3. Validate counter supports requested dimensions
4. Pass dimension info to AQL layer for measurement setup

3. Counter Allocation

The aql_counter_try_allocate_dimension() function allocates hardware counters for dimension-specific monitoring:

counter_reg_info_t* aql_counter_try_allocate_dimension(
    block_info_t *block,
    uint32_t event_id,
    struct perf_event *perf_event,
    const struct pmu_dimension_coords *dims,
    arch_t *arch)
{
    /* Calculate flat index from hierarchical coordinates */
    uint32_t flat_index = encode_dimension_index(
        dims->se, dims->sa, dims->wgp,
        arch->num_sa, arch->num_wgp_per_sa);
    
    /* Allocate counter */
    counter_reg_info_t *reg = aql_counter_try_allocate(
        block, event_id, perf_event);
    
    /* Store dimension index for packet generation */
    reg->allocation.instance_id = flat_index;
    
    return reg;
}

The encode_dimension_index() helper converts hierarchical SE/SA/WGP coordinates to a flat array index:

Formula: index = (se * num_sa * wgp_per_sa) + (sa * wgp_per_sa) + wgp

Example (GFX12: 4 SE × 2 SA × 4 WGP):
  SE=2, SA=1, WGP=3:
  index = (2 × 2 × 4) + (1 × 4) + 3 = 16 + 4 + 3 = 23

4. Hardware Configuration (GRBM_GFX_INDEX)

The hardware uses the GRBM_GFX_INDEX register to target specific SE/SA/WGP instances. The packet generation code sets this register before programming counters:

/* In generate_start_packet() */
if (reg_info->allocation.instance_id != 0) {
    /* Decode flat index back to coordinates */
    dimension_coords_t coords = decode_dimension_index(
        reg_info->allocation.instance_id,
        arch->num_sa,
        arch->num_wgp_per_sa);
    
    /* Set GRBM_GFX_INDEX to target specific dimension */
    pm4_set_grbm_index(buffer, 
                       arch->control_regs.grbm_gfx_index,
                       coords.wgp << 2,  /* instance_index */
                       coords.sa,
                       coords.se);
    
    /* Now configure counter - hardware will target specified dimension */
    generate_counter_config(buffer, arch, counter);
} else {
    /* Broadcast mode - target all instances */
    generate_grbm_broadcast(buffer, arch);
    generate_counter_config(buffer, arch, counter);
}

The PM4 packet functions (pm4_set_grbm_index) write to the GRBM_GFX_INDEX register:

typedef struct {
    uint32_t instance_index : 8;  /* WGP instance (shifted by 2) */
    uint32_t sa_index : 8;         /* Shader Array index */
    uint32_t se_index : 8;         /* Shader Engine index */
    uint32_t reserved : 8;
} pm4_grbm_gfx_index_t;

5. Counter Reading

When reading dimension-specific counters, the driver only reads from the targeted dimension instead of iterating through all instances:

/* In generate_read_packet() */
if (reg_info->allocation.instance_id != 0) {
    /* Dimension-specific: read only from targeted instance */
    dimension_coords_t coords = decode_dimension_index(...);
    
    pm4_set_grbm_index(buffer, ..., coords.wgp, coords.sa, coords.se);
    
    /* Copy counter value to memory (single instance) */
    pm4_append_copy_data(buffer, gpu_addr, 
                        reg_info->register_addr_lo, ...);
} else {
    /* Broadcast mode: iterate through all SE/SA/WGP instances */
    for (se = 0; se < num_se; se++) {
        for (sa = 0; sa < num_sa; sa++) {
            for (wgp = 0; wgp < num_wgp; wgp++) {
                pm4_set_grbm_index(...);
                pm4_append_copy_data(...);
            }
        }
    }
}

Usage Examples

Basic Dimension Targeting

# Monitor shader engine 0
perf stat -e amdgpu_pmu/sq_waves,se=0/ -a sleep 1

# Monitor shader engine 1, shader array 0
perf stat -e amdgpu_pmu/sq_waves,se=1,sa=0/ -a sleep 1

# Monitor full path: SE=2, SA=1, WGP=3
perf stat -e amdgpu_pmu/sq_waves,se=2,sa=1,wgp=3/ -a sleep 1

Comparing Dimensions

# Compare activity across shader engines
perf stat -e amdgpu_pmu/sq_waves,se=0/ \
          -e amdgpu_pmu/sq_waves,se=1/ \
          -e amdgpu_pmu/sq_waves,se=2/ \
          -e amdgpu_pmu/sq_waves,se=3/ \
          -a sleep 5

Raw config1 Encoding

# SE=2 (bits 8-15): 0x0200
perf stat -e amdgpu_pmu/sq_waves,config1=0x0200/ -a sleep 1

# SE=2, SA=1 (bits 8-23): 0x00010200  
perf stat -e amdgpu_pmu/sq_waves,config1=0x00010200/ -a sleep 1

# SE=3, SA=1, WGP=2 (bits 8-31): 0x02010300
perf stat -e amdgpu_pmu/sq_waves,config1=0x02010300/ -a sleep 1

Error Handling

The driver provides clear error messages for common mistakes:

# Invalid dimension (SE=99 exceeds maximum)
$ perf stat -e amdgpu_pmu/sq_waves,se=99/ -a sleep 1
Error: dimension out of range: se=99 (max: 3)

# Unsupported dimension for counter
$ perf stat -e amdgpu_pmu/grbm_count,se=0/ -a sleep 1
Error: counter 'grbm_count' does not support requested dimensions

# Dimension validation happens during event_init, before allocation

Dimension limits are determined from GPU architecture at module load time:

/* In amdgpu_pmu_init() */
global_dim_limits.max_xcc = arch->num_xcc - 1;
global_dim_limits.max_se = arch->num_se - 1;
global_dim_limits.max_sa = arch->num_sa - 1;
global_dim_limits.max_wgp = arch->num_wgp_per_sa - 1;
global_dim_limits.max_cu = arch->num_cu - 1;

Current Limitations and Future Work

Implemented

• ✅ Single dimension instance monitoring (one SE/SA/WGP at a time)
• ✅ Named parameter syntax via perf format attributes
• ✅ Raw config1 encoding support
• ✅ Dimension validation against hardware limits
• ✅ Counter compatibility checking
• ✅ GRBM_GFX_INDEX targeting for hardware configuration

Future Enhancements

Aggregate Mode (config1 bit 40):

# Monitor SE=0 but aggregate across all SA/WGP within that SE
perf stat -e amdgpu_pmu/sq_waves,se=0,aggregate=1/ -a sleep 1

Implementation: Allocate multiple counter instances, configure each for different SA/WGP within the SE, sum results.

Per-CU Monitoring: Currently focused on SE/SA/WGP. CU-level monitoring would require additional hardware support and more fine-grained GRBM_GFX_INDEX configuration.

Dynamic Dimension Discovery: Export per-GPU dimension limits via sysfs for runtime discovery:

/sys/bus/event_source/devices/amdgpu_pmu/caps/
├── max_xcc
├── max_se
├── max_sa
├── max_wgp
└── max_cu

Performance Considerations

Overhead:

• Dimension-specific monitoring has minimal overhead vs. broadcast mode
• Single targeted register write vs. broadcast affects all instances
• Counter read is faster (1 instance vs. SE×SA×WGP instances)

Scalability:

• Each dimension-specific event allocates one hardware counter
• Maximum concurrent events limited by available hardware counters per block
• Typical limits: 4-8 counters per block type (SQ, TA, GL2C, etc.)

Use Cases:

• Load Balancing Analysis: Compare activity across SEs to detect imbalance
• Hotspot Detection: Identify which SE/SA is most active
• Power Analysis: Monitor specific regions during power management
• Debugging: Isolate issues to specific hardware units

Testing

See test files:

• src/aql_c/tests/test_dimension_helpers.c - Unit tests for dimension encoding/decoding
• test/load_test.sh - Module load/unload verification
• test/perf_test.sh - Perf integration tests

References

• AMD GFX12 Architecture Documentation
• Linux Perf Subsystem Documentation
• GRBM_GFX_INDEX Register Specification
• PM4 Packet Format Reference