# DKMS Perf Interface Skeleton Module - Design Document ## Overview This document describes the design and implementation of a DKMS-compatible kernel module that implements a skeleton of the Linux perf interface. The module provides a minimal but complete PMU (Performance Monitoring Unit) driver that demonstrates proper integration with the Linux perf subsystem. ## Table of Contents 1. [Architecture Overview](#architecture-overview) 2. [File Structure](#file-structure) 3. [Core Components](#core-components) 4. [Implementation Details](#implementation-details) 5. [Usage Instructions](#usage-instructions) 6. [Testing Strategy](#testing-strategy) 7. [Extension Points](#extension-points) 8. [Troubleshooting](#troubleshooting) ## Architecture Overview ### Design Goals - **Educational**: Provide a clear, well-documented example of PMU driver implementation - **Minimal but Complete**: Include all required components for a working PMU driver - **DKMS Compatible**: Easy installation across different kernel versions - **Foundation**: Serve as a base for real GPU performance monitoring implementations ### Key Components ``` ┌─────────────────────────────────────────────────────────────┐ │ User Space │ ├─────────────────────────────────────────────────────────────┤ │ perf tools │ sysfs interface │ /proc/modules │ ├─────────────────────────────────────────────────────────────┤ │ Kernel Space │ ├─────────────────────────────────────────────────────────────┤ │ Linux Perf Subsystem │ ├─────────────────────────────────────────────────────────────┤ │ PMU Stub Driver │ │ ┌─────────────┐ ┌──────────────┐ ┌─────────────────────┐ │ │ │ PMU Core │ │ Event Handler│ │ Timer & Counters │ │ │ └─────────────┘ └──────────────┘ └─────────────────────┘ │ └─────────────────────────────────────────────────────────────┘ ``` ## File Structure ``` perf-dkms/ ├── CMakeLists.txt # Root build configuration ├── dkms.conf # DKMS configuration ├── src/ │ ├── CMakeLists.txt # Kbuild wrapper │ ├── amdgpu_pmu.h # Main header file │ ├── pmu_main.c # Core PMU implementation │ ├── pmu_events.c # Event handling utilities │ ├── pmu_dimension.h # Dimension encoding/validation │ ├── aql_perf.c/h # AQL session & GPU discovery │ ├── aql_packet_ops.c # PM4 packet operations │ ├── aql_pmu_integration.c # PMU-AQL bridge │ ├── aql_error_recovery.c # Error handling & recovery │ ├── aql_queue_manager.c/h # AQL queue lifecycle │ ├── kfd_ioctl_bridge.c/h # KFD ioctl bridge (vm_mmap) │ └── aql_c/ # Pure C library (kernel/userspace) │ ├── aql_queue.c/h # Ring buffer & IB pool │ ├── pm4_packets.c # Low-level PM4 primitives │ ├── packet_generation.c # PM4 sequence generation │ ├── counter_registry.c # Counter definitions │ ├── arch_creator.c # Architecture factory │ ├── gfx12_creator.c # GFX12 implementation │ ├── gfx9_creator.c # GFX9 implementation │ ├── tests/ # Validation tests │ └── tools/ # Packet tools ├── test/ │ ├── load_test.sh # Basic module load/unload test │ └── perf_test.sh # Perf integration test └── docs/ └── design.md # This document ``` ## Core Components ### 1. PMU Structure (`struct amdgpu_pmu`) The main data structure that represents our PMU driver: ```c struct amdgpu_pmu { struct pmu pmu; /* Base PMU structure */ struct device *dev; /* Device for sysfs */ /* Event management */ spinlock_t lock; /* Protects event_list */ struct amdgpu_pmu_event events[AMDGPU_PMU_MAX_EVENTS]; DECLARE_BITMAP(used_mask, AMDGPU_PMU_MAX_EVENTS); int num_events; /* Timer for simulating counter updates */ struct hrtimer timer; ktime_t timer_period; /* Statistics and counters */ atomic64_t total_events; atomic64_t total_samples; atomic64_t counter_cycles; atomic64_t counter_instructions; atomic64_t counter_cache_misses; atomic64_t counter_bandwidth; }; ``` ### 2. Event Types Four simulated performance events: - **cycles**: Simulated CPU cycles - **instructions**: Simulated instructions retired - **cache-misses**: Simulated cache misses - **bandwidth**: Simulated memory bandwidth ### 3. PMU Callbacks Required callbacks for perf integration: - `event_init()`: Validate and initialize events - `add()`: Schedule event on PMU - `del()`: Remove event from PMU - `start()`: Start event counting - `stop()`: Stop event counting - `read()`: Read current counter values ## Implementation Details ### Module Initialization 1. **Memory Allocation**: Allocate `struct amdgpu_pmu` instance 2. **Lock Initialization**: Set up spinlocks for thread safety 3. **Timer Setup**: Initialize high-resolution timer for counter updates 4. **PMU Registration**: Register with Linux perf subsystem via `perf_pmu_register()` 5. **Sysfs Creation**: Create sysfs entries for event discovery ### Event Management #### Event Creation Flow ``` perf_event_open() syscall ↓ amdgpu_pmu_event_init() ↓ Event validation ↓ amdgpu_pmu_add() ↓ Assign counter slot ↓ amdgpu_pmu_start() ↓ Begin counting ``` #### Counter Simulation - **High-Resolution Timer**: Updates counters every 50-100ms - **Realistic Increments**: Different increment rates per event type - **Thread-Safe Updates**: Atomic operations for counter updates - **Overflow Handling**: Proper 64-bit counter management ### DKMS Integration #### Configuration (`dkms.conf`) ```ini PACKAGE_NAME="perf-pmu-stub" PACKAGE_VERSION="1.0.0" BUILT_MODULE_NAME[0]="pmu_stub" BUILT_MODULE_LOCATION[0]="src" DEST_MODULE_LOCATION[0]="/kernel/drivers/perf" MAKE[0]="make -C ${kernel_source_dir} M=${dkms_tree}/${PACKAGE_NAME}/${PACKAGE_VERSION}/build/src modules" AUTOINSTALL="yes" ``` #### Build Process ```bash # Add module to DKMS sudo dkms add -m perf-pmu-stub -v 1.0.0 # Build for current kernel sudo dkms build -m perf-pmu-stub -v 1.0.0 # Install module sudo dkms install -m perf-pmu-stub -v 1.0.0 ``` ### Sysfs Interface The module creates the following sysfs structure: ``` /sys/bus/event_source/devices/amdgpu_pmu/ ├── events/ │ ├── cycles │ ├── instructions │ ├── cache-misses │ └── bandwidth ├── format/ │ └── config └── type ``` ## Usage Instructions ### Building and Installing 1. **Clone/Download** the module source 2. **Build** using make: ```bash cd perf-dkms cmake -B build && cmake --build build ``` 3. **Install via DKMS**: ```bash make dkms-install ``` ### Loading the Module ```bash # Manual load sudo modprobe amdgpu_pmu # Check if loaded lsmod | grep amdgpu_pmu # Check kernel messages dmesg | tail -10 ``` ### Using with Perf Tools #### List Available Events ```bash perf list | grep amdgpu_pmu ``` Expected output: ``` pmu_stub/bandwidth/ [Kernel PMU event] pmu_stub/cache-misses/ [Kernel PMU event] pmu_stub/cycles/ [Kernel PMU event] pmu_stub/instructions/ [Kernel PMU event] ``` #### Count Events ```bash # Single event perf stat -e amdgpu_pmu/cycles/ sleep 5 # Multiple events perf stat -e amdgpu_pmu/cycles/,pmu_stub/instructions/ sleep 5 # System-wide monitoring perf stat -a -e amdgpu_pmu/cache-misses/ sleep 10 ``` Example output: ``` Performance counter stats for 'sleep 5': 5,000 pmu_stub/cycles/ 5.002381063 seconds time elapsed ``` ### Module Parameters - `debug_enable`: Enable debug output (default: false) - `timer_period_ms`: Timer period in milliseconds (default: 100) ```bash # Load with debug enabled sudo modprobe amdgpu_pmu debug_enable=true timer_period_ms=50 ``` ## Testing Strategy ### Automated Tests #### 1. Basic Load Test (`test/load_test.sh`) - Module load/unload functionality - Sysfs interface validation - Parameter checking - Error handling #### 2. Perf Integration Test (`test/perf_test.sh`) - Event discovery via `perf list` - Event counting via `perf stat` - Multiple event handling - Invalid event rejection ### Manual Testing #### Verify Module Loading ```bash sudo ./test/load_test.sh ``` #### Verify Perf Integration ```bash sudo ./test/perf_test.sh ``` #### Debug Information ```bash # Module information modinfo amdgpu_pmu # Module parameters cat /sys/module/amdgpu_pmu/parameters/* # Event configuration cat /sys/bus/event_source/devices/amdgpu_pmu/events/* ``` ## Extension Points This skeleton can be extended for real hardware monitoring: ### 1. Hardware Integration Replace simulated counters with real hardware register access: ```c // Replace timer-based simulation static u64 read_hardware_counter(int counter_id) { // Read from actual hardware registers return readq(hw_counter_base + counter_id * 8); } ``` ### 2. GPU-Specific Events Add GPU-specific performance events: ```c enum gpu_events { GPU_SHADER_BUSY = 0x10, GPU_MEMORY_BANDWIDTH = 0x11, GPU_TEXTURE_CACHE_MISS = 0x12, GPU_COMPUTE_UTILIZATION = 0x13, }; ``` ### 3. Multi-Instance Support Support multiple GPU devices: ```c struct gpu_pmu_instance { struct amdgpu_pmu base; int gpu_id; void __iomem *registers; struct pci_dev *pdev; }; ``` ### 4. Advanced Features - **Sampling Support**: Implement interrupt-based sampling - **Context Switching**: Track per-process GPU usage - **Power Monitoring**: Integrate with GPU power management - **Trace Events**: Add ftrace integration ## Troubleshooting ### Common Issues #### Module Load Fails **Symptoms**: `insmod` returns error, no sysfs entries created **Debugging**: ```bash # Check kernel messages dmesg | tail -20 # Verify kernel headers ls /lib/modules/$(uname -r)/build # Check module dependencies modprobe --dry-run pmu_stub ``` **Solutions**: - Install kernel headers: `sudo apt install linux-headers-$(uname -r)` - Check DKMS build logs: `/var/lib/dkms/perf-pmu-stub/1.0.0/build/make.log` #### Events Not Visible in Perf **Symptoms**: `perf list` doesn't show PMU events **Debugging**: ```bash # Check PMU registration ls /sys/bus/event_source/devices/ # Verify events directory ls /sys/bus/event_source/devices/amdgpu_pmu/events/ # Check perf capabilities perf --version ``` **Solutions**: - Ensure module is loaded: `lsmod | grep amdgpu_pmu` - Check sysfs permissions - Verify perf tools version compatibility #### Counters Not Incrementing **Symptoms**: `perf stat` shows zero counts **Debugging**: ```bash # Check timer status cat /proc/interrupts | grep timer # Enable debug output echo 'module amdgpu_pmu +p' > /sys/kernel/debug/dynamic_debug/control # Monitor kernel messages dmesg -w ``` **Solutions**: - Verify timer initialization in module load - Check for deadlocks in timer handler - Increase timer frequency (decrease `timer_period_ms`) ### Debug Features #### Dynamic Debug Enable runtime debugging: ```bash # Enable all debug messages echo 'module amdgpu_pmu +p' > /sys/kernel/debug/dynamic_debug/control # Enable specific function echo 'func amdgpu_pmu_timer_handler +p' > /sys/kernel/debug/dynamic_debug/control ``` #### Module Statistics Check module statistics: ```bash # Via sysfs (if implemented) cat /sys/module/amdgpu_pmu/stats/* # Via kernel messages dmesg | grep "pmu_stub.*statistics" ``` ## Security Considerations ### Kernel Module Security - **Root Privileges**: Module loading requires root access - **Module Signing**: DKMS automatically signs modules for Secure Boot - **Input Validation**: All user inputs are validated - **Memory Safety**: Proper bounds checking and memory management ### User Permissions - **perf Access**: Standard perf permissions apply - **Capability Requirements**: CAP_PERFMON for system-wide monitoring - **Process Isolation**: Events isolated per process by default ## Performance Impact ### Overhead Analysis - **Timer Frequency**: 10-100Hz (configurable) - **Memory Usage**: ~4KB per module instance - **CPU Impact**: <0.1% on modern systems - **Lock Contention**: Minimal due to per-CPU design ### Optimization Opportunities - **Per-CPU Counters**: Reduce lock contention - **Hardware Timestamps**: Use TSC for better accuracy - **Batch Updates**: Group counter updates - **Lazy Allocation**: Allocate resources on demand This design document provides a comprehensive overview of the PMU stub implementation, serving as both documentation and a guide for future enhancements targeting real GPU performance monitoring hardware. ## Dimension-Aware Performance Monitoring ### Overview The dimension-aware performance monitoring feature enables users to monitor specific hardware dimensions (XCC, SE, SA, WGP, CU) instead of aggregated GPU-wide counters. This provides fine-grained performance visibility into different parts of the GPU hardware hierarchy. ### Hardware Dimension Hierarchy AMD GPUs are organized in a hierarchical structure: ``` GPU └── XCC (eXtended Compute Core) [0..N] └── SE (Shader Engine) [0..3 typically] └── SA (Shader Array) [0..1 typically] └── WGP (Work Group Processor) [0..3 typically] └── CU (Compute Unit) [0..63 typically] ``` For GFX12 architecture: - **XCC**: 1 instance (index 0) - **SE**: 4 instances (index 0-3) - **SA**: 2 instances per SE (index 0-1) - **WGP**: 4 instances per SA (index 0-3) - **CU**: 64 total (distributed across WGPs) ### config1 Bit Field Layout Dimension specifications are encoded in the `config1` field of perf_event_attr: ``` Bits Field Description ----- ------------ ----------------------------------------- 0-7 xcc XCC index (0-255) 8-15 se Shader Engine index (0-255) 16-23 sa Shader Array index (0-255) 24-31 wgp Work Group Processor index (0-255) 32-39 cu Compute Unit index (0-255) 40 aggregate Aggregate across dimensions (future) 41-63 reserved Reserved for future use ``` ### Default Behavior When dimensions are not specified (config1=0), the driver operates in **broadcast mode**: it reads all hardware instances and aggregates (sums) the results. When specific dimensions are provided, unspecified sub-dimensions default to 0. For example: - `se=2` is equivalent to `se=2,sa=0,wgp=0,cu=0` - targets SE 2, SA 0, WGP 0 - `se=2,sa=1` is equivalent to `se=2,sa=1,wgp=0,cu=0` - targets SE 2, SA 1, WGP 0 This allows hierarchical targeting of specific hardware units without requiring users to specify all levels of the hierarchy. ### Named Parameter Syntax Users can specify dimensions using named parameters that perf translates to config1: ```bash # Monitor specific shader engine perf stat -e amdgpu_pmu/sq_waves,se=2/ -a sleep 1 # Monitor specific shader engine and shader array perf stat -e amdgpu_pmu/sq_waves,se=2,sa=1/ -a sleep 1 # Monitor full hierarchy path perf stat -e amdgpu_pmu/sq_waves,se=2,sa=1,wgp=3/ -a sleep 1 # Raw config1 encoding (SE=2 is bit 8-15, so 0x0200) perf stat -e amdgpu_pmu/sq_waves,config1=0x0200/ -a sleep 1 ``` ### Counter Dimension Support Not all counters support all dimensions. Counter support is defined in the counter registry: | Counter Type | Supported Dimensions | Example Counters | |-------------|---------------------|-----------------| | Global | None (DIM_NONE) | grbm_count, grbm_busy | | Per-SE | SE, SA, WGP | sq_waves, sq_insts_valu | | Per-SA | SE, SA | gl2c_hit, gl2c_miss | | Per-WGP | SE, SA, WGP | ta_busy, ta_total_wavefronts | The driver validates that requested dimensions are supported by the counter before allowing measurement. ### Implementation Architecture #### 1. Format Attribute Exposure The driver exposes format attributes via sysfs that perf uses to parse parameters: ``` /sys/bus/event_source/devices/amdgpu_pmu/format/ ├── config (config:0-63) - Counter ID ├── config1 (config1:0-63) - Raw dimension encoding ├── xcc (config1:0-7) - XCC index ├── se (config1:8-15) - SE index ├── sa (config1:16-23) - SA index ├── wgp (config1:24-31) - WGP index ├── cu (config1:32-39) - CU index └── aggregate (config1:40) - Aggregate flag ``` When a user specifies `se=2`, perf reads the `se` format file, sees it maps to `config1:8-15`, and encodes the value 2 into those bits of event->attr.config1. #### 2. Dimension Extraction and Validation The `pmu_dimension.h` header provides helper functions: ```c /* Extract dimensions from config1 field */ static inline void pmu_extract_dimensions(u64 config1, struct pmu_dimension_coords *dims) { dims->xcc = (config1 >> PMU_DIM_XCC_SHIFT) & PMU_DIM_XCC_MASK; dims->se = (config1 >> PMU_DIM_SE_SHIFT) & PMU_DIM_SE_MASK; dims->sa = (config1 >> PMU_DIM_SA_SHIFT) & PMU_DIM_SA_MASK; dims->wgp = (config1 >> PMU_DIM_WGP_SHIFT) & PMU_DIM_WGP_MASK; dims->cu = (config1 >> PMU_DIM_CU_SHIFT) & PMU_DIM_CU_MASK; dims->aggregate = (config1 >> PMU_DIM_AGGREGATE_SHIFT) & 1; /* Mark as valid if any dimension or flag is non-zero */ dims->valid = (config1 != 0); } /* Validate dimensions against hardware limits */ static inline bool pmu_validate_dimensions( const struct pmu_dimension_coords *dims, const struct pmu_dimension_limits *limits) { if (!dims->valid) return true; /* No dimensions specified is valid */ return dims->xcc <= limits->max_xcc && dims->se <= limits->max_se && dims->sa <= limits->max_sa && dims->wgp <= limits->max_wgp && dims->cu <= limits->max_cu; } ``` During event initialization (`amdgpu_pmu_event_init`): 1. Extract dimensions from event->attr.config1 2. Validate against hardware limits from GPU architecture 3. Validate counter supports requested dimensions 4. Pass dimension info to AQL layer for measurement setup #### 3. Counter Allocation The `aql_counter_try_allocate_dimension()` function allocates hardware counters for dimension-specific monitoring: ```c counter_reg_info_t* aql_counter_try_allocate_dimension( block_info_t *block, uint32_t event_id, struct perf_event *perf_event, const struct pmu_dimension_coords *dims, arch_t *arch) { /* Calculate flat index from hierarchical coordinates */ uint32_t flat_index = encode_dimension_index( dims->se, dims->sa, dims->wgp, arch->num_sa, arch->num_wgp_per_sa); /* Allocate counter */ counter_reg_info_t *reg = aql_counter_try_allocate( block, event_id, perf_event); /* Store dimension index for packet generation */ reg->allocation.instance_id = flat_index; return reg; } ``` The `encode_dimension_index()` helper converts hierarchical SE/SA/WGP coordinates to a flat array index: ``` Formula: index = (se * num_sa * wgp_per_sa) + (sa * wgp_per_sa) + wgp Example (GFX12: 4 SE × 2 SA × 4 WGP): SE=2, SA=1, WGP=3: index = (2 × 2 × 4) + (1 × 4) + 3 = 16 + 4 + 3 = 23 ``` #### 4. Hardware Configuration (GRBM_GFX_INDEX) The hardware uses the GRBM_GFX_INDEX register to target specific SE/SA/WGP instances. The packet generation code sets this register before programming counters: ```c /* In generate_start_packet() */ if (reg_info->allocation.instance_id != 0) { /* Decode flat index back to coordinates */ dimension_coords_t coords = decode_dimension_index( reg_info->allocation.instance_id, arch->num_sa, arch->num_wgp_per_sa); /* Set GRBM_GFX_INDEX to target specific dimension */ pm4_set_grbm_index(buffer, arch->control_regs.grbm_gfx_index, coords.wgp << 2, /* instance_index */ coords.sa, coords.se); /* Now configure counter - hardware will target specified dimension */ generate_counter_config(buffer, arch, counter); } else { /* Broadcast mode - target all instances */ generate_grbm_broadcast(buffer, arch); generate_counter_config(buffer, arch, counter); } ``` The PM4 packet functions (`pm4_set_grbm_index`) write to the GRBM_GFX_INDEX register: ```c typedef struct { uint32_t instance_index : 8; /* WGP instance (shifted by 2) */ uint32_t sa_index : 8; /* Shader Array index */ uint32_t se_index : 8; /* Shader Engine index */ uint32_t reserved : 8; } pm4_grbm_gfx_index_t; ``` #### 5. Counter Reading When reading dimension-specific counters, the driver only reads from the targeted dimension instead of iterating through all instances: ```c /* In generate_read_packet() */ if (reg_info->allocation.instance_id != 0) { /* Dimension-specific: read only from targeted instance */ dimension_coords_t coords = decode_dimension_index(...); pm4_set_grbm_index(buffer, ..., coords.wgp, coords.sa, coords.se); /* Copy counter value to memory (single instance) */ pm4_append_copy_data(buffer, gpu_addr, reg_info->register_addr_lo, ...); } else { /* Broadcast mode: iterate through all SE/SA/WGP instances */ for (se = 0; se < num_se; se++) { for (sa = 0; sa < num_sa; sa++) { for (wgp = 0; wgp < num_wgp; wgp++) { pm4_set_grbm_index(...); pm4_append_copy_data(...); } } } } ``` ### Usage Examples #### Basic Dimension Targeting ```bash # Monitor shader engine 0 perf stat -e amdgpu_pmu/sq_waves,se=0/ -a sleep 1 # Monitor shader engine 1, shader array 0 perf stat -e amdgpu_pmu/sq_waves,se=1,sa=0/ -a sleep 1 # Monitor full path: SE=2, SA=1, WGP=3 perf stat -e amdgpu_pmu/sq_waves,se=2,sa=1,wgp=3/ -a sleep 1 ``` #### Comparing Dimensions ```bash # Compare activity across shader engines perf stat -e amdgpu_pmu/sq_waves,se=0/ \ -e amdgpu_pmu/sq_waves,se=1/ \ -e amdgpu_pmu/sq_waves,se=2/ \ -e amdgpu_pmu/sq_waves,se=3/ \ -a sleep 5 ``` #### Raw config1 Encoding ```bash # SE=2 (bits 8-15): 0x0200 perf stat -e amdgpu_pmu/sq_waves,config1=0x0200/ -a sleep 1 # SE=2, SA=1 (bits 8-23): 0x00010200 perf stat -e amdgpu_pmu/sq_waves,config1=0x00010200/ -a sleep 1 # SE=3, SA=1, WGP=2 (bits 8-31): 0x02010300 perf stat -e amdgpu_pmu/sq_waves,config1=0x02010300/ -a sleep 1 ``` ### Error Handling The driver provides clear error messages for common mistakes: ```bash # Invalid dimension (SE=99 exceeds maximum) $ perf stat -e amdgpu_pmu/sq_waves,se=99/ -a sleep 1 Error: dimension out of range: se=99 (max: 3) # Unsupported dimension for counter $ perf stat -e amdgpu_pmu/grbm_count,se=0/ -a sleep 1 Error: counter 'grbm_count' does not support requested dimensions # Dimension validation happens during event_init, before allocation ``` Dimension limits are determined from GPU architecture at module load time: ```c /* In amdgpu_pmu_init() */ global_dim_limits.max_xcc = arch->num_xcc - 1; global_dim_limits.max_se = arch->num_se - 1; global_dim_limits.max_sa = arch->num_sa - 1; global_dim_limits.max_wgp = arch->num_wgp_per_sa - 1; global_dim_limits.max_cu = arch->num_cu - 1; ``` ### Current Limitations and Future Work #### Implemented - ✅ Single dimension instance monitoring (one SE/SA/WGP at a time) - ✅ Named parameter syntax via perf format attributes - ✅ Raw config1 encoding support - ✅ Dimension validation against hardware limits - ✅ Counter compatibility checking - ✅ GRBM_GFX_INDEX targeting for hardware configuration #### Future Enhancements **Aggregate Mode** (config1 bit 40): ```bash # Monitor SE=0 but aggregate across all SA/WGP within that SE perf stat -e amdgpu_pmu/sq_waves,se=0,aggregate=1/ -a sleep 1 ``` Implementation: Allocate multiple counter instances, configure each for different SA/WGP within the SE, sum results. **Per-CU Monitoring**: Currently focused on SE/SA/WGP. CU-level monitoring would require additional hardware support and more fine-grained GRBM_GFX_INDEX configuration. **Dynamic Dimension Discovery**: Export per-GPU dimension limits via sysfs for runtime discovery: ``` /sys/bus/event_source/devices/amdgpu_pmu/caps/ ├── max_xcc ├── max_se ├── max_sa ├── max_wgp └── max_cu ``` ### Performance Considerations **Overhead**: - Dimension-specific monitoring has minimal overhead vs. broadcast mode - Single targeted register write vs. broadcast affects all instances - Counter read is faster (1 instance vs. SE×SA×WGP instances) **Scalability**: - Each dimension-specific event allocates one hardware counter - Maximum concurrent events limited by available hardware counters per block - Typical limits: 4-8 counters per block type (SQ, TA, GL2C, etc.) **Use Cases**: - **Load Balancing Analysis**: Compare activity across SEs to detect imbalance - **Hotspot Detection**: Identify which SE/SA is most active - **Power Analysis**: Monitor specific regions during power management - **Debugging**: Isolate issues to specific hardware units ### Testing See test files: - `src/aql_c/tests/test_dimension_helpers.c` - Unit tests for dimension encoding/decoding - `test/load_test.sh` - Module load/unload verification - `test/perf_test.sh` - Perf integration tests ### References - AMD GFX12 Architecture Documentation - Linux Perf Subsystem Documentation - GRBM_GFX_INDEX Register Specification - PM4 Packet Format Reference