TheRock/rocm-libraries/projects/rocwmma/CHANGELOG.md

Changelog for rocWMMA

Documentation for rocWMMA is available at https://rocm.docs.amd.com/projects/rocWMMA/en/latest.

Since last release (ROCm 7.12)

Added

• Added the following community samples for external contributions, with build support and documentation
  • simple_gemm_silu: demonstrates a GEMM + SiLU fused operator using the rocWMMA API
  • simple_gemm_fusion: demonstrates block-tile-level dual-GEMM fusion using the rocWMMA API
  • simple_gemm_swiglu: demonstrates a SwiGLU fused dual-GEMM kernel (LLaMA/Mistral FFN gate layer) using the rocWMMA API

Changed

• Updated the find_package search for OpenMP to prefer the openmp-config.cmake provided by ROCm, with a fallback to module search mode.
• Updated INSTALL_RPATH and added BUILD_RPATH for OpenMP

Resolved issues

• Improved HIP RTC regression test portability when deployed outside the default path

rocWMMA 2.2.0 for ROCm 7.2.0

Added

• Added sample perf_i8gemm to demonstrate int8_t as matrix input data type
• Added support for the gfx1150 target

Changed

• Removed unnecessary const keyword to avoid compiler warnings
• rocWMMA has been moved into the new rocm-libraries "monorepo" repository (https://github.com/ROCm/rocm-libraries). This repository consolidates a number of separate ROCm libraries and shared components.
  • The repository migration requires a few changes to the CMake configuration of rocWMMA
  • The repository migration rquired the GTest dependency to be updated to v1.16.0

Resolved issues

• Skip invalid test configurations when using 'register file' LDS mapping
• Ensured transform functions in samples are only available on the device

rocWMMA 2.1.0 for ROCm 7.1.1

Added

• Added more unit tests to increase the code coverage.

Changed

• Increased compile timeout and improved visualization in math-ci.

Removed

• Removed absolute paths from the RPATH of sample and test binary files.

Resolved issues

• Fixed issues caused by HIP changes:
  • Removed the .data member from HIP_vector_type.
  • Broadcast constructor now only writes to the first vector element.
• Fixed a bug related to int32_t usage in hipRTC_gemm for gfx942, caused by breaking changes in HIP.
• Replaced #pragma unroll with static for to fix a bug caused by the upgraded compiler which no longer supports using #pragma unroll with template parameter indices.
• Corrected test predicates for BLK and VW cooperative kernels.
• Modified compute_utils.sh in build-infra to ensure rocWMMA is built with gfx1151 target for ROCm 7.0 and beyond.

rocWMMA 2.0.0 for ROCm 7.0.0

Added

• Added internal register layout transforms to support interleaved MMA layouts
• Added support for the gfx950 target
• Added mixed input bf8 / fp8 types for MMA support
• Added fragment scheduler API objects to embed thread block cooperation properties in fragments

Changed

• Augmented load / store / MMA internals with static loop unrolling
• rocWMMA mma_sync API now supports wave tile fragment sizes
• rocWMMA cooperative fragments are now expressed with fragment scheduler template arguments
• rocWMMA cooperative fragments now use the same base API as non-cooperative fragments
• rocWMMA cooperative fragments register usage footprint has been reduced
• rocWMMA fragments now support partial tile sizes with padding

Optimized

• Added internal flow control barriers to improve assembly code generation and overall performance
• Enabled interleaved layouts by default in MMA to improve overall performance

Removed

• Removed support for the gfx940 and gfx941 targets
• Removed the rocWMMA cooperative API
• Removed wave count template parameters from transforms APIs

Resolved issues

• Fixed a validation issue for small precision compute types < B32 on gfx9
• Fixed CMake validation of compiler support for bf8 / fp8 types
• Fixed linkage of rocwmma::synchronize_workgroup to inline

rocWMMA 1.7.0 for ROCm 6.4.0

Added

• Added interleaved layouts that enhance the performance of GEMM operations
• Added emulation test suites. These suites are lightweight and well-suited for execution on emulator platforms

Changed

• Used GPU_TARGETS instead of AMDGPU_TARGETS in cmakelists.txt
• The binary sizes can be reduced on supported compilers by using the --offload-compress compiler flag

Resolved issues

• For a CMake bug workaround, set CMAKE_NO_BUILTIN_CHRPATH when BUILD_OFFLOAD_COMPRESS is unset

rocWMMA 1.6.0 for ROCm 6.3.0

Added

• Added OCP f8/bf8 datatype support
• Added support for gfx12 architecture targets

Changed

• Optimized some aos<->soa transforms with half-rotation offsets
• Refactored the rocBLAS reference entry point for validation and benchmarking
• ROCWMMA_* preprocessor configurations are now all assigned values
• Updated the default architecture targets for ASAN builds
• Updated the actor-critic implementation

Resolved issues

• Fixed a bug in f64 validation due to faulty typecasting
• Fixed a bug causing runtime compilation errors with hipRTC
• Various documentation updates and fixes

Upcoming changes

• rocWMMA 2.x plans to augment fragment API objects with additional meta-properties which will improve API expressiveness and configurability of parameters including multiple-wave cooperation. As part of this change, cooperative rocWMMA API functions load_matrix_coop_sync and store_matrix_coop_sync will become deprecated in a future ROCm release.

rocWMMA 1.5.0 for ROCm 6.2.0

Additions

• Added internal utilities for element-wise vector transforms
• Added internal utilities for cross-lane vector transforms
• Implemented internal aos<->soa transforms for block sizes of 16, 32, 64, 128 and 256 and vector widths of 2, 4, 8 and 16
• Added tests for new internal transforms

Changes

• Improved loading layouts by increasing vector width for fragments with blockDim > 32
• API applyDataLayout transform now accepts WaveCount template argument for cooperative fragments
• API applyDataLayout transform now physically applies aos<->soa transform as necessary
• Refactored entry-point of std library usage to improve hipRTC support
• Documentation updates for installation, programmer's guide and API reference

Fixes

• Fixed some header includes ordering to improve portability

rocWMMA 1.4.0 for ROCm 6.1.0

Additions

• Added bf16 support for hipRTC sample

Changes

• Changed Clang C++ version to C++17
• Updated rocwmma_coop API
• Linked rocWMMA to hiprtc

Fixes

• Fixed compile/runtime arch checks
• Built all test in large code model
• Removed inefficient branching in layout loop unrolling

rocWMMA 1.3.0 for ROCm 6.0.0

Additions

• Support for gfx940, gfx941, and gfx942 targets
• Support for f8, bf8, and xfloat32 data types
• support for HIP_NO_HALF, __ HIP_NO_HALF_CONVERSIONS__, and __ HIP_NO_HALF_OPERATORS__ (e.g., PyTorch environment)

Changes

• rocWMMA with hipRTC now supports bfloat16_t data type
• gfx11 WMMA now uses lane swap instead of broadcast for layout adjustment
• Updated samples GEMM parameter validation on host arch

Fixes

• Disabled GoogleTest static library deployment
• Extended tests now build in large code model

rocWMMA 1.2.0 for ROCm 5.7.0

Changes

• Fixed a synchronization bug
• Updated rocWMMA CMake versioning

rocWMMA 1.1.0 for ROCm 5.6.0

Additions

• Cross-lane operation backends (Blend, Permute, Swizzle, and Dpp)
• GPU kernels for rocWMMA unit test pre-process and post-process operations (fill, validation)
• Performance GEMM samples for half, single, and double precision
• rocWMMA CMake versioning
• Vectorized support in coordinate transforms
• Included ROCm SMI for runtime clock rate detection
• Fragment transforms for transpose and change data layout

Changes

• Default to GPU rocBLAS validation against rocWMMA
• Re-enabled int8 GEMM tests on gfx9
• Upgraded to C++17
• Restructured the unit test folder for consistency
• Consolidated rocWMMA samples common code

rocWMMA 1.0 for ROCm 5.5.0

Additions

• Support for Wave32 on gfx11+
• Infrastructure changes to support hipRTC
• Performance tracking system
• Library config to support multiple architectures
• Vector cross-lane operations support

Changes

• Modified the assignment of hardware information
• Modified data access for unsigned data types
• Refactored vector backend to be compatible with HIP_vector_type

rocWMMA 0.9 for ROCm 5.4.0

Additions

• GEMM driver APIs for flow control built-ins
• benchmark logging systems
• Restructured tests to follow naming convention; added macros for test generation

Changes

• Changed CMake to accommodate the modified test infrastructure
• Fine-tuned the multi-block kernels with and without lDs
• Adjusted maximum vector width to dWordx4
• Updated efficiencies to display as whole number percentages
• Updated throughput from GFlops/s to TFlops/s
• Reset the ad-hoc tests to use smaller sizes
• Modified the output validation to use CPU-based implementation against rocWMMA
• Modified the extended vector test to return error codes for memory allocation failures

rocWMMA 0.8 for ROCm 5.3.0

Additions

• Runtime checks to disable tests on non-target GPUS
• Workgroup-aware GEMM kernels
• Workgroup-aware validation and benchmark test suite
• Warm-up run to existing tests

Changes

• Refactored lds_mapping_util into GEMM global, local mapping, GEMM driver, GEMM config, and scheduling classes
• Modified resource allocation and tracking of GEMM and DLRM buffers
• Improved low-level data loading patterns
• Reduced branching on cooperative load and store
• Updated GEMV sample
• Updated GEMM sample

rocWMMA 0.7 for ROCm 5.2.0

Additions

• Unit tests for DLRM kernels
• GEMM sample
• DLRM sample
• SGEMV sample
• Unit tests for cooperative WMMA load and stores
• Unit tests for IOBarrier.h
• WMMA load and store tests for different matrix types (A, B, and Accumulator)
• More block sizes (1, 2, 4, 8) to test MmaSyncMultiTest
• Block sizes 4, 8 to test MmaSynMultiLdsTest
• Support for WMMA load and store layouts with a block dimension greater than 64
• IOShape structure to define the attributes of mapping and layouts for all WMMA matrix types
• CI testing for rocWMMA

Changes

• Renamed WMMA to rocWMMA in CMake, header files, and documentation
• Renamed library files
• Modified Layout.h to use different matrix offset calculations (base offset, incremental offset, and cumulative offset)
• Opaque load and store continue to use incremental offsets as they fill the entire block
• Cooperative load and store use cumulative offsets as they fill only small portions for the entire block
• Increased max split counts to 64 for cooperative load and store
• Moved all the WMMA definitions and API headers to the rocWMMA namespace
• Modified WMMA fill unit tests to validate all matrix types (A, B, Accumulator)

rocWMMA 0.6

Additions

• Unit tests for MappingUtil.h
• Unit tests for Layout.h
• Unit tests for non-native vector class in Types.h
• Unit tests for WMMA load and store contamination check
• Doxygen support for rocWMMA documentation
• MFMA barrier in IOBarrier.h
• A CMake flag to support WMMA kernel assembly code generation
• MMA sync test WMMA operation with LDS usage
• A script to generate the plots of different WMMA benchmarks
• Multi-block kernels with LDS usage
• Unit tests for multi-block WMMA kernels

Changes

• Modified GLlops calculation to accommodate multiple devices
• Removed half-types packing quirk with col major output
• Moved HIP resource management to HipResource class
• Fixed NaN errors during output comparison

rocWMMA 0.5

Additions

• Templatization for the amdgcn_convert class
• WMMA load, store, and fill support for integral data types and float64
• MFMA support for i8
• Support for bf16_1k MFMA instructions
• Code to identify the card type and its support during runtime

Changes

• Refactored and simplified IOBroadcast.h
• Modified the fragment interface compatible with NVIDIA's definition
• Modified CMake to create a lean build of the rocWMMA library

rocWMMA 0.4

Additions

• CMake support for the library and unit tests
• Integrated unit test with GoogleTest and OpenMP
• Host overload operators for hfloat16_t

Fixes

• Relative error calculation for non-integral data comparison
• Assembly generation of cooperative load and store code
• Compiler issues with new versions of ROCm

Changes

• Sped up compilation time by moving thread block sizes to function arguments instead of template parameters
• Moved all the existing unit tests to a test folder
• Moved all the header files to library/include
• Modified Layout.h to use RowNT/ColNT to eliminate LDS usage in mma_sync
• Deprecated buffer load/store and local load/store

rocWMMA 0.3

Additions

• support for the bfloat16 compute type

Changes

• Renamed __half to hfloat_16 for consistency
• Modified Convert.h to support native to bfloat16 conversion and vice versa
• Modified IOBroadCast.h to incorporate bfloat16 data packing
• Modified IOTraits.h to add bfloat16 packing traits
• Modified MFMA.h to add MFMA invocation calls to bfloat16 data
• Modified WMMA types to include bfloat16_t
• Modified the WMMA load, store, and MMA unit tests to validate bfloat16

rocWMMA 0.2

Additions

• Support for fp16 compute type
• Direct MFMA support for non-native __half data type

Changes

• Adjusted the vector storage to accommodate non-native types
• Fixed data comparison operators for fp16 data types
• Modified Convert.h to support native to __half conversion and vice versa
• Modified IOBroadCast.h to incorporate __half data packing
• Modified IOTraits.h to add __half packing traits
• Modified MFMA.h to add MFMA invocation calls to __half data
• Modified WMMA Types to include __half _t
• Modified the WMMA load, store, and MMA unit tests to validate __half

rocWMMA 0.1

Additions

• Defined a WMMA namespace with the supported matrix types, memory, and layouts
• Defined a fragment datatype to control the data transfer between HIP and MFMA
• Implemented the rocWMMA functions : load_matrix_sync, load_matrix_coop_sync, store_matrix_sync, fill_fragment, and mma_sync
• Implemented Types.h to define the supported data types
• Implemented the class IOTraits to define packing traits for the defined types as WMMA works on the packed registers
• Buffer load, store to support LLVM data instructions
• Opaque load, store
• Cooperative load, store to optimize the memory overhead
• Local load, store to perform register packing
• Implemented Convert.h to perform non-native data type conversion to native types and vice versa
• IOBroadcast class to perform packing for all input data (multiple registers)
• Implemented IOConfig to set the optimal input/output configurations for rocWMMA matrix types
• Implemented IOPack and IOUnpack to convert the unpacked device memory into packed registers and vice versa
• Layout class to define the data layout in matrix space
• MFMA to call the low-level MFMA hardware instructions
• Implemented the MappingUtil class to map from workgroup configurations to functional wave units
• Performance.h to compute GFLOPS based on hardware configurations
• Reference.h to implement the CPU GEMM operation
• Utils.h to implement matrix data operations
• rocBLASReference.h to invoke the rocBLAS GEMM function
• Unit tests to validate WMMA APIs (load, store, fill, and mma)
• Makefile support to build library and tests