TheRock/rocm-libraries/projects/rocwmma/CHANGELOG.md

# Changelog for rocWMMA

Documentation for rocWMMA is available at
[https://rocm.docs.amd.com/projects/rocWMMA/en/latest](https://rocm.docs.amd.com/projects/rocWMMA/en/latest).

## Since last release (ROCm 7.12)

### Added

* Added the following community samples for external contributions, with build support and documentation
  * `simple_gemm_silu`: demonstrates a GEMM + SiLU fused operator using the rocWMMA API
  * `simple_gemm_fusion`: demonstrates block-tile-level dual-GEMM fusion using the rocWMMA API
  * `simple_gemm_swiglu`: demonstrates a SwiGLU fused dual-GEMM kernel (LLaMA/Mistral FFN gate layer) using the rocWMMA API

### Changed

* Updated the `find_package` search for OpenMP to prefer the `openmp-config.cmake` provided by ROCm, with a fallback to module search mode.
* Updated `INSTALL_RPATH` and added `BUILD_RPATH` for OpenMP

### Resolved issues

* Improved HIP RTC regression test portability when deployed outside the default path

## rocWMMA 2.2.0 for ROCm 7.2.0

### Added

* Added sample `perf_i8gemm` to demonstrate `int8_t` as matrix input data type
* Added support for the gfx1150 target

### Changed

* Removed unnecessary const keyword to avoid compiler warnings
* rocWMMA has been moved into the new rocm-libraries "monorepo" repository (https://github.com/ROCm/rocm-libraries). This repository consolidates a number of separate ROCm libraries and shared components.
  * The repository migration requires a few changes to the CMake configuration of rocWMMA
  * The repository migration rquired the GTest dependency to be updated to v1.16.0

### Resolved issues

* Skip invalid test configurations when using 'register file' LDS mapping
* Ensured transform functions in samples are only available on the device

## rocWMMA 2.1.0 for ROCm 7.1.1

### Added

* Added more unit tests to increase the code coverage.

### Changed

* Increased compile timeout and improved visualization in `math-ci`.

### Removed

* Removed absolute paths from the `RPATH` of sample and test binary files.

### Resolved issues

* Fixed issues caused by HIP changes:
    * Removed the .data member from HIP_vector_type.
    * Broadcast constructor now only writes to the first vector element.
* Fixed a bug related to `int32_t` usage in `hipRTC_gemm` for gfx942, caused by breaking changes in HIP.
* Replaced `#pragma unroll` with `static for` to fix a bug caused by the upgraded compiler which no longer supports using `#pragma unroll` with template parameter indices.
* Corrected test predicates for `BLK` and `VW` cooperative kernels.
* Modified `compute_utils.sh` in `build-infra` to ensure rocWMMA is built with gfx1151 target for ROCm 7.0 and beyond.

## rocWMMA 2.0.0 for ROCm 7.0.0

### Added

* Added internal register layout transforms to support interleaved MMA layouts
* Added support for the gfx950 target
* Added mixed input `bf8` / `fp8` types for MMA support
* Added fragment scheduler API objects to embed thread block cooperation properties in fragments

### Changed

* Augmented load / store / MMA internals with static loop unrolling
* rocWMMA mma_sync API now supports `wave tile` fragment sizes
* rocWMMA cooperative fragments are now expressed with fragment scheduler template arguments
* rocWMMA cooperative fragments now use the same base API as non-cooperative fragments
* rocWMMA cooperative fragments register usage footprint has been reduced
* rocWMMA fragments now support partial tile sizes with padding

### Optimized

* Added internal flow control barriers to improve assembly code generation and overall performance
* Enabled interleaved layouts by default in MMA to improve overall performance

### Removed

* Removed support for the gfx940 and gfx941 targets
* Removed the rocWMMA cooperative API
* Removed wave count template parameters from transforms APIs

### Resolved issues

* Fixed a validation issue for small precision compute types `< B32` on gfx9
* Fixed CMake validation of compiler support for `bf8` / `fp8` types
* Fixed linkage of rocwmma::synchronize_workgroup to inline

## rocWMMA 1.7.0 for ROCm 6.4.0

### Added

* Added interleaved layouts that enhance the performance of GEMM operations
* Added emulation test suites. These suites are lightweight and well-suited for execution on emulator platforms

### Changed

* Used GPU_TARGETS instead of AMDGPU_TARGETS in `cmakelists.txt`
* The binary sizes can be reduced on supported compilers by using the `--offload-compress` compiler flag

### Resolved issues

* For a CMake bug workaround, set `CMAKE_NO_BUILTIN_CHRPATH` when `BUILD_OFFLOAD_COMPRESS` is unset

## rocWMMA 1.6.0 for ROCm 6.3.0

### Added

* Added OCP f8/bf8 datatype support
* Added support for gfx12 architecture targets

### Changed

* Optimized some aos<->soa transforms with half-rotation offsets
* Refactored the rocBLAS reference entry point for validation and benchmarking
* ROCWMMA_* preprocessor configurations are now all assigned values
* Updated the default architecture targets for ASAN builds
* Updated the actor-critic implementation

### Resolved issues

* Fixed a bug in f64 validation due to faulty typecasting
* Fixed a bug causing runtime compilation errors with hipRTC
* Various documentation updates and fixes

### Upcoming changes

* rocWMMA 2.x plans to augment fragment API objects with additional meta-properties which will improve API expressiveness and configurability of parameters including multiple-wave cooperation. As part of this change, cooperative rocWMMA API functions `load_matrix_coop_sync` and `store_matrix_coop_sync` will become deprecated in a future ROCm release.

## rocWMMA 1.5.0 for ROCm 6.2.0

### Additions

* Added internal utilities for element-wise vector transforms
* Added internal utilities for cross-lane vector transforms
* Implemented internal aos<->soa transforms for block sizes of 16, 32, 64, 128 and 256 and vector widths of 2, 4, 8 and 16
* Added tests for new internal transforms

### Changes

* Improved loading layouts by increasing vector width for fragments with blockDim > 32
* API applyDataLayout transform now accepts WaveCount template argument for cooperative fragments
* API applyDataLayout transform now physically applies aos<->soa transform as necessary
* Refactored entry-point of std library usage to improve hipRTC support
* Documentation updates for installation, programmer's guide and API reference

### Fixes

* Fixed some header includes ordering to improve portability

## rocWMMA 1.4.0 for ROCm 6.1.0

### Additions

* Added bf16 support for hipRTC sample

### Changes

* Changed Clang C++ version to C++17
* Updated rocwmma_coop API
* Linked rocWMMA to hiprtc

### Fixes

* Fixed compile/runtime arch checks
* Built all test in large code model
* Removed inefficient branching in layout loop unrolling

## rocWMMA 1.3.0 for ROCm 6.0.0

### Additions

* Support for gfx940, gfx941, and gfx942 targets
* Support for f8, bf8, and xfloat32 data types
* support for `HIP_NO_HALF`, `__ HIP_NO_HALF_CONVERSIONS__`, and
    `__ HIP_NO_HALF_OPERATORS__` (e.g., PyTorch environment)

### Changes

* rocWMMA with hipRTC now supports `bfloat16_t` data type
* gfx11 WMMA now uses lane swap instead of broadcast for layout adjustment
* Updated samples GEMM parameter validation on host arch

### Fixes

* Disabled GoogleTest static library deployment
* Extended tests now build in large code model

## rocWMMA 1.2.0 for ROCm 5.7.0

### Changes

* Fixed a synchronization bug
* Updated rocWMMA CMake versioning

## rocWMMA 1.1.0 for ROCm 5.6.0

### Additions

* Cross-lane operation backends (Blend, Permute, Swizzle, and Dpp)
* GPU kernels for rocWMMA unit test pre-process and post-process operations (fill, validation)
* Performance GEMM samples for half, single, and double precision
* rocWMMA CMake versioning
* Vectorized support in coordinate transforms
* Included ROCm SMI for runtime clock rate detection
* Fragment transforms for transpose and change data layout

### Changes

* Default to GPU rocBLAS validation against rocWMMA
* Re-enabled int8 GEMM tests on gfx9
* Upgraded to C++17
* Restructured the unit test folder for consistency
* Consolidated rocWMMA samples common code

## rocWMMA 1.0 for ROCm 5.5.0

### Additions

* Support for Wave32 on gfx11+
* Infrastructure changes to support hipRTC
* Performance tracking system
* Library config to support multiple architectures
* Vector cross-lane operations support

### Changes

* Modified the assignment of hardware information
* Modified data access for unsigned data types
* Refactored vector backend to be compatible with `HIP_vector_type`

## rocWMMA 0.9 for ROCm 5.4.0

### Additions

* GEMM driver APIs for flow control built-ins
* benchmark logging systems
* Restructured tests to follow naming convention; added macros for test generation

### Changes

* Changed CMake to accommodate the modified test infrastructure
* Fine-tuned the multi-block kernels with and without lDs
* Adjusted maximum vector width to dWordx4
* Updated efficiencies to display as whole number percentages
* Updated throughput from GFlops/s to TFlops/s
* Reset the ad-hoc tests to use smaller sizes
* Modified the output validation to use CPU-based implementation against rocWMMA
* Modified the extended vector test to return error codes for memory allocation failures

## rocWMMA 0.8 for ROCm 5.3.0

### Additions

* Runtime checks to disable tests on non-target GPUS
* Workgroup-aware GEMM kernels
* Workgroup-aware validation and benchmark test suite
* Warm-up run to existing tests

### Changes

* Refactored `lds_mapping_util` into GEMM global, local mapping, GEMM driver, GEMM config, and
  scheduling classes
* Modified resource allocation and tracking of GEMM and DLRM buffers
* Improved low-level data loading patterns
* Reduced branching on cooperative load and store
* Updated GEMV sample
* Updated GEMM sample

## rocWMMA 0.7 for ROCm 5.2.0

### Additions

* Unit tests for DLRM kernels
* GEMM sample
* DLRM sample
* SGEMV sample
* Unit tests for cooperative WMMA load and stores
* Unit tests for `IOBarrier.h`
* WMMA load and store tests for different matrix types (A, B, and Accumulator)
* More block sizes (1, 2, 4, 8) to test `MmaSyncMultiTest`
* Block sizes 4, 8 to test `MmaSynMultiLdsTest`
* Support for WMMA load and store layouts with a block dimension greater than 64
* IOShape structure to define the attributes of mapping and layouts for all WMMA matrix types
* CI testing for rocWMMA

### Changes

* Renamed WMMA to rocWMMA in CMake, header files, and documentation
* Renamed library files
* Modified `Layout.h` to use different matrix offset calculations (base offset, incremental offset, and
  cumulative offset)
* Opaque load and store continue to use incremental offsets as they fill the entire block
* Cooperative load and store use cumulative offsets as they fill only small portions for the entire block
* Increased max split counts to 64 for cooperative load and store
* Moved all the WMMA definitions and API headers to the rocWMMA namespace
* Modified WMMA fill unit tests to validate all matrix types (A, B, Accumulator)

## rocWMMA 0.6

### Additions

* Unit tests for `MappingUtil.h`
* Unit tests for `Layout.h`
* Unit tests for non-native vector class in `Types.h`
* Unit tests for WMMA load and store contamination check
* Doxygen support for rocWMMA documentation
* MFMA barrier in `IOBarrier.h`
* A CMake flag to support WMMA kernel assembly code generation
* MMA sync test WMMA operation with LDS usage
* A script to generate the plots of different WMMA benchmarks
* Multi-block kernels with LDS usage
* Unit tests for multi-block WMMA kernels

### Changes

* Modified GLlops calculation to accommodate multiple devices
* Removed half-types packing quirk with col major output
* Moved HIP resource management to `HipResource` class
* Fixed NaN errors during output comparison

## rocWMMA 0.5

### Additions

* Templatization for the `amdgcn_convert` class
* WMMA load, store, and fill support for integral data types and float64
* MFMA support for i8
* Support for `bf16_1k` MFMA instructions
* Code to identify the card type and its support during runtime

### Changes

* Refactored and simplified `IOBroadcast.h`
* Modified the fragment interface compatible with NVIDIA's definition
* Modified CMake to create a lean build of the rocWMMA library

## rocWMMA 0.4

### Additions

* CMake support for the library and unit tests
* Integrated unit test with GoogleTest and OpenMP
* Host overload operators for `hfloat16_t`

### Fixes

* Relative error calculation for non-integral data comparison
* Assembly generation of cooperative load and store code
* Compiler issues with new versions of ROCm

### Changes

* Sped up compilation time by moving thread block sizes to function arguments instead of template
  parameters
* Moved all the existing unit tests to a `test` folder
* Moved all the header files to `library/include`
* Modified `Layout.h` to use RowNT/ColNT to eliminate LDS usage in `mma_sync`
* Deprecated buffer load/store and local load/store

## rocWMMA 0.3

### Additions

* support for the bfloat16 compute type

### Changes

* Renamed `__half` to `hfloat_16` for consistency
* Modified `Convert.h` to support native to bfloat16 conversion and vice versa
* Modified `IOBroadCast.h` to incorporate bfloat16 data packing
* Modified `IOTraits.h` to add bfloat16 packing traits
* Modified `MFMA.h` to add MFMA invocation calls to bfloat16 data
* Modified WMMA types to include `bfloat16_t`
* Modified the WMMA load, store, and MMA unit tests to validate bfloat16

## rocWMMA 0.2

### Additions

* Support for fp16 compute type
* Direct MFMA support for non-native `__half` data type

### Changes

* Adjusted the vector storage to accommodate non-native types
* Fixed data comparison operators for fp16 data types
* Modified `Convert.h` to support native to `__half` conversion and vice versa
* Modified `IOBroadCast.h` to incorporate `__half` data packing
* Modified `IOTraits.h` to add `__half` packing traits
* Modified `MFMA.h` to add MFMA invocation calls to `__half` data
* Modified WMMA Types to include `__half _t`
* Modified the WMMA load, store, and MMA unit tests to validate `__half`

## rocWMMA 0.1

### Additions

* Defined a WMMA namespace with the supported matrix types, memory, and layouts
* Defined a fragment datatype to control the data transfer between HIP and MFMA
* Implemented the rocWMMA functions : `load_matrix_sync`, `load_matrix_coop_sync`,
  `store_matrix_sync`, `fill_fragment`, and `mma_sync`
* Implemented `Types.h` to define the supported data types
* Implemented the class `IOTraits` to define packing traits for the defined types as WMMA works on the
  packed registers
* Buffer load, store to support LLVM data instructions
* Opaque load, store
* Cooperative load, store to optimize the memory overhead
* Local load, store to perform register packing
* Implemented `Convert.h` to perform non-native data type conversion to native types and vice versa
* `IOBroadcast` class to perform packing for all input data (multiple registers)
* Implemented `IOConfig` to set the optimal input/output configurations for rocWMMA matrix types
* Implemented `IOPack` and `IOUnpack` to convert the unpacked device memory into packed registers
  and vice versa
* `Layout` class to define the data layout in matrix space
* MFMA to call the low-level MFMA hardware instructions
* Implemented the `MappingUtil` class to map from workgroup configurations to functional wave units
* `Performance.h` to compute GFLOPS based on hardware configurations
* `Reference.h` to implement the CPU GEMM operation
* `Utils.h` to implement matrix data operations
* `rocBLASReference.h` to invoke the rocBLAS GEMM function
* Unit tests to validate WMMA APIs (`load`, `store`, `fill`, and `mma`)
* Makefile support to build library and tests