TheRock/rocm-libraries/projects/hipblaslt/CHANGELOG.md

Changelog for hipBLASLt

Full documentation for hipBLASLt is available at rocm.docs.amd.com/projects/hipBLASLt.

hipBLASLt 1.4.1 for ROCm 7.14

Added

• Introduced a new API: hipBLASLt-ext::isSolutionSupported(). This API is used by new hipBLASLt integration from rocBLAS to check if a given solution is supported for a certain GPU and Problem Type.

hipBLASLt 1.4.0

Added

• Complex datatype support for gfx942 and gfx950.
• hipblasLtSetSmCountTarget() / hipblasLtGetSmCountTarget() handle-level helpers (the analogue of cuBLAS's cublasSetSmCountTarget / cublasGetSmCountTarget). int32_t, default 0 meaning "use all compute units"; negative values are rejected with HIPBLAS_STATUS_INVALID_VALUE.
• HIPBLASLT_MATMUL_DESC_SM_COUNT_TARGET matmul-descriptor attribute and HIPBLASLT_MATMUL_PREF_SM_COUNT_TARGET preference attribute (same semantics). When a per-matmul / per-preference value is non-zero it takes precedence over the handle-level value. Lets callers convey an estimate of how many CUs hipBLASLt should target for kernel selection and persistent-grid sizing — useful when another kernel (e.g. RCCL) is co-running on the device or when a persistent grid should be sized for a known CU budget. (This is a hint, not a CU reservation.)
• HIPBLASLT_MATMUL_DESC_STREAMK_TILE_SCHEDULING_EXT and matching ext API (setStreamKTileSchedulingMode() / getStreamKTileSchedulingMode()) accept the tri-state hipblasLtStreamKTileSchedulingMode_t enum (OFF=0 static SK3, ON=1 dynamic SK4, AUTO=2 origami heuristic). Invalid values return HIPBLAS_STATUS_INVALID_VALUE. hipblaslt-bench --streamk_tile_scheduling and --sm_count_target forward into the matmul descriptor (see clients/bench/README.md).

Changed

• StreamK=5 tile scheduling default is now OFF (0, static SK3 sub-path) instead of AUTO (2). Set HIPBLASLT_MATMUL_DESC_SM_COUNT_TARGET (or hipblasLtSetSmCountTarget) to a positive value to engage the origami hybrid-mode heuristic per launch even when the mode is OFF; use HIPBLASLT_STREAMK_TILE_SCHEDULING_AUTO (2) to always delegate to the heuristic regardless of sm_count_target.

hipBLASLt 1.3.0

Added

• General Batched GEMM support.
• HIPBLASLT_CHECK_NUMERICS environment variable: opt-in post-GEMM NaN scanner for hipblasLtMatmul output (D matrix). Accepts numeric (1, 2) or word values (info, warn, none/off). Output goes to the standard hipBLASLt log sink (or stderr if no log sink is configured). Per-call cost is one scanner kernel launch only -- no hipStreamSynchronize, no per-call alloc/free. A persistent 4-byte device flag is allocated once in the handle constructor and drained once at handle destruction with a single hipDeviceSynchronize; a teardown log line reports the first matmul call_id at which NaN was observed (or that none was, with a sampling caveat when the scanner only ran on a subset of calls). Companion env vars allow sampling (HIPBLASLT_CHECK_NUMERICS_SCAN_EVERY) and a bisect window (HIPBLASLT_CHECK_NUMERICS_SCAN_FROM / _SCAN_UNTIL). A C API hipblasLtCheckNumericsDrain(handle, &first_nan_call_id) lets frameworks drive a drain on demand.

Changed

• Replaced install.sh with an invoke-based task runner (tasks.py) to support cross-platform builds including Windows (ROCm 7.0+).
• gtest and msgpack-cxx are now fetched automatically via CMake FetchContent if not found on the system.
• Greatly improved MXFP4 GEMM performance when using HIPBLASLT_MATMUL_MATRIX_SCALE_BLK32_UE8M0_32_8_EXT

hipBLASLt 1.2.2 for ROCm 7.2.1

Added

• Support for AMD SMI.

Changed

• Migrate HIPBLASLT_ENABLE_LLVM to HIPBLASLT_ENABLE_YAML and synchronize with tensilelite's build library format.

Deprecation

• ROCm SMI is deprecated and dependencies are removed.

hipBLASLt 1.2.1 for ROCm 7.2.1

Resolved issues

• Fix issue where users might encounter a HIPBLAS_STATUS_INTERNAL_ERROR with various sizes in CPX mode.

hipBLASLt 1.2.0 for ROCm 7.2.0

Added

• Support for 'BF16' input with 'FP32' output data type for gfx90a.
• Support for hipBLASLtExt operation APIs on gfx11XX and gfx12XX.
• Added HIPBLASLT_MATMUL_MATRIX_SCALE_BLK32_UE8M0_32_8_EXT to support pre-swizzled block scaling data.

hipBLASLt 1.1.0 for ROCm 7.1.0

Added

• Fused Clamp GEMM for HIPBLASLT_EPILOGUE_CLAMP_EXT and HIPBLASLT_EPILOGUE_CLAMP_BIAS_EXT. This feature requires the minimum (HIPBLASLT_MATMUL_DESC_EPILOGUE_ACT_ARG0_EXT) and maximum (HIPBLASLT_MATMUL_DESC_EPILOGUE_ACT_ARG1_EXT) to be set.
• Support for ReLU/Clamp activation functions with auxiliary output for the f16 and bf16 data types for gfx942 to capture intermediate results. This feature is enabled for HIPBLASLT_EPILOGUE_RELU_AUX, HIPBLASLT_EPILOGUE_RELU_AUX_BIAS, HIPBLASLT_EPILOGUE_CLAMP_AUX_EXT, and HIPBLASLT_EPILOGUE_CLAMP_AUX_BIAS_EXT.
• Support for HIPBLAS_COMPUTE_32F_FAST_16BF for FP32 data type for gfx950 only.
• Added the CPP extension APIs setMaxWorkspaceBytes and getMaxWorkspaceBytes.
• Added the ability to print logs (using HIPBLASLT_LOG_MASK=32) for Grouped GEMM.
• Support for swizzleA by using the hipblaslt-ext cpp API.
• Support for hipBLASLt extop for gfx11xx and gfx12xx.

Changed

• hipblasLtMatmul() now returns an error when the workspace size is insufficient, rather than causing a segmentation fault.

Resolved issues

• Fix incorrect results when using ldd and ldc with some solutions

hipBLASLt 1.0.0 for ROCm 7.0.0

Added

• Stream-K GEMM support has been enabled for the FP32, FP16, BF16, FP8, and BF8 data types on the MI300A APU. To activate this feature, set the TENSILE_SOLUTION_SELECTION_METHOD environment variable to 2, for example, export TENSILE_SOLUTION_SELECTION_METHOD=2.
• Fused Swish/SiLU GEMM in hipBLASLt (enabled by HIPBLASLT_EPILOGUE_SWISH_EXT and HIPBLASLT_EPILOGUE_SWISH_BIAS_EXT)
• Added support for HIPBLASLT_EPILOGUE_GELU_AUX_BIAS for gfx942
• Added HIPBLASLT_TUNING_USER_MAX_WORKSPACE to constrain max workspace size for user offline tuning
• Added HIPBLASLT_ORDER_COL16_4R16 and HIPBLASLT_ORDER_COL16_4R8 to hipblasLtOrder_t to support FP16/BF16 swizzle GEMM and FP8/BF8 swizzle GEMM respectively.
• Added TF32 emulation on gfx950

Changed

• HIPBLASLT_MATMUL_DESC_A_SCALE_POINTER_VEC_EXT and HIPBLASLT_MATMUL_DESC_B_SCALE_POINTER_VEC_EXT are removed. Use the HIPBLASLT_MATMUL_DESC_A_SCALE_MODE and HIPBLASLT_MATMUL_DESC_B_SCALE_MODE attributes to set scalar (HIPBLASLT_MATMUL_MATRIX_SCALE_SCALAR_32F) or vector (HIPBLASLT_MATMUL_MATRIX_SCALE_OUTER_VEC_32F).
• The non-V2 APIs (GemmPreference, GemmProblemType, GemmEpilogue, GemmTuning, GemmInputs) in the Cpp header are now the same as the V2 APIs (GemmPreferenceV2, GemmProblemTypeV2, GemmEpilogueV2, GemmTuningV2, GemmInputsV2). The original non-V2 APIs are removed.
• hipblasltExtAMaxWithScale API is removed.

Optimized

• Improved performance for 8-bit (FP8/BF8/I8) NN/NT cases by adding s_delay_alu to reduce stalls from dependent ALU operations on gfx12+.
• Improved performance for 8-bit and 16-bit (FP16/BF16) TN cases by enabling software dependency check (Expert Scheduling Mode) under certain restrictions to reduce redundant hardware dependency checks on gfx12+.
• Improved performance for 8-bit, 16-bit, and 32-bit batched GEMM with a better heuristic search algorithm for gfx942.

Upcoming changes

• V2 APIs (GemmPreferenceV2, GemmProblemTypeV2, GemmEpilogueV2, GemmTuningV2, GemmInputsV2) are deprecated.

hipBLASLt 0.12.1 for ROCm 6.4.2

Added

• Support for gfx1151

hipBLASLt 0.12.1 for ROCm 6.4.1

Resolved issues

• Fixed an accuracy issue that occurred for some solutions using an FP32 or TF32 data type with a TT transpose.

hipBLASLt 0.12.0 for ROCm 6.4.0

Added

• Support roctx if HIPBLASLT_ENABLE_MARKER=1 is set
• Output the profile logging if HIPBLASLT_LOG_MASK=64 is set
• Support FP16 compute type
• Add memory bandwidth information in hipblaslt-bench output
• Support user offline tuning mechanism
• Add more samples

Changed

• Output the bench command along with solution index if HIPBLASLT_LOG_MASK=32 is set

Optimized

• Improve the overall performance of XF32/FP16/BF16/FP8/BF8 data type
• Reduce library size

Resolved issues

• Fix multi-threads bug
• Fix multi-streams bug

hipBLASLt 0.10.0 for ROCm 6.3.0

Added

• Support the V2 CPP extension API for backward compatibility
• Support for data type Int8 in with Int8 out
• Support for data type FP32/FP64 for gfx110x
• Add the Extension API hipblaslt_ext::matmulIsTuned
• Output atol and rtol for hipblaslt-bench validation
• Output the bench command for hipblaslt CPP ext API path if HIPBLASLT_LOG_MASK=32 is set
• Support odd sizes for FP8/BF8 GEMM

Changed

• Reorganize and add more sample code
• Add a dependency with the hipblas-common package and remove the dependency with the hipblas package

Optimized

• Support fused kernel for HIPBLASLT_MATMUL_DESC_AMAX_D_POINTER for FP8/BF8 data type
• Improve the library loading time
• Improve the overall performance of first returned solution

Upcoming changes

• The V1 CPP extension API will be deprecated in a future release of hipBLASLt

hipBLASLt 0.8.0

Added

• Extension APIs:
  • hipblasltExtAMaxWithScale
• GemmTuning extension parameter to set wgm by user
• Support HIPBLASLT_MATMUL_DESC_AMAX_D_POINTER for the FP8/BF8 data types
• Support for FP8/BF8 input, FP32/FP16/BF16/F8/BF8 output (only for the gfx94x architectures)
• Support HIPBLASLT_MATMUL_DESC_COMPUTE_INPUT_TYPE_A_EXT and HIPBLASLT_MATMUL_DESC_COMPUTE_INPUT_TYPE_B_EXT for FP16 input data type to use FP8/BF8 mfma
• Support for the gfx110x architecture

Optimized

• Improve the library loading time

hipBLASLt 0.7.0

Additions

• Extension APIs:
  • hipblasltExtSoftmax
  • hipblasltExtLayerNorm
  • hipblasltExtAMax
• GemmTuning extension parameter to set split-k by user
• Support for mixed-precision datatype: FP16/FP8 in with FP16 out
• Add CMake support for documentation

Deprecations

• algoGetHeuristic() ext API for GroupGemm will be deprecated in a future release of hipBLASLt

hipBLASLt 0.6.0

Additions

• New UserArguments variable for GroupedGemm
• Support for datatype: FP16 in with FP32 out
• Support for datatype: Int8 in Int32 out
• Support for gfx94x platform
• Support for FP8/BF8 datatype (only for gfx94x platform)
• Support scalar A,B,C,D for FP8/BF8 datatype
• Added samples

Changes

• Replaced hipblasDatatype_t with hipDataType
• Replaced hipblasLtComputeType_t with hipblasComputeType_t

Removals

• Deprecated HIPBLASLT_MATMUL_DESC_D_SCALE_VECTOR_POINTER

hipBLASLt 0.3.0

Additions

• Added getAllAlgos extension APIs
• TensileLite support for new epilogues: gradient gelu, gradient D, gradient A/B, aux
• Added a sample package that includes three sample apps
• Added a new C++ GEMM class in the hipBLASLt extension

Changes

• Refactored GroupGemm APIs as C++ class in the hipBLASLt extension
• Changed the scaleD vector enum to HIPBLASLT_MATMUL_DESC_D_SCALE_VECTOR_POINTER

Fixes

• Enabled norm check validation for CI

Optimizations

• GSU kernel: wider memory, PGR N
• Updated logic yaml to improve some FP16 NN sizes
• GroupGemm support for GSU kernel
• Added grouped GEMM tuning for aldebaran

hipBLASLt 0.2.0

Additions

• Added CI tests for TensileLite
• Initialized extension group GEMM APIs (FP16 only)
• Added a group GEMM sample app: example_hipblaslt_groupedgemm

Fixes

• Fixed incorrect results for the ScaleD kernel

Optimizations

• Tuned equality sizes for the HHS data type
• Reduced host-side overhead for hipblasLtMatmul()
• Removed unused kernel arguments
• Schedule values setup before first s_waitcnt
• Refactored TensileLite host codes
• Optimized build time

hipBLASLt 0.1.0

Additions

• Enabled hipBLASLt APIs
• Support for gfx90a
• Support for problem type: FP32, FP16, BF16
• Support activation: relu, gelu
• Support for bias vectors
• Integrated with TensileLite kernel generator
• Added Gtest: hipblaslt-test
• Added the full function tool hipblaslt-bench
• Added the sample app example_hipblaslt_preference

Optimizations

• gridBase solution search algorithm for untuned size
• Tuned 10k sizes for each problem type