# hipblaslt-bench test ``` # Go to hipBLASLt build directory cd hipBLASLt; cd build/release # run hipblaslt-bench ./clients/hipblaslt-bench --help ./clients/hipblaslt-bench [ --data | --yaml ] ... --sizem |-m Specific matrix size: the number of rows or columns in matrix. (Default value is: 128) --sizen |-n Specific matrix the number of rows or columns in matrix (Default value is: 128) --sizek |-k Specific matrix size: the number of columns in A and rows in B. (Default value is: 128) --lda Leading dimension of matrix A. --ldb Leading dimension of matrix B. --ldc Leading dimension of matrix C. --ldd Leading dimension of matrix D. --lde Leading dimension of matrix E. --any_stride Do not modify input strides based on leading dimensions --stride_a Specific stride of strided_batched matrix A, second dimension * leading dimension. --stride_b Specific stride of strided_batched matrix B, second dimension * leading dimension. --stride_c Specific stride of strided_batched matrix C, second dimension * leading dimension. --stride_d Specific stride of strided_batched matrix D, second dimension * leading dimension. --stride_e Specific stride of strided_batched matrix E, second dimension * leading dimension. --alpha specifies the scalar alpha (Default value is: 1) --beta specifies the scalar beta (Default value is: 0) --function |-f BLASLt function to test. Options: matmul (Default value is: matmul) --precision |-r Precision of matrix A,B,C,D Options: f32_r, f16_r, bf16_r, f64_r, i32_r, i8_r, f32_c, f64_c (Default value is: f16_r) --a_type Precision of matrix A. Options: f32_r, f64_r, i32_r, f16_r, bf16_r, i8_r, f32_c, f64_c --b_type Precision of matrix B. Options: f32_r, f64_r, i32_r, f16_r, bf16_r, i8_r, f32_c, f64_c --c_type Precision of matrix C. Options: f32_r, f64_r, i32_r, f16_r, bf16_r, i8_r, f32_c, f64_c --d_type Precision of matrix D. Options: f32_r, f64_r, i32_r, f16_r, bf16_r, i8_r, f32_c, f64_c --compute_type Precision of computation. Options: s,f32_r,x,xf32_r,f64_r,i32_r (Default value is: f32_r) --compute_input_typeA Options: f32_r, f16_r, bf16_r, f8_r, bf8_r, f8_fnuz_r, bf8_fnuz_r, The default value indicates that the argument has no effect. (Default value is: INVALID) --compute_input_typeB Options: f32_r, f16_r, bf16_r, f8_r, bf8_r, f8_fnuz_r, bf8_fnuz_r, The default value indicates that the argument has no effect. (Default value is: INVALID) --scale_type Precision of scalar. Options: f16_r,bf16_r, f32_c, f64_c --initialization Initialize matrix data.Options: rand_int, trig_float, hpl(floating), special, zero, norm_dist, uniform_01 (Default value is: hpl) --transA N = no transpose, T = transpose, C = conjugate (Default value is: N) --transB N = no transpose, T = transpose, C = conjugate (Default value is: N) --batch_count Number of matrices. Only applicable to batched and strided_batched routines (Default value is: 1) --HMM Parameter requesting the use of HipManagedMemory --verify |-v Validate GPU results with CPU? --iters |-i Iterations to run inside timing loop (Default value is: 10) --cold_iters |-j Cold Iterations to run before entering the timing loop (Default value is: 2) --adaptive Enable adaptive timing; not compatible with --iters / --cold_iters --adaptive_warmup_time Warm up (ms) until this much wall-time has elapsed; requires --adaptive --adaptive_sample_time Wall-time (ms) of each timed sample = back-to-back batch size (>= measure_time gives one batch); requires --adaptive --adaptive_measure_time Minimum total measurement time (ms); requires --adaptive --adaptive_max_measure_time Measurement ceiling (ms, 0 = unbounded); requires --adaptive --adaptive_min_iters Floor on total timed iterations; requires --adaptive --adaptive_max_iters Ceiling on total timed iterations (0 = unbounded); requires --adaptive --adaptive_noise_threshold Convergence target: past the floor, keep running until the mean's relative standard error drops below this fraction (converged), or the robust spread plateaus (stable), or the ceiling is hit (noisy); 0 disables both (run to the ceiling); requires --adaptive --adaptive_stability_threshold Noise-plateau fallback: stop with status=stable once the recent rel_iqr readings vary by less than this fraction; 0 disables the fallback (a non-converging run then goes to the ceiling, status=noisy); requires --adaptive --adaptive_stability_window Number of recent rel_iqr readings the stability fallback tests for a plateau (>= 2); requires --adaptive --adaptive_stability_interval Record one rel_iqr reading for the stability fallback every N samples (>= 1); requires --adaptive (adaptive defaults are shown by --help / -h) --algo_method Use different algorithm search API. Options: heuristic, all, index. (Default value is: heuristic) --solution_index Used with --algo_method 2. Specify solution index to use in benchmark. (Default value is: -1) --requested_solution Requested solution num. Set to -1 to get all solutions. Only valid when algo_method is set to heuristic. (Default value is: 1) --activation_type Options: None, gelu, relu, swish, clamp (Default value is: none) --activation_arg1 First extra argument for activation function if needed. (Default value is: 0) --activation_arg2 Second extra argument for activation function if neeeded. (Default value is: inf) --bias_type Precision of bias vector.Options: f16_r,bf16_r,f32_r,default(same with D type) --bias_source Choose bias source: a, b, d (Default value is: d) --bias_vector Apply bias vector --scaleA Apply scale for A buffer --scaleB Apply scale for B buffer --scaleAlpha_vector Apply scaleAlpha vector --amaxScaleA Apple scale for A buffer by abs max of A buffer --amaxScaleB Apple scale for B buffer by abs max of B buffer --use_e Apply AUX output/ gradient input --gradient Enable gradient --grouped_gemm Use grouped_gemm. --use_user_args Use UserArguments located in device memory for grouped gemm. --device Set default device to be used for subsequent program runs (Default value is: 0) --c_equal_d C and D are stored in same memory --workspace Set fixed workspace memory size instead of using hipblaslt managed memory (Default value is: 0) --log_function_name Function name precedes other items. --function_filter Simple strstr filter on function name only without wildcards --api_method Use extension API. c: C style API. mix: declaration with C hipblasLtMatmul Layout/Desc but set, initialize, and run the problem with C++ extension API. cpp: Using C++ extension API only. Options: c, mix, cpp. (Default value is: c) --print_kernel_info Print solution, kernel name and solution index. --rotating Use rotating memory blocks for each iteration, size in MB. (Default value is: 0) --use_gpu_timer Use hipEventElapsedTime to profile elapsed time. (Default value is: false) --splitk [Tuning parameter] Set split K for a solution, 0 is use solution's default value. (Only support GEMM + api_method mix or cpp) --wgm [Tuning parameter] Set workgroup mapping for a solution, 0 is use solution's default value. (Only support GEMM + api_method mix or cpp) --flush Flush icache --sm_count_target Target compute-unit (CU) count for the matmul kernel selection and persistent-grid sizing. 0 (default) means use all CUs the device exposes. Negative values are rejected. (Default value is: 0) --streamk_tile_scheduling Select the StreamK=5 tile scheduling sub-path via the HIPBLASLT_MATMUL_DESC_STREAMK_TILE_SCHEDULING_EXT extension attribute. Accepts off|0 (SK3 static; default when unset), on|1 (force SK4 dynamic), auto|2 (always run origami heuristic), case-insensitive. Omit to leave the attribute unset so the library default (off) applies; combine with `--sm_count_target` > 0 to engage the heuristic without setting auto|2. (Default value is: unset) --help |-h produces this help message --version Prints the version number ``` # demo Run fp32 GEMM with validation ``` ./clients/hipblaslt-bench --precision f32_r -v [0]:transA,transB,grouped_gemm,batch_count,m,n,k,alpha,lda,stride_a,beta,ldb,stride_b,ldc,stride_c,ldd,stride_d,a_type,b_type,c_type,d_type,compute_type,scaleA,scaleB,scaleC,scaleD,amaxD,activation_type,bias_vector,bias_type,hipblaslt-Gflops,hipblaslt-GB/s,us,CPU-Gflops,CPU-us,norm_error,atol,rtol N,N,0,1,128,128,128,1,128,16384,0,128,16384,128,16384,128,16384,f32_r,f32_r,f32_r,f32_r,f32_r,0,0,0,0,0,none,0,non-supported type,403.298,17.6063,10.4,0.0960059,43688,2.44879e-07,1e-05,1e-05 ``` Show the frequency with environment variable ``` HIPBLASLT_BENCH_FREQ=1 ./clients/hipblaslt-bench -m 16 -n 16 -k 4096 --transA T --transB N --a_type bf16_r --b_type bf16_r --c_type bf16_r --d_type bf16_r --activation_type none --compute_type f32_r [0]:transA,transB,grouped_gemm,batch_count,m,n,k,alpha,lda,stride_a,beta,ldb,stride_b,ldc,stride_c,ldd,stride_d,a_type,b_type,c_type,d_type,compute_type,scaleA,scaleB,scaleC,scaleD,amaxD,swizzle_a,swizzle_b,activation_type,bias_vector,bias_type,aux_type,rotating_buffer,flush,use_gpu_timer,lowest_avg_freq,lowest_median_freq,avg_MCLK,median_MCLK,hipblaslt-Gflops,hipblaslt-GB/s,us T,N,0,1,16,16,4096,1,4096,65536,0,4096,65536,16,256,16,256,bf16_r,bf16_r,bf16_r,bf16_r,f32_r,0,0,0,0,0,0,0,none,0,bf16_r,bf16_r,0,0,0,191,191,2000,2000,275.941,32.1865,7.6 ``` Show the multi-XCD frequencies with environment variable ``` HIPBLASLT_BENCH_FREQ_ALL=1 ./clients/hipblaslt-bench -m 16 -n 16 -k 4096 --transA T --transB N --a_type bf16_r --b_type bf16_r --c_type bf16_r --d_type bf16_r --activation_type none --compute_type f32_r [0]:transA,transB,grouped_gemm,batch_count,m,n,k,alpha,lda,stride_a,beta,ldb,stride_b,ldc,stride_c,ldd,stride_d,a_type,b_type,c_type,d_type,compute_type,scaleA,scaleB,scaleC,scaleD,amaxD,swizzle_a,swizzle_b,activation_type,bias_vector,bias_type,aux_type,rotating_buffer,flush,use_gpu_timer,avg_freq0,avg_freq1,avg_freq2,avg_freq3,avg_freq4,avg_freq5,avg_freq6,avg_freq7,median_freq0,median_freq1,median_freq2,median_freq3,median_freq4,median_freq5,median_freq6,median_freq7,avg_MCLK,median_MCLK,hipblaslt-Gflops,hipblaslt-GB/s,us T,N,0,1,16,16,4096,1,4096,65536,0,4096,65536,16,256,16,256,bf16_r,bf16_r,bf16_r,bf16_r,f32_r,0,0,0,0,0,0,0,none,0,bf16_r,bf16_r,0,0,0,223,205,221,217,209,211,191,191,223,205,221,217,209,211,191,191,2000,2000,249.661,29.1211,8.4 ``` Show the efficiency and other performance related args with environment variable ``` HIPBLASLT_BENCH_PERF=1 ./clients/hipblaslt-bench -m 4096 -n 4864 -k 32896 --transA N --transB N --a_type bf16_r --b_type bf16_r --c_type bf16_r --d_type bf16_r --compute_type f32_r --iters 416 --cold_iters 416 --use_gpu_timer [0]:transA,transB,grouped_gemm,batch_count,m,n,k,alpha,lda,stride_a,beta,ldb,stride_b,ldc,stride_c,ldd,stride_d,a_type,b_type,c_type,d_type,compute_type,scaleA,scaleB,scaleC,scaleD,amaxD,swizzle_a,swizzle_b,activation_type,bias_vector,bias_type,aux_type,rotating_buffer,flush,use_gpu_timer,num_cu,tiles_per_cu,tile0_gran,tile1_gran,cu_gran,wave_gran,total_gran,mem_read_bytes,mem_write_bytes,lowest_avg_freq,lowest_median_freq,avg_MCLK,median_MCLK,efficiency,hipblaslt-Gflops,hipblaslt-GB/s,us N,N,0,1,4096,4864,32896,1,4096,134742016,0,32896,160006144,4096,19922944,4096,19922944,bf16_r,bf16_r,bf16_r,bf16_r,f32_r,0,0,0,0,0,0,0,none,0,bf16_r,bf16_r,0,0,1,256,0,0.984615,1,-nan,1,-nan,13312511180,119537664,1119,1205,2000,2000,92.3889,1.08405e+06,484.741,1209.14 ``` ## Adaptive timing By default the benchmark runs `--cold_iters` warmups followed by a single timed loop of `--iters` enqueues and reports the mean. Pass **`--adaptive`** to instead take a distribution-based measurement that is robust to noise and lets fast kernels accumulate more iterations automatically — without precomputing iteration counts per problem. The fixed-count default is unchanged, so existing commands behave exactly as before. A *sample* is a batch of back-to-back iterations run with no synchronization between them, timed only at its start and end (so the sample's per-iteration time is that span divided by the batch size). The batch is sized from a quick warmup so each sample spans roughly `--adaptive_sample_time` ms. The run lasts **at least** the floor — `--adaptive_min_iters` and `--adaptive_measure_time` — and **at most** the ceiling — `--adaptive_max_measure_time` and `--adaptive_max_iters`. Past the floor, it stops on the first of: the mean's relative standard error (stddev / mean / sqrt(n)) falling below `--adaptive_noise_threshold` (**converged**, `status=converged`); or, when that target cannot be met, the robust spread (IQR / median) flattening out so more samples will not change the result (**stable**, `status=stable`) — a fallback for heavy-tailed or unlocked-clock kernels; otherwise the ceiling (`status=noisy`). The stable fallback is tuned by `--adaptive_stability_window` / `--adaptive_stability_interval` and can be turned off with `--adaptive_stability_threshold 0` (a non-converging run then runs to the ceiling and reports `noisy`). With `--adaptive_noise_threshold 0` both the convergence check and the fallback are disabled, so the run goes to the ceiling and `status` is `-`. A ceiling is required — at least one of `--adaptive_max_measure_time` / `--adaptive_max_iters` must be > 0 (it is by default), otherwise the run is rejected — so a non-converging run is always bounded. (To run a fixed budget, set the floor and ceiling equal, e.g. `--adaptive_measure_time 200 --adaptive_max_measure_time 200`.) The reported `us`/`Gflops` are the **median**; extra columns expose the distribution: `batch`, `samples`, `hot_iters`, `mean_us`, `min_us`, `cv` (stddev/mean), `rel_iqr` (interquartile range / median — a drift-robust dispersion measure), and `status` (`converged` / `stable` / `noisy` / `-`). Convergence requires at least 10 samples so the stddev is trustworthy. `--adaptive` is the gate; the `--adaptive_*` options tune it and **require** `--adaptive` (passing one without it is an error). Run `--help` for their current default values. Under `--adaptive`, warmup and per-sample batch size are determined adaptively, so `--iters` and `--cold_iters` have no effect and **passing either with `--adaptive` is an error** (use `--adaptive_warmup_time` for warmup and `--adaptive_sample_time` to influence the batch). Without `--adaptive`, timing is the fixed-count `--cold_iters` / `--iters` path and no distribution columns are printed. Note on the statistic: the **median** is reported as the headline because it is robust to the occasional slow sample (OS/power jitter, clock dips) that inflates the mean on an unlocked GPU. The **mean** is kept as a column for comparison and matches the fixed-count number exactly (where median, mean and min coincide). ``` ./clients/hipblaslt-bench -m 4096 -n 4096 -k 4096 --transA N --transB T \ --a_type bf16_r --b_type bf16_r --c_type bf16_r --d_type bf16_r --compute_type f32_r \ --rotating 512 --adaptive ```