derek.holloway
7502bf64a4
CodeQL / Analyze (actions) (push) Canceled after 0s
CodeQL / Analyze (python) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Multi-Arch CI / setup (push) Canceled after 0s
Publish build_manylinux_x86_64 images / publish_build_manylinux_x86_64 (push) Canceled after 0s
Setup (Multi-Arch) / setup (push) Canceled after 0s
Multi-Arch CI / Linux::${{ fromJSON(needs.setup.outputs.linux_build_config || '{}').build_variant_label || 'skip' }} (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish no_rocm_image_ubi10 image / publish_no_rocm_image_ubi10 (push) Canceled after 0s
Multi-Arch CI / Windows::${{ fromJSON(needs.setup.outputs.windows_build_config || '{}').build_variant_label || 'skip' }} (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Multi-Arch CI / Manifest Diff (push) Canceled after 0s
TheRock Manifest Diff Report / Generate Manifest Diff Report (push) Canceled after 0s
Multi-Arch CI / CI Summary (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04 (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04_ocl_rt images / publish_no_rocm_image_ubuntu24_04_ocl_rt (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_rocgdb (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04_rocgdb images / publish_no_rocm_image_ubuntu24_04_rocgdb (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_ocl_rt (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Unit Tests / Unit Tests :: ubuntu-24.04 (push) Canceled after 0s
Unit Tests / Unit Tests :: windows-2022 (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_media (push) Canceled after 0s
Unit Tests / Unit Tests Summary (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_openmpi (push) Canceled after 0s
184 lines
14 KiB
ReStructuredText
184 lines
14 KiB
ReStructuredText
.. meta::
|
|
:description: A guide to memory allocation in rocBLAS
|
|
:keywords: rocBLAS, ROCm, API, Linear Algebra, documentation, memory
|
|
|
|
.. _memory-alloc:
|
|
.. _Device Memory Allocation Usage:
|
|
|
|
********************************************************************
|
|
Device memory allocation in rocBLAS
|
|
********************************************************************
|
|
|
|
For temporary device memory, rocBLAS uses a per-handle memory allocation with out-of-band management.
|
|
For more information, see the device memory allocation section of the :ref:programmers-guide.
|
|
|
|
The following computational functions use temporary device memory.
|
|
|
|
+------------------------------------------------+------------------------------------------------------+
|
|
|Function |Use of temporary device memory |
|
|
+================================================+======================================================+
|
|
|L1 reduction functions | Reduction array |
|
|
| | |
|
|
| - rocblas_Xasum | |
|
|
| - rocblas_Xasum_batched | |
|
|
| - rocblas_Xasum_strided_batched | |
|
|
| - rocblas_Xdot | |
|
|
| - rocblas_Xdot_batched | |
|
|
| - rocblas_Xdot_strided_batched | |
|
|
| - rocblas_Xmax | |
|
|
| - rocblas_Xmax_batched | |
|
|
| - rocblas_Xmax_strided_batched | |
|
|
| - rocblas_Xmin | |
|
|
| - rocblas_Xmin_batched | |
|
|
| - rocblas_Xmin_strided_batched | |
|
|
| - rocblas_Xnrm2 | |
|
|
| - rocblas_Xnrm2_batched | |
|
|
| - rocblas_Xnrm2_strided_batched | |
|
|
| - rocblas_dot_ex | |
|
|
| - rocblas_dot_batched_ex | |
|
|
| - rocblas_dot_strided_batched_ex | |
|
|
| - rocblas_nrm2_ex | |
|
|
| - rocblas_nrm2_batched_ex | |
|
|
| - rocblas_nrm2_strided_batched_ex | |
|
|
+------------------------------------------------+------------------------------------------------------+
|
|
|L2 functions | Result array before overwriting input |
|
|
| | |
|
|
| - rocblas_Xgemv (optional) | Column reductions of skinny transposed matrices |
|
|
| - rocblas_Xgemv_batched | applicable for ``gemv`` functions |
|
|
| - rocblas_Xgemv_strided_batched | |
|
|
| - rocblas_Xtbmv | |
|
|
| - rocblas_Xtbmv_batched | |
|
|
| - rocblas_Xtbmv_strided_batched | |
|
|
| - rocblas_Xtpmv | |
|
|
| - rocblas_Xtpmv_batched | |
|
|
| - rocblas_Xtpmv_strided_batched | |
|
|
| - rocblas_Xtrmv | |
|
|
| - rocblas_Xtrmv_batched | |
|
|
| - rocblas_Xtrmv_strided_batched | |
|
|
| - rocblas_Xtrsv | |
|
|
| - rocblas_Xtrsv_batched | |
|
|
| - rocblas_Xtrsv_strided_batched | |
|
|
| - rocblas_Xhemv | |
|
|
| - rocblas_Xhemv_batched | |
|
|
| - rocblas_Xhemv_strided_batched | |
|
|
| - rocblas_Xsymv | |
|
|
| - rocblas_Xsymv_batched | |
|
|
| - rocblas_Xsymv_strided_batched | |
|
|
| - rocblas_Xtrsv_ex | |
|
|
| - rocblas_Xtrsv_batched_ex | |
|
|
| - rocblas_Xtrsv_strided_batched_ex | |
|
|
+------------------------------------------------+------------------------------------------------------+
|
|
|L3 GEMM-based functions | Block of matrix |
|
|
| | |
|
|
| - rocblas_Xtrsm | |
|
|
| - rocblas_Xtrsm_batched | |
|
|
| - rocblas_Xtrsm_strided_batched | |
|
|
| - rocblas_Xsymm | |
|
|
| - rocblas_Xsymm_batched | |
|
|
| - rocblas_Xsymm_strided_batched | |
|
|
| - rocblas_Xsyrk | |
|
|
| - rocblas_Xsyrk_batched | |
|
|
| - rocblas_Xsyrk_strided_batched | |
|
|
| - rocblas_Xsyr2k | |
|
|
| - rocblas_Xsyr2k_batched | |
|
|
| - rocblas_Xsyr2k_strided_batched | |
|
|
| - rocblas_Xsyrkx | |
|
|
| - rocblas_Xsyrkx_batched | |
|
|
| - rocblas_Xsyrkx_strided_batched | |
|
|
| - rocblas_Xtrmm | |
|
|
| - rocblas_Xtrmm_batched | |
|
|
| - rocblas_Xtrmm_strided_batched | |
|
|
| - rocblas_Xhemm | |
|
|
| - rocblas_Xhemm_batched | |
|
|
| - rocblas_Xhemm_strided_batched | |
|
|
| - rocblas_Xherk | |
|
|
| - rocblas_Xherk_batched | |
|
|
| - rocblas_Xherk_strided_batched | |
|
|
| - rocblas_Xher2k | |
|
|
| - rocblas_Xher2k_batched | |
|
|
| - rocblas_Xher2k_strided_batched | |
|
|
| - rocblas_Xherkx | |
|
|
| - rocblas_Xherkx_batched | |
|
|
| - rocblas_Xherkx_strided_batched | |
|
|
| - rocblas_Xgemm | |
|
|
| - rocblas_Xgemm_batched | |
|
|
| - rocblas_Xgemm_strided_batched | |
|
|
| - rocblas_gemm_ex | |
|
|
| - rocblas_gemm_ex_batched | |
|
|
| - rocblas_gemm_ex_strided_batched | |
|
|
| - rocblas_Xtrtri | |
|
|
| - rocblas_Xtrtri_batched | |
|
|
| - rocblas_Xtrtri_strided_batched | |
|
|
+------------------------------------------------+------------------------------------------------------+
|
|
|
|
.. _rocblas_device_memory_size:
|
|
|
|
Environment variable for preallocating memory
|
|
=============================================
|
|
|
|
The environment variable ``ROCBLAS_DEVICE_MEMORY_SIZE`` is used to set how much memory to preallocate:
|
|
|
|
* If it is greater than 0, it sets the default handle device memory size to the specified size (in bytes).
|
|
* If it is equal to 0 or unset, it lets rocBLAS manage the device memory. It uses a default size, like 32MiB or 128MiB, and expands it when necessary.
|
|
|
|
Memory allocation functions
|
|
==============================================
|
|
|
|
rocBLAS includes functions for manually setting the memory size and determining the memory requirements.
|
|
|
|
Functions for manually setting the memory size
|
|
----------------------------------------------
|
|
|
|
* ``rocblas_set_device_memory_size``
|
|
* ``rocblas_get_device_memory_size``
|
|
* ``rocblas_is_user_managing_device_memory``
|
|
|
|
Function for setting a user-owned workspace
|
|
----------------------------------------------
|
|
|
|
* ``rocblas_set_workspace``
|
|
|
|
Functions for determining memory requirements
|
|
----------------------------------------------
|
|
|
|
* ``rocblas_start_device_memory_size_query``
|
|
* ``rocblas_stop_device_memory_size_query``
|
|
* ``rocblas_is_managing_device_memory``
|
|
|
|
See the API section for information about these functions.
|
|
|
|
rocBLAS function return values for insufficient device memory
|
|
=============================================================
|
|
|
|
If the user preallocates or manually allocates, that size is used as the limit and no resizing or synchronizing ever occurs.
|
|
The following two function return values indicate insufficient memory:
|
|
|
|
* ``rocblas_status == rocblas_status_memory_error`` : indicates there is insufficient device memory for a rocBLAS function.
|
|
* ``rocblas_status == rocblas_status_perf_degraded`` : indicates that a slower algorithm was used because of insufficient device memory for the optimal algorithm.
|
|
|
|
.. _stream order alloc:
|
|
|
|
Stream-ordered memory allocation
|
|
========================================
|
|
|
|
Stream-ordered device memory allocation is added to rocBLAS. The asynchronous allocators
|
|
``hipMallocAsync()`` and ``hipFreeAsync()`` are used to allow allocation and deallocation to happen in stream order.
|
|
|
|
This is a non-default beta option that can be enabled by setting the environment variable ``ROCBLAS_STREAM_ORDER_ALLOC``.
|
|
|
|
To check whether the device supports stream-order allocation, call ``hipDeviceGetAttribute()`` with the
|
|
device attribute ``hipDeviceAttributeMemoryPoolsSupported``.
|
|
|
|
Enabling stream-ordered memory allocation
|
|
----------------------------------------------
|
|
|
|
On supported platforms, the environment variable ``ROCBLAS_STREAM_ORDER_ALLOC`` is used to enable stream-ordered memory allocation.
|
|
|
|
* If it is greater than 0 (``> 0``), it sets the allocation to be stream-ordered and uses ``hipMallocAsync/hipFreeAsync`` to manage device memory.
|
|
* If it is equal to zero (``= 0``) or unset, it uses ``hipMalloc`` and ``hipFree`` to manage device memory.
|
|
|
|
Switching streams without synchronization
|
|
----------------------------------------------
|
|
|
|
Stream-order memory allocation lets the application switch streams without having to call ``hipStreamSynchronize()``.
|