derek.holloway
7502bf64a4
CodeQL / Analyze (actions) (push) Canceled after 0s
CodeQL / Analyze (python) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Multi-Arch CI / setup (push) Canceled after 0s
Publish build_manylinux_x86_64 images / publish_build_manylinux_x86_64 (push) Canceled after 0s
Setup (Multi-Arch) / setup (push) Canceled after 0s
Multi-Arch CI / Linux::${{ fromJSON(needs.setup.outputs.linux_build_config || '{}').build_variant_label || 'skip' }} (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish no_rocm_image_ubi10 image / publish_no_rocm_image_ubi10 (push) Canceled after 0s
Multi-Arch CI / Windows::${{ fromJSON(needs.setup.outputs.windows_build_config || '{}').build_variant_label || 'skip' }} (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Multi-Arch CI / Manifest Diff (push) Canceled after 0s
TheRock Manifest Diff Report / Generate Manifest Diff Report (push) Canceled after 0s
Multi-Arch CI / CI Summary (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04 (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04_ocl_rt images / publish_no_rocm_image_ubuntu24_04_ocl_rt (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_rocgdb (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04_rocgdb images / publish_no_rocm_image_ubuntu24_04_rocgdb (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_ocl_rt (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Unit Tests / Unit Tests :: ubuntu-24.04 (push) Canceled after 0s
Unit Tests / Unit Tests :: windows-2022 (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_media (push) Canceled after 0s
Unit Tests / Unit Tests Summary (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_openmpi (push) Canceled after 0s
52 lines
2.7 KiB
ReStructuredText
52 lines
2.7 KiB
ReStructuredText
.. _libcudacxx-extended-api-memory-resources:
|
|
|
|
Memory Resources
|
|
================
|
|
|
|
.. toctree::
|
|
:hidden:
|
|
:maxdepth: 1
|
|
|
|
memory_resource/properties
|
|
Resources <memory_resource/resource>
|
|
Resource wrapper <memory_resource/resource_ref>
|
|
|
|
The ``<cuda/memory_resource>`` header provides a standard C++ interface for *heterogeneous*, *stream-ordered* memory
|
|
allocation tailored to the needs of CUDA C++ developers. This design builds off of the success of the `RAPIDS Memory Manager (RMM) <https://github.com/rapidsai/rmm>`__
|
|
project and evolves the design based on lessons learned.
|
|
|
|
``<cuda/memory_resource>`` is not intended to replace RMM, but instead moves the definition of the memory allocation
|
|
interface to a more centralized home in CCCL. RMM will remain as a collection of implementations of the ``cuda::mr``
|
|
interfaces.
|
|
|
|
We are still experimenting with the design, so for now the contents of ``<cuda/memory_resource>`` are only available if
|
|
``LIBCUDACXX_ENABLE_EXPERIMENTAL_MEMORY_RESOURCE`` is defined.
|
|
|
|
At a high level, the header provides:
|
|
|
|
.. list-table::
|
|
:widths: 30 50 20
|
|
:header-rows: 0
|
|
|
|
* - :ref:`cuda::get_property <libcudacxx-extended-api-memory-resources-properties>`
|
|
- Infrastructure to tag a user defined type with a given property
|
|
- CCCL 2.2.0 / CUDA 12.3
|
|
* - :ref:`cuda::mr::{async}_resource <libcudacxx-extended-api-memory-resources-resource>` and
|
|
:ref:`cuda::mr::{async}_resource_with <libcudacxx-extended-api-memory-resources-resource>`
|
|
- Concepts that provide proper constraints for arbitrary memory resources.
|
|
- CCCL 2.2.0 / CUDA 12.3
|
|
* - :ref:`cuda::mr::{async}_resource_ref <libcudacxx-extended-api-memory-resources-resource-ref>`
|
|
- A non-owning type-erased memory resource wrapper that enables consumers to specify properties of resources that they expect.
|
|
- CCCL 2.2.0 / CUDA 12.3
|
|
|
|
These features are an evolution of `std::pmr::memory_resource <https://en.cppreference.com/w/cpp/header/memory_resource>`__
|
|
that was introduced in C++17. While ``std::pmr::memory_resource`` provides a polymorphic memory resource that can be
|
|
adopted through inheritance, it is not properly suited for heterogeneous systems.
|
|
|
|
With the current design it ranges from cumbersome to impossible to verify whether a memory resource provides allocations
|
|
that are e.g. accessible on device, or whether it can utilize other allocation mechanisms.
|
|
|
|
To better support asynchronous CUDA `stream-ordered allocations <https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#stream-ordered-memory-allocator>`__
|
|
libcu++ provides :ref:`cuda::stream_ref <libcudacxx-extended-api-streams-stream-ref>` as a wrapper around
|
|
``cudaStream_t``. The definition of ``cuda::stream_ref`` can be found in the ``<cuda/stream_ref>`` header.
|