derek.holloway
7502bf64a4
CodeQL / Analyze (actions) (push) Canceled after 0s
CodeQL / Analyze (python) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Multi-Arch CI / setup (push) Canceled after 0s
Publish build_manylinux_x86_64 images / publish_build_manylinux_x86_64 (push) Canceled after 0s
Setup (Multi-Arch) / setup (push) Canceled after 0s
Multi-Arch CI / Linux::${{ fromJSON(needs.setup.outputs.linux_build_config || '{}').build_variant_label || 'skip' }} (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish no_rocm_image_ubi10 image / publish_no_rocm_image_ubi10 (push) Canceled after 0s
Multi-Arch CI / Windows::${{ fromJSON(needs.setup.outputs.windows_build_config || '{}').build_variant_label || 'skip' }} (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Multi-Arch CI / Manifest Diff (push) Canceled after 0s
TheRock Manifest Diff Report / Generate Manifest Diff Report (push) Canceled after 0s
Multi-Arch CI / CI Summary (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04 (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04_ocl_rt images / publish_no_rocm_image_ubuntu24_04_ocl_rt (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_rocgdb (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04_rocgdb images / publish_no_rocm_image_ubuntu24_04_rocgdb (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_ocl_rt (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Unit Tests / Unit Tests :: ubuntu-24.04 (push) Canceled after 0s
Unit Tests / Unit Tests :: windows-2022 (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_media (push) Canceled after 0s
Unit Tests / Unit Tests Summary (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_openmpi (push) Canceled after 0s
34 lines
1.8 KiB
Markdown
34 lines
1.8 KiB
Markdown
# ROCm OpenSHMEM (rocSHMEM)
|
|
|
|
The ROCm OpenSHMEM (rocSHMEM) runtime is part of an AMD and AMD Research
|
|
initiative to provide GPU-centric networking through an OpenSHMEM-like interface.
|
|
This intra-kernel networking library simplifies application
|
|
code complexity and enables more fine-grained communication/computation
|
|
overlap than traditional host-driven networking.
|
|
rocSHMEM uses a single symmetric heap that is allocated on GPU memories.
|
|
|
|
There are currently three backends for rocSHMEM;
|
|
IPC, Reverse Offload (RO), and GDA.
|
|
The backends primarily differ in their implementations of intra-kernel networking.
|
|
|
|
The IPC backend implements communication primitives using load/store operations issued from the GPU.
|
|
|
|
The Reverse Offload (RO) backend has the GPU runtime forward rocSHMEM networking operations
|
|
to the host-side runtime, which calls into a traditional MPI or OpenSHMEM
|
|
implementation. This forwarding of requests is transparent to the
|
|
programmer, who only sees the GPU-side interface.
|
|
|
|
The GPU Direct Async (GDA) backend allows for rocSHMEM to issue communication operations to the NIC directly from the device-side code, without involving a CPU proxy.
|
|
within the GPU.
|
|
During initialization we prepare network resources for each NIC vendor using the vendor-appropriate
|
|
Direct Verbs APIs.
|
|
When calling the device-side rocSHMEM API, the device threads are used to construct Work Queue Entries (WQEs) and post the communication to the send queues of the NIC directly.
|
|
Completion Queues (CQs) are polled from the device-side code as well.
|
|
|
|
The RO and GDA backend is provided as-is with limited support from AMD or AMD Research.
|
|
|
|
## Installation and using rocSHMEM
|
|
|
|
For information on how to install and use rocSHMEM,
|
|
[please see our documentation](https://rocm.docs.amd.com/projects/rocSHMEM/en/latest/).
|