derek.holloway
7502bf64a4
CodeQL / Analyze (actions) (push) Canceled after 0s
CodeQL / Analyze (python) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Multi-Arch CI / setup (push) Canceled after 0s
Publish build_manylinux_x86_64 images / publish_build_manylinux_x86_64 (push) Canceled after 0s
Setup (Multi-Arch) / setup (push) Canceled after 0s
Multi-Arch CI / Linux::${{ fromJSON(needs.setup.outputs.linux_build_config || '{}').build_variant_label || 'skip' }} (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish no_rocm_image_ubi10 image / publish_no_rocm_image_ubi10 (push) Canceled after 0s
Multi-Arch CI / Windows::${{ fromJSON(needs.setup.outputs.windows_build_config || '{}').build_variant_label || 'skip' }} (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Multi-Arch CI / Manifest Diff (push) Canceled after 0s
TheRock Manifest Diff Report / Generate Manifest Diff Report (push) Canceled after 0s
Multi-Arch CI / CI Summary (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04 (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04_ocl_rt images / publish_no_rocm_image_ubuntu24_04_ocl_rt (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_rocgdb (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04_rocgdb images / publish_no_rocm_image_ubuntu24_04_rocgdb (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_ocl_rt (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Unit Tests / Unit Tests :: ubuntu-24.04 (push) Canceled after 0s
Unit Tests / Unit Tests :: windows-2022 (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_media (push) Canceled after 0s
Unit Tests / Unit Tests Summary (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_openmpi (push) Canceled after 0s
32 lines
1.7 KiB
Markdown
32 lines
1.7 KiB
Markdown
# NCCL EP (Expert Parallelism) - Release Notes
|
||
NCCL EP is a high-performance NCCL API extension for efficient Mixture-of-Experts (MoE) communication.
|
||
It provides optimized dispatch and combine primitives for Expert Parallelism (EP) across distributed GPU systems
|
||
implemented on top of NCCL Device API: Load-Store Accessible (LSA) and GPU-Initiated Networking (GIN) operations.
|
||
|
||
|
||
## Known Limitations
|
||
|
||
This release has the following limitations:
|
||
|
||
| Limitation | Description |
|
||
|----------------------------------|-------------|
|
||
| **Limited QA coverage** | This release is considered experimental and had limited Quality Assurance (QA) testing |
|
||
| **No FP8 support** | FP8 data types are not currently supported |
|
||
| **Fabric selection** | Current version of API does not provide a way to force RDMA-only communication |
|
||
| **Up to 8 nodes** | Maximum of 64 GPUs (8 nodes × 8 GPUs) is supported |
|
||
| **Up to 8 ranks per node** | Only up to 8 GPUs per node is supported |
|
||
| **max_tokens_per_rank required** | `NCCL_EP_AUTO` is not yet supported; `max_tokens_per_rank` must be set to the per-rank batch size (max tokens any single rank will dispatch) |
|
||
| **Max Num Tokens Support** | Currently implementation of HT kernels is using a define variable configured in `common.h` as `MAX_SUPPORTED_TOKENS_PER_RANK` |
|
||
| **HT mode performance** | HT mode is currently being tuned for performance and may demonstrate sub-optimal results in this release |
|
||
|
||
|
||
## References
|
||
|
||
- [NCCL Documentation](https://docs.nvidia.com/deeplearning/nccl/)
|
||
- [GPU-Initiated Networking Paper](https://arxiv.org/abs/2511.15076)
|
||
- [NCCL EP Readme](README.md)
|
||
|
||
## License
|
||
|
||
See LICENSE.txt for license information.
|