derek.holloway
7502bf64a4
CodeQL / Analyze (actions) (push) Canceled after 0s
CodeQL / Analyze (python) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Multi-Arch CI / setup (push) Canceled after 0s
Publish build_manylinux_x86_64 images / publish_build_manylinux_x86_64 (push) Canceled after 0s
Setup (Multi-Arch) / setup (push) Canceled after 0s
Multi-Arch CI / Linux::${{ fromJSON(needs.setup.outputs.linux_build_config || '{}').build_variant_label || 'skip' }} (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish no_rocm_image_ubi10 image / publish_no_rocm_image_ubi10 (push) Canceled after 0s
Multi-Arch CI / Windows::${{ fromJSON(needs.setup.outputs.windows_build_config || '{}').build_variant_label || 'skip' }} (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Multi-Arch CI / Manifest Diff (push) Canceled after 0s
TheRock Manifest Diff Report / Generate Manifest Diff Report (push) Canceled after 0s
Multi-Arch CI / CI Summary (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04 (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04_ocl_rt images / publish_no_rocm_image_ubuntu24_04_ocl_rt (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_rocgdb (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04_rocgdb images / publish_no_rocm_image_ubuntu24_04_rocgdb (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_ocl_rt (push) Canceled after 0s
Publish TheRock Docker image / build-and-push-image (push) Canceled after 0s
Unit Tests / Unit Tests :: ubuntu-24.04 (push) Canceled after 0s
Unit Tests / Unit Tests :: windows-2022 (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_media (push) Canceled after 0s
Unit Tests / Unit Tests Summary (push) Canceled after 0s
Publish no_rocm_image_ubuntu24_04 images / publish_no_rocm_image_ubuntu24_04_openmpi (push) Canceled after 0s
55 lines
2.5 KiB
Plaintext
55 lines
2.5 KiB
Plaintext
/*
|
|
* Portions of this file are adapted from DeepEP (https://github.com/deepseek-ai/DeepEP).
|
|
* Copyright (c) 2025 DeepSeek. Licensed under the MIT License.
|
|
* SPDX-License-Identifier: MIT
|
|
*/
|
|
/*
|
|
* SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
|
* SPDX-License-Identifier: Apache-2.0
|
|
* See LICENSE.txt for more license information.
|
|
*/
|
|
|
|
#pragma once
|
|
|
|
// ============================================================================
|
|
// HT-specific configuration constants
|
|
// ============================================================================
|
|
#define HYBRIDEP_MAX_NUM_SMS_PER_RANK 16
|
|
// ============================================================================
|
|
// Dispatch configuration constants
|
|
// ============================================================================
|
|
#define HYBRIDEP_DISPATCH_NUM_OF_STAGES 12
|
|
#define HYBRIDEP_DISPATCH_NUM_OF_IN_FLIGHT_S2G 4
|
|
#define HYBRIDEP_DISPATCH_NUM_OF_BLOCKS HYBRIDEP_MAX_NUM_SMS_PER_RANK
|
|
#define HYBRIDEP_DISPATCH_NUM_OF_PIPELINES_PER_BLOCK 2
|
|
#define HYBRIDEP_DISPATCH_N2N_WARPS 2
|
|
// Maximum consecutive tokens batched into a single RDMA put in dispatch N2N.
|
|
// Larger batches reduce NIC doorbell overhead but may delay first-byte latency.
|
|
#define HYBRIDEP_DISPATCH_RDMA_BATCH_SIZE 4
|
|
|
|
|
|
// ============================================================================
|
|
// Combine configuration constants
|
|
// ============================================================================
|
|
// Single-node configuration: optimized for intra-node only (2 pipelines, deep FIFO)
|
|
#define HYBRIDEP_COMBINE_SINGLENODE_NUM_OF_STAGES_G2S 12
|
|
#define HYBRIDEP_COMBINE_SINGLENODE_NUM_OF_STAGES_S2G 2
|
|
|
|
// Multi-node configuration: optimized for inter-node RDMA (1 pipeline, shallow FIFO)
|
|
#define HYBRIDEP_COMBINE_MULTINODE_NUM_OF_STAGES_G2S 4
|
|
#define HYBRIDEP_COMBINE_MULTINODE_NUM_OF_STAGES_S2G 2
|
|
|
|
#define HYBRIDEP_COMBINE_NUM_OF_TOKENS_PER_GROUP 4
|
|
#define HYBRIDEP_COMBINE_NUM_OF_BLOCKS HYBRIDEP_MAX_NUM_SMS_PER_RANK
|
|
#define HYBRIDEP_COMBINE_NUM_OF_ADDITIONAL_IN_FLIGHT_S2G 2
|
|
|
|
// Streaming overlap: tokens between drain+signal from reduction warp to RDMA warp.
|
|
// 0 = disable streaming (fall back to chunk-level mbarrier only).
|
|
#define HYBRIDEP_COMBINE_RDMA_STREAMING_BATCH 8
|
|
|
|
// ============================================================================
|
|
// Preprocessing kernel configuration
|
|
// ============================================================================
|
|
#define HYBRIDEP_NUM_THREADS_PER_BLOCK_PREPROCESSING 512
|
|
#define HYBRIDEP_NUM_BLOCKS_PREPROCESSING HYBRIDEP_MAX_NUM_SMS_PER_RANK
|