TheRock/rocm-systems/shared/kpack/docs/runtime_api.md

ROCm Kpack Runtime API

Overview

The ROCm Kpack Runtime API provides a C library for loading and decompressing GPU kernels from .kpack archive files. This enables ROCm components (CLR runtime, kernel libraries) to dynamically load architecture-specific device code without embedding all variants in host binaries.

Design Decisions

Library Type

• Static or shared library via BUILD_SHARED_LIBS CMake option
• Default: Static library (librocm_kpack.a)
• Shared library (librocm_kpack.so) available with -DBUILD_SHARED_LIBS=ON
• C++17 implementation with C API for maximum compatibility

Symbol Visibility

• Hidden by default: All symbols compiled with -fvisibility=hidden on POSIX
• Explicit exports: Only public API functions marked with KPACK_API are exported
• Cross-platform: Uses __attribute__((visibility("default"))) on GCC/Clang, __declspec(dllexport/dllimport) on Windows
• Shared library: Exports only the 7 public API functions
• Static library: Compiled with hidden visibility for consistency across platforms

Dependencies

• msgpack-cxx: Header-only MessagePack library for TOC parsing
• libzstd: Zstandard decompression library
• Both are external dependencies via find_package()

Thread Safety

• Multiple threads can open different archives concurrently
• Multiple threads can query/load from same archive concurrently
• Internal locking protects shared state (decompression context, TOC cache)
• User must not call kpack_close() while other threads use the archive

Error Handling

• Fail-fast philosophy: Return errors immediately, no silent failures
• Error codes only: Functions return descriptive error codes
• No global state: No thread-local or global error strings
• OS error details: Use errno (POSIX) or GetLastError() (Windows) for I/O error details
• No partial parsing or fallback behavior on corrupted data

Kpack File Format

Archive Structure

┌─────────────────────────────────────────┐
│ Fixed Binary Header (16 bytes)         │
│  - Magic: "KPAK" (4 bytes)             │
│  - Version: uint32 (4 bytes)           │
│  - TOC Offset: uint64 (8 bytes)        │
├─────────────────────────────────────────┤
│ Padding to 64-byte boundary             │
├─────────────────────────────────────────┤
│ Blob Data (compression-scheme specific) │
│  - Zstd: [num_kernels][frame]*         │
│  - NoOp: Concatenated kernel bytes     │
├─────────────────────────────────────────┤
│ MessagePack TOC (at TOC Offset)         │
│  - Metadata + per-kernel TOC entries    │
└─────────────────────────────────────────┘

Compression Schemes

NoOp (baseline):

• Direct concatenation of .hsaco kernel files
• TOC stores byte offset and size for each kernel
• Used for testing and baseline comparison

Zstd Per-Kernel (production):

• Each kernel compressed independently with zstd level 3
• Blob structure: [num_kernels: uint32][frame_size: uint32][zstd_frame]...
• TOC stores ordinal (frame index) for each kernel
• Enables O(1) random access: decompress only requested kernel

TOC Format

MessagePack-encoded table of contents:

{
    "format_version": 1,
    "group_name": "rocm",
    "gfx_arch_family": "gfx1100",
    "gfx_arches": ["gfx1100", "gfx1101", "gfx1102"],
    "compression_scheme": "zstd-per-kernel",
    "zstd_offset": 64,
    "zstd_size": 245628,
    "toc": {
        "lib/libamdhip64.so.6": {
            "gfx1100": {"type": "hsaco", "ordinal": 0, "original_size": 138456}
        }
    },
}

API Reference

Types and Constants

// Opaque handle to an opened kpack archive
typedef struct kpack_archive* kpack_archive_t;

// Error codes
typedef enum {
    KPACK_SUCCESS = 0,
    KPACK_ERROR_INVALID_ARGUMENT,
    KPACK_ERROR_FILE_NOT_FOUND,
    KPACK_ERROR_INVALID_FORMAT,
    KPACK_ERROR_UNSUPPORTED_VERSION,
    KPACK_ERROR_KERNEL_NOT_FOUND,
    KPACK_ERROR_DECOMPRESSION_FAILED,
    KPACK_ERROR_OUT_OF_MEMORY,
    KPACK_ERROR_NOT_IMPLEMENTED,
} kpack_error_t;

Archive Lifecycle

// Open a kpack archive for reading
// Returns KPACK_SUCCESS on success, error code otherwise
kpack_error_t kpack_open(const char* path, kpack_archive_t* archive);

// Close archive and free all resources
// Must not be called while other threads are using the archive
void kpack_close(kpack_archive_t archive);

Discovery and Querying

// Get list of architectures available in archive
// Caller must free the returned array with kpack_free_string_array()
kpack_error_t kpack_get_architectures(
    kpack_archive_t archive,
    char*** arches,
    size_t* count
);

// Get list of binary names that have kernels
// Caller must free the returned array with kpack_free_string_array()
kpack_error_t kpack_get_binaries(
    kpack_archive_t archive,
    char*** binaries,
    size_t* count
);

// Free string array returned by query functions
void kpack_free_string_array(char** array, size_t count);

Kernel Loading

// Load kernel for specific binary and architecture
// Returns pointer to decompressed kernel data
// Caller must free with kpack_free_kernel()
kpack_error_t kpack_get_kernel(
    kpack_archive_t archive,
    const char* binary_name,
    const char* arch,
    void** kernel_data,
    size_t* kernel_size
);

// Free kernel data returned by kpack_get_kernel()
void kpack_free_kernel(kpack_archive_t archive, void* kernel_data);

All functions that can fail return a kpack_error_t code. For I/O errors (KPACK_ERROR_IO_ERROR), check errno on POSIX or GetLastError() on Windows for OS-level error details.

Integration Points

CLR Runtime Integration

The CLR (HIP runtime) needs to detect kpack references and lazy-load kernels:

1. Detect HIPK magic in __hipRegisterFatBinary():

   • Check for HIPK magic (vs. HIPF for fat binaries)
   • Parse .rocm_kpack_ref ELF section for kpack path

2. Lazy loading in StatCO::digestFatBinary():

   • Query current GPU architecture
   • Implement fallback logic: gfx1101 → gfx1100 → gfx11-generic
   • Call kpack_get_kernel() to load device code
   • Cache loaded kernels to avoid repeated decompression

Kernel Library Integration

Libraries like rocBLAS, rocFFT, hipBLASLt:

1. Library initialization:

   • Open kpack archive from known location
   • Query available architectures
   • Select best match for current GPU

2. Kernel selection:

   • Map problem parameters to TOC key
   • Load kernel on-demand with kpack_get_kernel()
   • Cache frequently-used kernels

3. Cleanup:

   • Free kernels with kpack_free_kernel()
   • Close archive with kpack_close() on library shutdown

Implementation Status

Phase 1: Scaffolding (Current)

• API design documented
• Directory structure created
• CMake build system
• Header files with API declarations
• Stub implementations

Phase 2: Core Functionality

• Archive opening and TOC parsing
• NoOp decompression (uncompressed archives)
• Zstd per-kernel decompression
• Error handling and reporting

Phase 3: Advanced Features

• Thread-safe operations
• Kernel caching
• Performance optimization
• Integration tests with real kpack files

Build Instructions

# Configure with default static library
cmake -B build -S /develop/rocm-kpack

# Or configure with shared library
cmake -B build -S /develop/rocm-kpack -DBUILD_SHARED_LIBS=ON

# Build
cmake --build build

# Run tests
ctest --test-dir build

Testing

Unit tests verify:

• Archive opening with valid/invalid files
• TOC parsing with various formats
• Decompression correctness
• Error handling for all failure modes
• Thread safety under concurrent access

Integration tests use real kpack files from test_assets/ created by the Python implementation.

References

• Python reference implementation: python/rocm_kpack/kpack.py
• Kpack format specification: See TOC structure above
• MessagePack spec: https://msgpack.org/
• Zstandard spec: https://github.com/facebook/zstd