This directory contains six HPC training examples covering GPU-accelerated scientific computing patterns: Jacobi iterative solvers (HIP, Fortran+OpenMP), matrix exponentiation with GPU streams, and hardware queue stream overlap. These examples demonstrate key HPC concepts including iterative PDE solving, halo exchange communication, GPU stream concurrency, and compute-communication overlap.

Sub-Examples

jacobi-hip

GPU-accelerated 2D Jacobi iterative solver using HIP. Performs halo exchange between subdomains, launches Jacobi iteration and Laplacian norm kernels, and demonstrates multi-GPU domain decomposition patterns.

Source files: JacobiMain.hip, JacobiIteration.hip, JacobiSetup.hip, JacobiRun.hip, HaloExchange.hip, Laplacian.hip, Norm.hip, Input.hip, defines.hpp, Jacobi.hpp, markers.h

Dependencies: HIP, roctracer (for markers)

jacobi-fortran-targetdata-markers

Fortran Jacobi solver using OpenMP target data directives with profiling markers for GPU offloading analysis.

Source files: main.f90, jacobi.f90, laplacian.f90, boundary.f90, mesh.f90, norm.f90, update.f90, input.f90, kind.f90

Dependencies: Fortran compiler with OpenMP target support

jacobi-fortran-usm

Fortran Jacobi solver using Unified Shared Memory (USM) for automatic data migration between host and device. Requires a GPU with XNACK support — see OpenMP USM documentation for details.

Source files: Same structure as jacobi-fortran-targetdata-markers with USM modifications.

Dependencies: Fortran compiler with OpenMP USM support, GPU with XNACK support (HSA_XNACK=1)

matrix-exponential-streams-sync-hip

Computes a matrix exponential via truncated Taylor series using HIP and rocBLAS, with one GPU stream per OpenMP thread for concurrent DGEMM operations. Demonstrates GPU stream-level parallelism.

Source files: mat-exp.cpp

Dependencies: HIP, rocBLAS, OpenMP, rocprofiler-sdk-roctx (for markers)

Parameters: N=20 truncation terms, t=0.5 evaluation point, 4 OpenMP threads (= 4 HIP streams)

split-copy-compute-hw-queues

Demonstrates compute-communication overlap using separate hardware queues for data transfers and kernel execution, maximizing GPU utilization through pipelined operations.

Source files: compute-comm-overlap.hip

Dependencies: HIP

julia

Julia language vector addition example demonstrating GPU profiling of Julia workloads.

Source files: vecadd.jl

Dependencies: Julia runtime, AMDGPU package (julia -e 'import Pkg; Pkg.add("AMDGPU")')

Prerequisites

CMake 3.25+
HIP runtime and hipcc compiler (for HIP examples)
rocBLAS (for matrix exponential)
Fortran compiler with OpenMP target support (for Fortran examples)
Julia runtime (for Julia example)

Building

Standalone build:

cmake -B <build_dir> -S <project_root>/examples/hpc -DCMAKE_PREFIX_PATH=/opt/rocm
cmake --build <build_dir>

As part of the examples suite:

cmake -B <build_dir> -S <project_root>/examples/ -DCMAKE_PREFIX_PATH=/opt/rocm
cmake --build <build_dir> <target_name>

NOTE: Individual sub-examples can be built by target name as defined in their respective CMakeLists.txt files.

Profiling with rocprofiler-systems

# Profile Jacobi HIP solver
rocprof-sys-run -- ./jacobi-hip

# Profile matrix exponential
rocprof-sys-run -- ./matrix-exponential-streams-sync-hip

Recommended Configuration

Variable	Value	Purpose
`ROCPROFSYS_ROCM_DOMAINS`	`hip_runtime_api,kernel_dispatch,memory_copy`	Trace HIP API and GPU operations
`ROCPROFSYS_TRACE`	`true`	Generate Perfetto trace for stream overlap analysis
`ROCPROFSYS_PROFILE`	`true`	Generate call-stack profile

rocprof-sys-run \
    -e ROCPROFSYS_ROCM_DOMAINS=hip_runtime_api,kernel_dispatch,memory_copy \
    -e ROCPROFSYS_TRACE=true \
    -- ./jacobi-hip