derek.holloway
7502bf64a4
hipFile examples
This directory contains working examples of the hipFile API, grouped by what
they demonstrate. The basics and async programs verify their results with
an FNV-1a hash and print OK … on success.
Most examples move data through the GPU on hipFile's fast path, which opens
files with O_DIRECT; running them as written therefore requires an AMD GPU
supported by ROCm and source/destination paths on an O_DIRECT-capable local
filesystem (ext4 mounted data=ordered, or xfs). O_DIRECT is not a hipFile
requirement, though — files can be registered without it and routed through the
POSIX compat path (see basics/no-odirect-write). Verify fast-path support with
/opt/rocm/bin/ais-check. The api/ examples are the exception — they only
query the library and need neither a GPU nor an O_DIRECT filesystem.
The directories
| Directory | What's in it |
|---|---|
api |
Minimal examples of the non-I/O API — calls that query or configure the library (e.g. get-version). No O_DIRECT filesystem or file arguments needed. |
basics |
Small, single-purpose programs that each exercise one facet of the synchronous API: buffer registration, the O_DIRECT fast path vs. compat fallback, chunked reads, device-buffer offsets, sub-region writes, GPU memory types, and a full round trip. |
async |
Examples of the asynchronous, stream-based API (hipFileReadAsync / hipFileWriteAsync), including single-stream, non-blocking-stream, and concurrent multi-stream round trips. |
aiscp |
A standalone cp-like utility built on hipFile (aiscp SOURCE DEST). |
common |
Not an example — a small static library (examples_common) of helpers shared by basics and async (alignment math, pattern fill, hashing, file open/register). |
Building
Each example directory contains a ready-to-use CMakeLists.txt that finds the
installed hipFile via find_package. To build a directory, configure and build
it in place:
cd basics
cmake -B build -DCMAKE_PREFIX_PATH="/opt/rocm;/path/to/hipfile"
cmake --build build --parallel
This is an out-of-source build, so the example binaries land in build/. The
run snippets below assume you have cd build'd into that directory first.
Drop the -DCMAKE_PREFIX_PATH argument if ROCm and hipFile are installed in
standard locations. If the install tree is read-only, copy the directory (and,
for basics/async, the sibling common directory) somewhere
writable first.
basics and async link the shared helpers in common and pull it
in via add_subdirectory(../common …), so keep the common directory alongside
them when you build.
Finding the library at runtime
When you build an example, CMake records the location of libhipfile.so (taken
from the same find_package result used above) in the example binary, so it
runs in place even when hipFile is installed outside the default library search
path — no LD_LIBRARY_PATH needed.
That recorded path is fixed at build time. If you later move the hipFile
installation, the binaries can no longer find the library and fail to start with
error while loading shared libraries: libhipfile.so.0. Point the loader at the
new location to recover (or rebuild the examples):
cd build
LD_LIBRARY_PATH=/new/path/to/hipfile/lib ./bufregister-write out.bin
Per-example payload and chunk sizes are compile-time #defines (e.g.
-DBRW_SIZE=…, -DIR_CHUNK_SIZE=…, documented at the top of each .cpp). To
build an example with different sizes, pass the corresponding -D flag at
configure time.
api
Minimal examples of the non-I/O parts of the hipFile API — calls that query or
configure the library rather than move data through the GPU. These do not
require an O_DIRECT-capable filesystem or even file arguments.
| Program | What it shows | Args |
|---|---|---|
get-version |
Read the hipFile version both ways: the HIPFILE_VERSION_* header macros (compile-time) and hipFileGetVersion() (runtime). |
none |
Running
cd build
./get-version
Prints the version from the header symbols and from the runtime call. No file or GPU memory is touched.
basics
Small, single-purpose programs that each exercise one facet of the hipFile C
API: buffer registration, the O_DIRECT fast path vs. compat fallback, chunked
reads, device buffer offsets, sub-region writes, the three GPU memory types, and
a full round trip. They drive the API directly from main() and use the shared
helpers in common (open_file, hash_file_range, align_up,
fill_pattern, …). Every example verifies its result with an FNV-1a hash and
prints OK … on success.
| Program | What it shows | Args |
|---|---|---|
bufregister-write |
Write a GPU buffer registered with hipFileBufRegister straight to disk (the fast path). |
OUTPUT [GPUID] |
no-bufregister-write |
Same write, but without registering the buffer — hipFile copies through its internal pool. | OUTPUT [GPUID] |
no-odirect-write |
Register a file opened without O_DIRECT. On an O_DIRECT-capable filesystem hipFile transparently reopens it with O_DIRECT and still takes the fast path; only on a filesystem that can't do O_DIRECT does it use the POSIX compat fallback (on by default — set HIPFILE_ALLOW_COMPAT_MODE=true only if you previously disabled it). |
OUTPUT [GPUID] |
iterative-read |
Chunked read into GPU memory where the host pointer advances each iteration, then one write. | INPUT OUTPUT [GPUID] |
iterative-devmem-offset-read |
Same chunked read, but the base device pointer is fixed and the buf_offset argument advances. |
INPUT OUTPUT [GPUID] |
subregion-write |
Read a whole file, then write only the bytes at/after an offset using a non-zero buffer_offset. |
INPUT OUTPUT [GPUID] |
various-mem-rw |
Round-trip a file using device (1), managed (2), or pinned-host (3) memory as the transfer buffer. |
INPUT OUTPUT MODE [GPUID] |
roundtrip-verify |
Write a known pattern, read it back through the GPU, write a copy, and assert both files hash-match. | CREATED COPIED [GPUID] |
GPUID is optional and defaults to 0. Payload/chunk sizes are compile-time
#defines (e.g. -DBRW_SIZE=…, -DIR_CHUNK_SIZE=…) documented at the top of
each .cpp.
Running
Reads need an existing input file; create one with dd:
cd build
dd if=/dev/urandom of=input.bin bs=1M count=1
The input and output paths must live on an O_DIRECT-capable local
filesystem (ext4 mounted data=ordered, or xfs); verify with
/opt/rocm/bin/ais-check:
./bufregister-write out_bufregister.bin
./no-bufregister-write out_no_bufregister.bin
./no-odirect-write out_no_odirect.bin # hipFile reopens with O_DIRECT; no env var needed
./iterative-read input.bin out_iter.bin
./iterative-devmem-offset-read input.bin out_iter_off.bin
./subregion-write input.bin out_subregion.bin
./various-mem-rw input.bin out_vmrw.bin 1 # 1=device 2=managed 3=pinned
./roundtrip-verify rtv_created.bin rtv_copied.bin
async
Examples of hipFile's asynchronous, stream-based API
(hipFileReadAsync / hipFileWriteAsync). Each one seeds an input file with a
deterministic pattern, issues a GPU-mediated read+write round trip on one or
more HIP streams, synchronizes, and verifies the output by FNV-1a hash. They
share the helpers in common and print OK … on success.
Note: The
O_DIRECTfast path is not currently supported for asynchronous I/O — async operations always run through the POSIX compat (fallback) path, regardless of whether the file or filesystem isO_DIRECT-capable. Fast-path async support is planned for the future.
| Program | What it shows |
|---|---|
roundtrip-async |
Async read + write on the default stream, a single hipStreamSynchronize, then verify. |
roundtrip-async-nonblocking-stream |
Same round trip on an explicit hipStreamNonBlocking stream (no implicit sync with the legacy default stream). |
roundtrip-async-multi-stream |
NUM_STREAMS read/write pairs run concurrently, each on its own non-blocking stream covering a distinct file slice. |
roundtrip-async-multi-stream-registered |
Same concurrent multi-stream run, but each stream is registered with hipFileStreamRegister (fixed-offset / fixed-size / page-aligned hints) so the driver skips per-submission validation. |
All four take the same arguments:
PROGRAM READ_FILE WRITE_FILE [GPUID]
READ_FILE is created/truncated and seeded by the program itself, so it does
not need to exist beforehand. WRITE_FILE receives the round-tripped
payload. GPUID is optional (default 0). Sizes and stream counts are
compile-time #defines documented at the top of each .cpp.
Running
Both paths must live on an O_DIRECT-capable local filesystem (ext4 mounted
data=ordered, or xfs); verify with /opt/rocm/bin/ais-check:
cd build
./roundtrip-async in.bin out.bin
./roundtrip-async-nonblocking-stream in.bin out.bin
./roundtrip-async-multi-stream in.bin out.bin
./roundtrip-async-multi-stream-registered in.bin out.bin
aiscp
aiscp is a simple test program that uses hipFile to copy a file. It works
like the Linux cp command:
aiscp SOURCE DEST
common
Not an example — a small static library (examples_common) of helpers shared
by the basics and async examples. It was pulled out to
remove verbatim duplication; each example still drives the hipFile API directly
in its own main() so the example flow stays readable top-to-bottom.
There is nothing to run or build here on its own. The basics/async
directories pull it in via add_subdirectory(../common …), so the helper code
is compiled once and shared rather than duplicated into each example. Keep this
directory alongside basics/async when building them.
What's in it
See examples_common.h for the full, documented
API.