TheRock/rocm-systems/projects/hipfile/docs/how-to/use-python-api.rst

.. Generated by ROCm docgen prompt_version=2026-06-03-v18-include-python-and-tighten-meta-leaks at 2026-06-03T14:05:21Z

.. meta::
   :description: Task-oriented guide to using the hipFile Python bindings for GPU-accelerated file I/O.
   :keywords: hipFile, Python, ROCm, GPU, direct I/O, bindings, API

.. provenance:
..   mode: NEW
..   sources:
..     - python/hipfile/__init__.py
..     - python/hipfile/driver.py
..     - python/hipfile/buffer.py
..     - python/hipfile/file.py
..     - python/hipfile/error.py
..     - python/hipfile/enums.py
..     - python/hipfile/properties.py
..     - python/main.py

*******************
Use the Python API
*******************

The hipFile Python bindings let you perform GPU-accelerated file I/O from Python without writing C code. This page walks through each step of a typical workflow: opening the driver, registering buffers, reading and writing files, handling errors, and querying version and driver properties.

For a complete worked example that copies a file through GPU memory and verifies the result with SHA-256 hashes, see :doc:`/tutorials/python-gpu-io`. For full API signatures and class details, see :doc:`/reference/api-python`.

Import hipFile
=================

Import the top-level ``hipfile`` package. All public classes, functions, enums, and exceptions are re-exported from the package root:

.. code-block:: python

   from hipfile import (
       Driver,
       FileHandle,
       Buffer,
       HipFileException,
       OpError,
       FileHandleType,
       get_version,
       driver_get_properties,
   )

Open and close the driver
============================

The ``Driver`` class manages the hipFile driver lifecycle. Use it as a context manager so the driver closes automatically when the block exits:

.. code-block:: python

   with Driver() as drv:
       print(f"Driver reference count: {drv.use_count()}")
       # ... perform I/O inside this block ...

The driver is reference-counted internally. Each call to ``open()`` increments the count, and each call to ``close()`` decrements it. ``Driver.use_count()`` returns the current count as an integer.

You can also call ``open()`` and ``close()`` explicitly:

.. code-block:: python

   drv = Driver()
   drv.open()
   # ... perform I/O ...
   drv.close()

Create and register a GPU buffer
=================================

hipFile operates on GPU memory. After allocating device memory (for example, through ``ctypes`` bindings to ``hipMalloc``), wrap the pointer in a ``Buffer`` and register it with the driver.

``Buffer.from_ctypes_void_p`` accepts a ``ctypes.c_void_p``, the buffer length in bytes, and registration flags (pass ``0`` for default behavior):

.. code-block:: python

   from hipfile.hipMalloc import hipMalloc, hipFree

   size = 1024 * 1024  # 1 MiB
   gpu_ptr = hipMalloc(size)

   with Buffer.from_ctypes_void_p(gpu_ptr, size, 0) as buf:
       # buf is registered; use it for reads and writes
       ...

   hipFree(gpu_ptr)

When used as a context manager, ``Buffer`` calls ``register()`` on entry and ``deregister()`` on exit. You can also call these methods directly:

.. code-block:: python

   buf = Buffer.from_ctypes_void_p(gpu_ptr, size, 0)
   buf.register()
   # ... use buf ...
   buf.deregister()

If the ``ctypes.c_void_p`` is null, ``from_ctypes_void_p`` raises ``ValueError``.

Open a file handle
===================

``FileHandle`` wraps ``os.open`` together with hipFile handle registration. Pass a filesystem path, the ``os.open`` flags, and optionally a file mode and handle type:

.. code-block:: python

   import os

   with FileHandle("/path/to/input.bin", os.O_RDONLY | os.O_DIRECT) as fh:
       bytes_read = fh.read(buf, size, file_offset=0, buffer_offset=0)

For writing, include ``os.O_WRONLY`` or ``os.O_RDWR`` with ``os.O_CREAT`` and ``os.O_DIRECT``:

.. code-block:: python

   with FileHandle(
       "/path/to/output.bin",
       os.O_RDWR | os.O_DIRECT | os.O_CREAT | os.O_TRUNC,
   ) as fh_out:
       bytes_written = fh_out.write(buf, size, file_offset=0, buffer_offset=0)

The default file creation mode is ``0o644``. The default handle type is ``FileHandleType.OPAQUE_FD``.

Read and write data
======================

``FileHandle.read`` and ``FileHandle.write`` each take four positional arguments:

``buffer``
   A registered ``Buffer`` instance.

``size``
   Number of bytes to transfer.

``file_offset``
   Byte offset within the file.

``buffer_offset``
   Byte offset within the GPU buffer.

Both methods return the number of bytes transferred:

.. code-block:: python

   bytes_read = fh.read(buf, size, file_offset=0, buffer_offset=0)
   bytes_written = fh.write(buf, size, file_offset=0, buffer_offset=0)

If a system-level I/O error occurs, an ``OSError`` is raised. If a hipFile or HIP driver error occurs, a ``HipFileException`` is raised.

Handle errors
===============

``HipFileException`` wraps both the ``hipFileOpError_t`` code and the underlying ``hipError_t`` from the HIP runtime:

.. code-block:: python

   try:
       fh.read(buf, size, 0, 0)
   except HipFileException as e:
       print(f"hipFile error code: {e.hipfile_err}")
       print(f"HIP runtime error code: {e.hip_err}")
       print(f"Description: {e}")

``hipfile_err``
   The ``hipFileOpError_t`` integer code describing the failure.

``hip_err``
   The ``hipError_t`` integer code from the HIP runtime. This value is only meaningful when ``hipfile_err`` equals ``OpError.HIP_DRIVER_ERROR``.

The string representation of the exception includes the error description returned by the C library.

Classify errors with ``OpError``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The ``OpError`` enum mirrors every value from ``hipFileOpError_t``. Use it to match specific error conditions:

.. code-block:: python

   from hipfile import OpError

   try:
       fh.read(buf, size, 0, 0)
   except HipFileException as e:
       if e.hipfile_err == OpError.DIO_NOT_SET:
           print("O_DIRECT was not set on the file descriptor.")
       elif e.hipfile_err == OpError.HIP_DRIVER_ERROR:
           print(f"HIP runtime error: {e.hip_err}")

Query version and driver properties
====================================

``get_version()`` returns a ``(major, minor, patch)`` tuple:

.. code-block:: python

   major, minor, patch = get_version()
   print(f"hipFile version: {major}.{minor}.{patch}")

``driver_get_properties()`` returns a dictionary of driver property names mapped to their integer values:

.. code-block:: python

   props = driver_get_properties()
   for name, value in props.items():
       print(f"{name}: {value}")

Both functions raise ``HipFileException`` if the underlying C call fails.