TheRock/rocm-libraries/projects/miopen/README.md

# MIOpen

MIOpen is AMD's library for high-performance machine learning primitives.

You can find sources and binaries in the [MIOpen folder](https://github.com/ROCm/rocm-libraries/tree/develop/projects/miopen)
of the [rocm-libraries GitHub](https://github.com/ROCm/rocm-libraries) repository.

> [!NOTE]
> The published MIOpen documentation is available at [MIOpen](https://rocm.docs.amd.com/projects/MIOpen/en/latest/index.html) in an organized, easy-to-read format, with search and a table of contents. The documentation source files reside in the MIOpen/docs folder of this repository. As with all ROCm projects, the documentation is open source. For more information, see [Contribute to ROCm documentation](https://rocm.docs.amd.com/en/latest/contribute/contributing.html).

MIOpen supports the [HIP](https://rocm.docs.amd.com/projects/HIP/en/latest/) programming model (backend).

## Building our documentation

To build the MIOpen documentation locally, run the following code from within the `docs` folder of the MIOpen project:

``` shell
sudo apt install doxygen

cd docs

pip3 install -r sphinx/requirements.txt

python3 -m sphinx -T -E -b html -d _build/doctrees -D language=en . _build/html
```

## Installing MIOpen

To install MIOpen, you must first install these prerequisites:

* A [ROCm](https://rocm.docs.amd.com/)-enabled platform
* A base software stack that includes:
  * HIP (HIP and HCC libraries and header files)
* [ROCm CMake](https://github.com/ROCm/rocm-cmake): provides CMake modules for common build
  tasks needed for the ROCm software stack
* [Half](http://half.sourceforge.net/): IEEE 754-based, half-precision floating-point library
* [SQLite3](https://sqlite.org/index.html): A reading and writing performance database
* lbzip2: A multi-threaded compress or decompress utility
* [rocBLAS](https://github.com/ROCm/rocm-libraries/tree/develop/projects/rocblas): AMD's library for Basic Linear Algebra Subprograms
  (BLAS) on the ROCm platform.
  * Minimum version branch for pre-ROCm 3.5 [master-rocm-2.10](https://github.com/ROCm/rocBLAS/tree/master-rocm-2.10)
  * Minimum version branch for post-ROCm 3.5 [master-rocm-3.5](https://github.com/ROCm/rocBLAS/tree/master-rocm-3.5)
* [hipBLASLt](https://github.com/ROCm/rocm-libraries/tree/develop/projects/hipblaslt): AMD's flexible Basic Linear Algebra Subprograms
  (BLAS) API.
* [hipBLAS](https://github.com/ROCm/rocm-libraries/tree/develop/projects/hipblas): AMD's (BLAS) marshalling library.
* [Multi-Level Intermediate Representation (MLIR)](https://github.com/ROCm/rocMLIR) with its
  MIOpen dialect to support and complement kernel development
* [Composable Kernel](https://github.com/ROCm/composable_kernel): A C++ templated device library
  for GEMM-like and reduction-like operators.

### Installing with pre-built packages

You can install MIOpen on Ubuntu using `apt-get install miopen-hip`.

### Installing with a kernels package

MIOpen provides an optional pre-compiled kernels package to reduce startup latency. These
precompiled kernels comprise a select set of popular input configurations. We'll expand these kernels
in future releases to include additional coverage.

Note that all compiled kernels are locally cached in the `$HOME/.cache/miopen/` folder, so
precompiled kernels reduce the startup latency only for the first run of a neural network. Precompiled
kernels don't reduce startup time on subsequent runs.

To install the kernels package for your GPU architecture, use the following command:

``` shell
apt-get install miopen-hip-<arch>kdb
```

Where ``<arch>`` is the GPU architecture (e.g., `gfx900`, `gfx906`, `gfx1030` ).

>[!NOTE]
>Not installing these packages doesn't impact the functioning of MIOpen, since MIOpen compiles
>them on the target machine once you run the kernel. However, the compilation step may significantly
>increase the startup time for different operations.

The `utils/install_precompiled_kernels.sh` script provided as part of MIOpen automates the preceding
process. It queries the user machine for the GPU architecture and then installs the appropriate
package. You can invoke it using:

``` shell
./utils/install_precompiled_kernels.sh
```

The preceding script depends on the `rocminfo` package to query the GPU architecture. Refer to
[Installing pre-compiled kernels](https://rocm.docs.amd.com/projects/MIOpen/en/latest/cache.html#installing-pre-compiled-kernels)
for more information.

## Installing dependencies

You can install dependencies using the `install_deps.cmake` script (`cmake -P install_deps.cmake`).

>[!NOTE]
> You can run this script from the ``rocm-libraries/projects/miopen`` directory.

By default, this installs to `/usr/local`, but you can specify another location using the `--prefix`
argument:

``` shell
cmake -P install_deps.cmake --prefix <miopen-dependency-path>
```

An example CMake step is:

``` shell
cmake -P install_deps.cmake --minimum --prefix /root/MIOpen/install_dir
```

You can use this prefix to specify the dependency path during the configuration phase using
`CMAKE_PREFIX_PATH`.

MIOpen's HIP backend uses [rocBLAS](https://github.com/ROCm/rocm-libraries/tree/develop/projects/rocblas) by default. You can install
rocBLAS' minimum release using `apt-get install rocblas`. To disable rocBLAS, set the configuration flag
`-DMIOPEN_USE_ROCBLAS=Off`.

MIOpen's HIP backend can use [hipBLASLt](https://github.com/ROCm/rocm-libraries/tree/develop/projects/hipblaslt). You can install hipBLASLt's minimum
release using ``apt-get install hipblaslt``. In addition to needing hipblaslt, you will also need to install [hipBLAS](https://github.com/ROCm/rocm-libraries/tree/develop/projects/hipblas).
You can install hipBLAS's minimum release using ``apt-get install hipblas``.
To disable hipBLASLt, set the configuration flag ``-DMIOPEN_USE_HIPBLASLT=Off``.

## Building MIOpen from source

You can build MIOpen from source using the HIP backend.

### HIP backend

First, create a build directory:

```shell
mkdir build; cd build;
```

Next, configure CMake. You can set the backend using the `-DMIOPEN_BACKEND` CMake variable.

Set the C++ compiler to `clang++`. For the HIP backend (ROCm 3.5 and later), run:

```shell
export CXX=<location-of-clang++-compiler>
cmake -DMIOPEN_BACKEND=HIP -DCMAKE_PREFIX_PATH="<hip-installed-path>;<rocm-installed-path>;<miopen-dependency-path>" ..
```

An example CMake step is:

```shell
export CXX=/opt/rocm/llvm/bin/clang++ && \
cmake -DMIOPEN_BACKEND=HIP -DCMAKE_PREFIX_PATH="/opt/rocm/;/opt/rocm/hip;/root/MIOpen/install_dir" ..
```

>[!NOTE]
>When specifying the path for the `CMAKE_PREFIX_PATH` variable, **do not** use the tilde (`~`)
>shorthand to represent the home directory.

### Setting up locations

By default, the install location is set to `/opt/rocm`. You can change this using
`CMAKE_INSTALL_PREFIX`:

```shell
cmake -DMIOPEN_BACKEND=HIP -DCMAKE_INSTALL_PREFIX=<miopen-installed-path> ..
```

### System performance database and user database

The default path to the system performance database (System PerfDb) is `miopen/share/miopen/db/`
within the install location. The default path to the user performance database (User PerfDb) is
`~/.config/miopen/`. For development purposes, setting `BUILD_DEV` changes the default path to both
database files to the source directory:

```shell
cmake -DMIOPEN_BACKEND=HIP -DBUILD_DEV=On ..
```

Database paths can be explicitly customized using the `MIOPEN_SYSTEM_DB_PATH` (System PerfDb)
and `MIOPEN_USER_DB_PATH` (User PerfDb) CMake variables.

To learn more, refer to the
[performance database](https://rocm.docs.amd.com/projects/MIOpen/en/latest/perfdatabase.html)
documentation.

### Persistent program cache

By default, MIOpen caches device programs in the `~/.cache/miopen/` directory. Within the cache
directory, there is a directory for each version of MIOpen. You can change the location of the cache
directory during configuration using the `-DMIOPEN_CACHE_DIR=<cache-directory-path>` flag.

You can also disable the cache during runtime using the `MIOPEN_DISABLE_CACHE=1` environmental
variable.

#### For MIOpen version 2.3 and earlier

If the compiler changes, or you modify the kernels, then you must delete the cache for the MIOpen
version in use (e.g., `rm -rf ~/.cache/miopen/<miopen-version-number>`). You can find more
information in the [cache](https://rocm.docs.amd.com/projects/MIOpen/en/latest/cache.html)
documentation.

#### For MIOpen version 2.4 and later

MIOpen's kernel cache directory is versioned so that your cached kernels won't collide when upgrading
from an earlier version.

### Changing the CMake configuration

The configuration can be changed after running CMake (using `ccmake`):

`ccmake ..` **or** `cmake-gui`: `cmake-gui ..`

The `ccmake` program can be downloaded as a Linux package (`cmake-curses-gui`), but is not available
on Windows.

## Building the library

You can build the library from the `build` directory using the 'Release' configuration:

`cmake --build . --config Release` **or** `make`

You can install it using the 'install' target:

`cmake --build . --config Release --target install` **or** `make install`

This installs the library to the `CMAKE_INSTALL_PREFIX` path that you specified.

## Building the driver

MIOpen provides an [application-driver](https://github.com/ROCm/rocm-libraries/tree/develop/projects/miopen/driver) that
you can use to run any layer in isolation, and measure library performance and verification.

You can build the driver using the `MIOpenDriver` target:

` cmake --build . --config Release --target MIOpenDriver ` **or** ` make MIOpenDriver `

To learn more, refer to the [driver](https://rocm.docs.amd.com/projects/MIOpen/en/latest/driver.html)
documentation.

## Running the tests

You can run tests using the 'check' target:

` cmake --build . --config Release --target check ` **OR** ` make check `

To build and run a single test, use the following code:

```shell
cmake --build . --config Release --target test_tensor
./bin/test_tensor
```
Check gtests formats

```shell
cd ./test/utils && python3 gtest_formating_checks.py
```


## Formatting the code

The easiest way to format the repo is to run `make format` from your build directory.  You can also
use the methods below if you need custom formating behaviour.

All the code is formatted using `clang-format`. To format a file, use:

```shell
clang-format -style=file -i <path-to-source-file>
```

To format the code per commit, you can install githooks:

```shell
./.githooks/install
```

## Storing large files using Data Versioning System

[Data Versioning System (DVS)](https://dvc.org/) replaces large files, such as audio samples, videos, datasets, and
graphics with text pointers inside Git, while storing the file contents on a remote server. In MIOpen, we use DVC to
store our large files, such as our kernel database files (*.kdb) that are normally > 0.5 GB.

You can install DVC using the [instructions provided for your platform here](https://dvc.org/doc/install).

You can [pull](https://dvc.org/doc/command-reference/pull) all large files or a single large file using:

```shell
dvc pull
or
dvc pull "filename"
```

If you are familiar with using Git LFS, a key difference with DVC is that you must manually run `dvc pull` after you
switch branches or merge changes in Git to ensure any large binaries are kept in sync with your checkout.

## Installing the dependencies manually

If you're using Ubuntu, you can install the `BZip2` packages using:

```shell
sudo apt-get install libbz2-dev
```

You must install the `half` header from the [half website](http://half.sourceforge.net/).

## Using Docker

The easiest way to build MIOpen is via Docker. Building the MIOpen Docker image requires [Docker Buildx](https://docs.docker.com/build/buildx/). Ensure it is available before proceeding:

```shell
docker buildx version
```

The Dockerfile supports two build modes controlled by the `BUILD_TYPE` build argument:

### Option 1: Using a prebuilt ROCm/TheRock image (default)

This is the standard path for development. It pulls a pre-built `rocm/miopen:therock` base image from Docker Hub and builds the MIOpen environment on top of it, targeting the `miopen` stage:

```shell
docker buildx build \
  --load \
  --target miopen \
  --tag miopen-image:gfx1101 \
  --build-arg PREFIX=/opt/rocm \
  --build-arg THEROCK_ASIC=gfx1101 \
  -f ../../projects/miopen/Dockerfile \
  ../../projects/.
```

### Option 2: Building ROCm/TheRock from source (nightly)

This path clones and builds TheRock from source before building the MIOpen environment in a single step. Use this when you need to build against a specific TheRock commit or when no prebuilt image is available:

```shell
docker buildx build \
  --load \
  --target miopen \
  --tag miopen-image:gfx1101 \
  --build-arg BUILD_TYPE=build \
  --build-arg THEROCK_GIT_HASH=<commit-hash> \
  --build-arg PREFIX=/opt/rocm \
  --build-arg THEROCK_ASIC=gfx1101 \
  -f ../../projects/miopen/Dockerfile \
  ../../projects/.
```

Then, to enter the development environment, use `docker run`. For example:

```shell
docker run -it -v $HOME:/data --privileged --rm --device=/dev/kfd --device /dev/dri:/dev/dri:rw  --volume /dev/dri:/dev/dri:rw -v /var/lib/docker/:/var/lib/docker --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined miopen-image
```

You can find prebuilt Docker images on [ROCm's public Docker Hub](https://hub.docker.com/r/rocm/miopen/tags). These images are multi arch CI images.

## Porting from cuDNN to MIOpen

Our
[porting guide](https://rocm.docs.amd.com/projects/MIOpen/en/latest/MIOpen_Porting_Guide.html)
highlights the key differences between cuDNN and MIOpen APIs.