## Build Instructions
```
pip install -r requirements.txt
# Preferred since 0.12; build inside a container if possible to avoid https://github.com/ROCm/aotriton/issues/166
pip wheel third_party/triton -w /tmp/triton_wheel/
mkdir build
cd build
export PKG_CONFIG_PATH="${PKG_CONFIG_PATH}:${CONDA_PREFIX}/lib/pkgconfig"
# See https://github.com/ROCm/FlyDSL for build instructions
cmake .. -DCMAKE_INSTALL_PREFIX=./install_dir -DCMAKE_BUILD_TYPE=Release -DAOTRITON_GPU_BUILD_TIMEOUT=0 -DAOTRITON_USE_LOCAL_TRITON_WHEEL=/tmp/triton_wheel/<triton_wheel>.whl -DAOTRITON_USE_LOCAL_FLYDSL_WHEEL=<absolute path to flydsl>.whl -G Ninja
# Use ccmake to tweak options
ninja install/strip # Use `ninja install` to keep symbols
```
The library and the header file can be found under `build/install_dir` afterwards.
You may ignore the `export PKG_CONFIG_PATH` part if you're not building with conda.
Note: do not run `ninja` separately, due to the limit of the current build
system, `ninja install` will run the whole build process unconditionally.
The FlyDSL wheel must be compiled from `third_party/flydsl-compiler.txt` with
LLVM from `third_party/flydsl-llvm.txt`. Image-mode builds only:
`AOTRITON_NOIMAGE_MODE` needs neither the FlyDSL nor the Triton wheel.
* As of 0.14b development cycle, LLVM used by FlyDSL wheels suffers from https://github.com/llvm/llvm-project/pull/223465
### Prerequisites
* `python >= 3.10`
- For `python >= 3.14`, Triton needs patch https://github.com/triton-lang/triton/commit/c44b870bdd9e1ea8933fd4057b6b59a5e6e5407b
to fix `ast.Num` error.
* `gcc >= 8` or `clang >= 10`
- For Designated initializers, but only gcc >= 9 is tested.
- The binary delivery is compiled with gcc13
* `cmake >= 3.27`
- Only `cmake >= 3.30` is tested
* `ninja`
- Only `ninja >= 1.11` is tested
- `ninja >= 1.13.1` on Windows due to https://github.com/ninja-build/ninja/issues/2616
* `liblzma`
- Common names are `liblzma-dev` or `xz-devel`.
* `pkg-config`
- Required by the build system to find `liblzma`.
- On RHEL and its derivatives this dependency is ensured by `xz-devel`.
- On Debian and its derivatives this dependency is met by installing `pkgconf`
or `pkg-config` in older releases.
## Generation
The kernel definition for generation is done in
[rules.py](https://github.com/ROCm/aotriton/blob/main/python/rules.py). Edits
to this file are needed for each new kernel, but it is extensible and generic.
Include files can be added in
[this](https://github.com/ROCm/aotriton/tree/main/include/aotriton) directory.
The final build output is an archive object file any new project may link
against.
The archive file and header files are installed in the path specified by
`CMAKE_INSTALL_PREFIX`.
## Debugging
Set `AOTRITON_DEBUG_LEVEL=<n>` to enable runtime logging: 1=ERROR,
2=WARNING, 3=INFO, 4=DEBUG, 5=EXTRA_DEBUG (default: 0, silent).
**Windows limitation:** log messages that include file paths (e.g. at
`LOG_DEBUG` level) use UTF-8 encoding. If your kernel package path contains
non-ASCII characters, ensure your terminal and console code page support UTF-8
(e.g. `chcp 65001`) to avoid garbled output.
## Kernel Support
Currently the first kernel supported is FlashAttention as based on the
[algorithm from Tri Dao](https://github.com/Dao-AILab/flash-attention).
## PyTorch Consumption & Compatibility
AOTriton is consumed in PyTorch through
the [SDPA kernels](https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/transformers/hip/flash_attn/aot/mha_all_aot.hip).
The precompiled binaries will be downloaded and shipped with PyTorch during [builds](https://github.com/pytorch/pytorch/blob/main/cmake/External/aotriton.cmake).
CAVEAT: As a fast moving target, AOTriton's FlashAttention API changes over
time. Hence, a specific PyTorch release is only compatible with a few versions
of AOTriton. The compatibility matrix is shown below.
| PyTorch Upstream | AOTriton Feature Release |
|-----------------------|-------------------------------------------------|
| 2.9 (upcoming) | 0.11b, 0.10b<sup>(1)</sup> |
| 2.8 | 0.10b<sup>(2)</sup>, 0.9b<sup>(2)</sup> |
| 2.7 | 0.9b<sup>(4)</sup>, 0.10b<sup>drop-in only(3)</sup> |
| 2.6 | 0.8b<sup>(5)</sup> |
| 2.5 | 0.7b, 0.8b<sup>(6)</sup> |
| 2.4 | 0.6b |
| 2.3 | 0.4b |
| 2.2 and earlier | N/A, no support |
1. Using AOTriton 0.10b on PyTorch 2.9 will re-introduce Context Parallelism Bug:
[Issue 156012](https://github.com/pytorch/pytorch/issues/156012)
2. PyTorch 2.8 will lose sliding window attention (SWA) support if built with
0.9b since this feature is newly added in 0.10b. In addition,
https://github.com/pytorch/pytorch/pull/159773 is needed to properly
integrate SWA into PyTorch.
3. 0.10b is backward compatible with 0.9b's API. Hence it can be used as a drop-in
replacement for installed PyTorch wheels by symlinking
`libaotriton_v2.so.0.9.2` to `libaotriton_v2.so.0.10.0`. However, 0.10b
cannot be built with PyTorch 2.7 due to the integrity check in the
integration code.
4. To be specific, it is shipped with 0.9.2b. Other versions like 0.9b and 0.9.1b
should not be used in order to avoid linking issues, and also avoid
confusion about version strings.
5. PyTorch 2.6 requires some 0.8b-only features. Hence even if PyTorch 2.6
can compile with 0.7b due to API compatibility, the end product will
suffer from runtime errors.
6. 0.8b's API is backward compatible with 0.7b, but the packaging scheme
has changed drastically.
ROCm's PyTorch release/\<version\> branch is slightly different from PyTorch
upstream and may support more recent versions of AOTriton.
| PyTorch ROCm Fork | AOTriton Feature Release |
|-----------------------|-------------------------------------------------|
| 2.9 (upcoming) | 0.11b |
| 2.8 | 0.10b (backported) |
| 2.7 | 0.9b (backported) |
| 2.6 | 0.9b (backported) |
| 2.5 | 0.9b (backported) |
| 2.4 | 0.10b (backported) |
| 2.3 | 0.4b |
| 2.2 and earlier | N/A, no support |
### Point Release
AOTriton's point releases maintain ABI compatibility and can be used as drop-in
replacement of their corresponding feature releases.
### Windows Limitations
1. AOTriton on Windows currently isn't able to build kernel images by itself.
This is because triton is not officially available on Windows yet.
2. To build on Windows, set `AOTRITON_NOIMAGE_MODE` and use the `aotriton.images`
folder from a Linux build.