build-flydsl

name	build-flydsl
description	Build and install FlyDSL (Flexible Layout Python DSL) on a remote host or Docker container. FlyDSL is a Python DSL and MLIR-based compiler stack for authoring high-performance GPU kernels with explicit layouts and tiling on AMD GPUs. Requires building LLVM/MLIR from source (~30min) then FlyDSL C++ and Python bindings (~5min). Usage: /build-flydsl [container@host]
allowed-tools	Bash

Build and Install FlyDSL

Build FlyDSL from source on a remote host or Docker container. FlyDSL requires a custom LLVM/MLIR build with Python bindings, followed by the FlyDSL C++ dialect and Python package.

Arguments

Argument	Required	Description
[container@host]	No	Target in format container@hostname. If omitted, build locally. Example: [e-mail]

Prerequisites

• ROCm 6.x or 7.x (for GPU execution)
• cmake >= 3.20, C++17 compiler, ninja (recommended)
• Python 3.10+ with pip
• Git (to clone LLVM)
• Disk space: ~50GB for LLVM build + FlyDSL
• CPU cores: more is better (use -j$(nproc) or -j128)

Build Steps

Step 1: Build LLVM/MLIR (~30 min with -j128)

This clones ROCm/llvm-project, checks out the commit specified in thirdparty/llvm-hash.txt, and builds MLIR with Python bindings.

cd /FlyDSL
bash scripts/build_llvm.sh -j128

What it does:

1. Clones https://github.com/ROCm/llvm-project.git to ../llvm-project/
2. Checks out the pinned commit from thirdparty/llvm-hash.txt
3. CMake configure + build + install to ../llvm-project/mlir_install/
4. Creates tarball ../llvm-project/mlir_install.tgz

If you already have an MLIR build, skip this step and set:

export MLIR_PATH=/path/to/llvm-project/mlir_install

Step 2: Build FlyDSL (~5 min)

cd /FlyDSL
bash scripts/build.sh -j128

What it does:

1. Auto-detects MLIR_PATH from common locations (or uses env var)
2. CMake configure + build the Fly dialect (C++) and Python bindings
3. Output: build-fly/python_packages/flydsl/ with embedded MLIR bindings

Step 3: Install (editable mode)

cd /FlyDSL
pip install -e .

Or without installing (just set paths):

export PYTHONPATH=/FlyDSL/build-fly/python_packages:$(pwd):$PYTHONPATH
export LD_LIBRARY_PATH=/FlyDSL/build-fly/python_packages/flydsl/_mlir/_mlir_libs:$LD_LIBRARY_PATH

Step 4: Verify

python3 -c "import flydsl; print('FlyDSL OK')"
bash scripts/run_tests.sh  # GEMM correctness tests (~15s)

Remote/Docker Execution

For building inside a Docker container on a remote host:

# SSH command pattern
ssh -o LogLevel=ERROR <HOST> 'docker exec <CONTAINER> bash -c "cd /FlyDSL && CMD"'

# Full build sequence (run each step, wait for completion)
ssh ... 'docker exec -d <CONTAINER> bash -c "cd /FlyDSL && bash scripts/build_llvm.sh -j128 > /tmp/build_llvm.log 2>&1"'
# Monitor: ssh ... 'docker exec <CONTAINER> tail -5 /tmp/build_llvm.log'
# Wait for "LLVM_BUILD_DONE" or "Creating tarball..." followed by completion

ssh ... 'docker exec <CONTAINER> bash -c "cd /FlyDSL && bash scripts/build.sh -j128"'
ssh ... 'docker exec <CONTAINER> bash -c "cd /FlyDSL && pip install -e ."'
ssh ... 'docker exec <CONTAINER> bash -c "python3 -c \"import flydsl; print(\\\"FlyDSL OK\\\")\""'

Build Artifacts (Do NOT Commit)

The following directories are build artifacts generated by CMake and should NOT be committed to git:

• python/flydsl/_mlir/ — MLIR Python bindings (compiled .so files, platform-specific)
• build/, build-fly/ — CMake build directories

These are already in .gitignore. After a fresh clone, run the build steps above to regenerate.

Rebuild After Code Changes

# C++ changes (dialect, MLIR passes):
bash scripts/build.sh -j128

# Python-only changes:
# No rebuild needed — editable install picks up changes automatically.

# Clear kernel cache if stale results:
rm -rf ~/.flydsl/cache
# Or disable disk cache (in-memory cache remains active):
export FLYDSL_RUNTIME_ENABLE_CACHE=0

Troubleshooting

• std::gcd not found or redeclaration errors: Wrong LLVM picked up. unset MLIR_PATH and let build.sh auto-detect.
• No module named flydsl: Run pip install -e . or set PYTHONPATH.
• MLIR .so load errors: Set LD_LIBRARY_PATH to include build-fly/python_packages/flydsl/_mlir/_mlir_libs/.
• Build OOM: Reduce parallelism (e.g., -j64 instead of -j128).
• Docker exec -d for long builds: Use background mode and monitor log file. LLVM build takes ~30min with 128 cores.

Verified Environments

Container	Image	Host	Status
hungry_dijkstra	rocm/pytorch:rocm7.2_ubuntu24.04_py3.12_pytorch_release_2.8.0	hjbog-srdc-39.amd.com	Verified 2026-03-10