| name | albucore-benchmarks |
| description | Running Albucore micro-benchmarks under benchmarks/, synthetic router timings, and comparing PyPI releases with uv --no-project. Use when adding benchmarks, comparing performance across versions, or documenting benchmark workflow. |
Albucore Benchmarks
Layout
benchmarks/ - Python timing scripts. Run from repo root: uv run python benchmarks/<script>.py.
benchmarks/timing.py - Shared median_ms helper for scripts executed as python benchmarks/foo.py.
./benchmark.sh - Dataset-driven runner; expects an external benchmark package that is not always present in-tree. Prefer synthetic scripts for CI-style checks.
benchmarks/benchmark_router_synthetic.py - Times public routers on synthetic uint8 and float32 arrays: HWC, plus NHWC for mean, std, and mean_std only.
benchmarks/compare_router_json.py - Builds a Markdown table from two JSON outputs.
Canonical Shape Grid
Benchmark shape sweeps use channel-last Albucore conventions.
HWC images:
128x128 with 1, 3, 9 channels - small / warm-cache.
256x256 with 1, 3, 9 channels - mid-size crop.
512x512 with 1, 3, 9 channels - typical augmentation training crop.
1024x1024 with 1, 3, 9 channels - high-res / full-image pass.
DHWC volumes:
16x128x128x1, 16x128x128x3 - thin slab.
32x128x128x1, 32x128x128x3 - common nnU-Net patch depth.
64x128x128x3 - deeper slab.
128x128x128x1 - isotropic single-channel cube.
48x256x256x3 - large in-plane, multi-channel.
NDHWC batch of volumes:
2x32x128x128x1
2x32x128x128x3
2x64x128x128x3
4x16x128x128x3
Channel choices: 1 for grayscale, 3 for RGB / 3-channel, and 9 for hyperspectral paths that exceed MAX_OPENCV_WORKING_CHANNELS=4.
Compare Current Tree vs PyPI Release
uv run python benchmarks/benchmark_router_synthetic.py --output-json benchmarks/results/router-main.json
uv run --no-project --with albucore==0.0.40 --with opencv-python-headless \
--with simsimd --with stringzilla --with numpy \
python benchmarks/benchmark_router_synthetic.py --output-json benchmarks/results/router-0.0.40.json
uv run python benchmarks/compare_router_json.py benchmarks/results/router-main.json \
benchmarks/results/router-0.0.40.json benchmarks/results/REPORT_router_compare.md
Use --quick for smaller shape/channel grids while iterating.
Docs
- NumKong tables and methodology:
docs/numkong-performance.md
- Research notes:
docs/research/