albucore-benchmarks

name	albucore-benchmarks
description	Running Albucore micro-benchmarks under benchmarks/, synthetic router timings, and comparing PyPI releases with uv --no-project. Use when adding benchmarks, comparing performance across versions, or documenting benchmark workflow.

Albucore Benchmarks

Layout

• benchmarks/ - Python timing scripts. Run from repo root: uv run python benchmarks/<script>.py.
• benchmarks/timing.py - Shared median_ms helper for scripts executed as python benchmarks/foo.py.
• ./benchmark.sh - Dataset-driven runner; expects an external benchmark package that is not always present in-tree. Prefer synthetic scripts for CI-style checks.
• benchmarks/benchmark_router_synthetic.py - Times public routers on synthetic uint8 and float32 arrays: HWC, plus NHWC for mean, std, and mean_std only.
• benchmarks/compare_router_json.py - Builds a Markdown table from two JSON outputs.

Canonical Shape Grid

Benchmark shape sweeps use channel-last Albucore conventions.

HWC images:

• 128x128 with 1, 3, 9 channels - small / warm-cache.
• 256x256 with 1, 3, 9 channels - mid-size crop.
• 512x512 with 1, 3, 9 channels - typical augmentation training crop.
• 1024x1024 with 1, 3, 9 channels - high-res / full-image pass.

DHWC volumes:

• 16x128x128x1, 16x128x128x3 - thin slab.
• 32x128x128x1, 32x128x128x3 - common nnU-Net patch depth.
• 64x128x128x3 - deeper slab.
• 128x128x128x1 - isotropic single-channel cube.
• 48x256x256x3 - large in-plane, multi-channel.

NDHWC batch of volumes:

• 2x32x128x128x1
• 2x32x128x128x3
• 2x64x128x128x3
• 4x16x128x128x3

Channel choices: 1 for grayscale, 3 for RGB / 3-channel, and 9 for hyperspectral paths that exceed MAX_OPENCV_WORKING_CHANNELS=4.

Compare Current Tree vs PyPI Release

uv run python benchmarks/benchmark_router_synthetic.py --output-json benchmarks/results/router-main.json

uv run --no-project --with albucore==0.0.40 --with opencv-python-headless \
  --with simsimd --with stringzilla --with numpy \
  python benchmarks/benchmark_router_synthetic.py --output-json benchmarks/results/router-0.0.40.json

uv run python benchmarks/compare_router_json.py benchmarks/results/router-main.json \
  benchmarks/results/router-0.0.40.json benchmarks/results/REPORT_router_compare.md

Use --quick for smaller shape/channel grids while iterating.

Docs

• NumKong tables and methodology: docs/numkong-performance.md
• Research notes: docs/research/