| name | performant-py-rust |
| description | Use when making Python or Rust code faster, optimizing a slow function or hot loop, chasing a performance bottleneck, or deciding whether to vectorize / parallelize / rewrite in Rust — before changing any code to improve its speed. Covers CPU-bound and IO-bound work (single-thread, multicore, async); not GPU. |
Performant Python & Rust
Core principle
Every optimization is a hypothesis; a benchmark is the only thing that confirms it.
Never optimize code you haven't measured, and never keep a change that doesn't move a benchmark you trust. Fast-but-wrong is worthless, so correctness is re-checked on every change. This is TDD for speed: the correctness oracle and the benchmark are your test; the optimization is the code that has to pass both.
The failure this skill prevents is not laziness — it's skipping steps that feel unnecessary because the answer seems obvious: guessing input sizes, guessing the hot spot, asserting "this should be faster," and shipping without confirming the win scales. On a toy loop you get away with it. On real code you optimize the wrong axis, at the wrong hot spot, and can't prove you helped.
The workflow
Copy this checklist into your working notes and fill each slot in before moving on. A blank slot means you're guessing.
Optimization run:
- [ ] Phase 0 — Target: <e.g. "32×20k batch < 50 ms"> Evidence this code is the bottleneck: <...>
- [ ] Phase 1 — Dimensions table filled Dominating dim: <...> Bound: <cpu | memory | io>
- [ ] Phase 2 — Target O(<...>) along <dim> Approach: <lever>
- [ ] Phase 3 — Correctness oracle ✓ Parameterized benchmark ✓ Baseline: <number + units>
- [ ] Phase 4 — Profiled hot spot: <...> change → re-measure → correctness ✓ (repeat) Stopped because: <...>
Phase 0 — Is optimization warranted?
The cheapest optimization is the one you don't do. Before anything:
- Is there a measured problem or a concrete target (a latency, throughput, or memory budget)?
- Is this code actually on the hot path?
If nothing has measured this as slow, stop and measure first, or don't optimize it (YAGNI). Don't gold-plate a prototype or a cold path.
Artifact: one sentence — the target, and the evidence this code is the bottleneck.
Phase 1 — Characterize the workload
Performance is meaningless without knowing what runs through the code.
- List every input dimension: rows, columns, sequence length, batch size, number of variants, alphabet size, file size, request rate…
- For each, get the realistic range and the max, and mark which dimension(s) dominate or grow without bound in real use. This one decision drives the whole design — optimize the wrong axis and the work is wasted.
- Determine the bound: is the work CPU-bound, memory-bandwidth-bound, or IO/latency-bound? This picks the levers in Phase 2 (and decides parallelism vs. concurrency).
Do not fabricate sizes. Derive them from the code and the data, and confirm the ranges with the user — they know the real workload; you are guessing. If you genuinely cannot reach the user, write each assumed range down as an explicit assumption and make the Phase-3 harness sweep it rather than hardcoding one guess.
Artifact — dimensions table:
| dimension | typical | max | grows? | notes |
|---|
| n_genes | 20,000 | 60,000 | fixed | dominates inner cost |
| n_samples | 32 | ~1,000 | grows per study | parallelizable |
Phase 2 — Design for the dominating dimension
- Aim for the best practical asymptotic behavior along the dominating dimension(s).
- Practical caveats beat pure asymptotics. Constant factors, cache behavior, and SIMD-friendliness usually decide the winner at real sizes. Don't reach for galactic algorithms (Strassen matmul and friends) whose crossover point is past your real N. Pick memory layout for the access pattern: contiguous arrays, struct-of-arrays over array-of-structs, avoid pointer-chasing.
- Match the lever to the bound:
| Bound | Lever |
|---|
| CPU-bound, Python | Climb the ladder only as far as the target needs: vectorized NumPy/Polars → Numba → Rust/PyO3. Stop at the first rung that hits the target. |
| CPU-bound, either | Data parallelism (rayon in Rust; multiprocessing/joblib or a GIL-releasing extension in Python) — only after single-thread is tuned. |
| IO/latency-bound | Concurrency, not cores: async (asyncio/tokio) or threads to overlap waits. |
| Memory-bandwidth-bound | Fewer passes, fewer allocations, better layout. More cores won't help. |
Artifact: one line — target O(<...>) along <dim>; approach: <lever>.
Phase 3 — Build the harness before optimizing
Two artifacts, both before you touch the code for speed. See tools.md for tool specifics.
-
Correctness oracle. Keep the current/naive implementation as the reference. Assert every candidate's output equals it — exactly, or to a documented tolerance — over representative and edge cases (empty, all-zeros, degenerate, unequal lengths). A candidate that fails the oracle is not a faster version; it's a bug. This is the one step the baseline agents did well — keep doing it.
-
Parameterized benchmark + baseline number. Use the smallest input that is representative of the regime that matters: big enough that the dominating cost dominates (not lost in call overhead or noise), small enough to run in ~seconds so you can iterate. Parameterize by the Phase-1 dimensions and sweep the dominating one to confirm the scaling you designed for — don't just assert "it's faster at real N." Control noise (warmup, multiple reps, report median/min + spread); prefer criterion (Rust) / pytest-benchmark or timeit (Python) over hand-rolled timing. Record the baseline number now — it's what every change is measured against.
Minimal Python harness embodying oracle + sweep + baseline:
import timeit, numpy as np
def reference(*args): ...
def candidate(*args): ...
def make_inputs(n_samples, n_genes, n_blacklist, seed=0):
rng = np.random.default_rng(seed)
...
return counts, lengths, blacklist
for shape in [(4, 100, 5), (8, 4000, 0), (2, 10, 10)]:
a = make_inputs(*shape)
assert np.allclose(candidate(*a), reference(*a), equal_nan=True), shape
for n_genes in (2_000, 20_000, 200_000):
a = make_inputs(32, n_genes, 500)
t = min(timeit.repeat(lambda: candidate(*a), number=5, repeat=5)) / 5
print(f"n_genes={n_genes:>7}: {t*1e3:f} ms/call")
Phase 4 — Measure → optimize → repeat
Loop until diminishing returns:
- Profile to find the actual hot spot — don't guess it. By Amdahl's law only the biggest slice is worth your time. Python:
pyinstrument (default). Rust: samply / flamegraph. Full tool set and platform notes in tools.md.
- Form one hypothesis, make one change, re-run the oracle and the benchmark. Keep it only if it wins and stays correct; revert otherwise.
- For compiled/Rust changes, verify the mechanism — don't assert it. Use
cargo-show-asm to confirm the compiler actually vectorized, inlined, or dropped bounds checks. "This should autovectorize" is a hypothesis until you've read the asm.
- Re-profile — the hot spot moves — and repeat.
Stop when you've hit the Phase-0 target, or the next hot spot's share is too small to matter (Amdahl ceiling), or the remaining gain isn't worth the added complexity/maintenance. State which one stopped you.
Red flags — you're guessing
| Sign | Fix |
|---|
| Editing for speed before profiling | Profile first — you don't know the hot spot |
| No baseline number recorded | You can't claim a speedup you didn't measure |
| No correctness oracle | You may be shipping fast garbage |
| Benchmark size chosen by vibes | You're measuring overhead or noise, not the workload |
| "This should vectorize / be faster" with no asm and no benchmark | A hypothesis stated as a fact — confirm it |
| Optimizing code no profile says is hot | Premature; likely wasted effort |
| "Scales better at real N" without sweeping N | Unverified — sweep the dimension |
| Fabricated "realistic" input sizes | Derive from data / ask the user; the dominating dim decides everything |
Quick reference
| Phase | Required artifact | Primary tools |
|---|
| 0 Warranted? | Target + bottleneck evidence | — |
| 1 Characterize | Dimensions table + bound | profiler on real inputs |
| 2 Design | Target complexity + lever | — |
| 3 Harness | Correctness oracle + swept benchmark + baseline | criterion / pytest-benchmark / timeit |
| 4 Optimize | Profile → change → re-measure loop | pyinstrument; samply, cargo-show-asm |
Tooling, install commands, and the macOS-vs-Linux tool matrix (perf/callgrind are Linux-only): see tools.md.
