Systematically bisect an end-to-end performance regression between two refs (tags/branches/commits) down to the exact offending commit(s). Use this when CI/customer benchmarks show one version is measurably slower than another and you need to attribute the cost to specific PRs.
Run full verification before committing or creating a PR. Use this when you want to create a PR.
Review C code changes for correctness, safety, and style. Use this when you want to review C code changes or PRs.
Review Rust code changes for unsafe correctness, security and robustness, documentation quality, and C-to-Rust porting fidelity. Use this when you want to review Rust changes before merging.
Compile the project to verify changes build successfully. Use this to verify your changes build properly together with the complete project and dependencies, and make sure to use it before running end to end tests.
Investigate a RediSearch flaky-test report from a Jira key, CI failure, test id, or local logs. Use this to collect evidence, identify a supported root cause, and propose a real fix; if the evidence is insufficient, say so and ask for the missing data instead of suggesting workaround-only fixes or skip_until.
Fix merge conflicts in a jj change and its ancestors, starting from the oldest conflict. Use this when a jj change or its ancestors have conflicts that need resolving.
| name | bisect-perf-regression |
| description | Systematically bisect an end-to-end performance regression between two refs (tags/branches/commits) down to the exact offending commit(s). Use this when CI/customer benchmarks show one version is measurably slower than another and you need to attribute the cost to specific PRs. |
Pin a perf regression to specific commits using drift-cancelled, repeated, cold benchmarks.
You will need (or need to build) a small harness around the workload of interest. The exact shape of the harness does not matter, as long as it satisfies the properties described in step 1 below (interleaved labels, cold per-iteration state, median + CV reporting).
$ARGUMENTS should identify:
Before bisecting anything, prove the two endpoints are reliably distinguishable.
--cpus=8 for a Docker harness) and run on a quiet Linux
host. Shared/laptop Docker setups (macOS Docker Desktop, Codespaces, noisy CI
runners) are generally too jittery for sub-3% deltas.N_RUNS adaptively from observed CV — don't burn 5–8 runs by default.
N_RUNS=3 (the minimum that yields a stdev/CV).| observed max CV across labels | action |
|---|---|
| ≤ 1.5 % | 3 runs is enough, accept the medians |
| 1.5 – 2.5 % | rerun with N_RUNS=5 for the labels involved in the suspect step |
| 2.5 – 4 % | rerun with N_RUNS=8, and only trust deltas ≥ 2 × max CV |
| > 4 % | the host is too noisy — fix it before bisecting (see below) |
max(3 %, 2 × max CV across the two labels).N_RUNS=8, increase the per-run workload size, or
fix the host (other processes, CPU governor, thermal throttling, neighbour VMs)
before continuing — bisection on noisy data attributes regressions to the
wrong commit.Always start with broad strokes and narrow down — never start at the commit level.
v2.10.12 … v2.10.30, run all of them as labels in a single
interleaved sweep at N_RUNS=3. Look for the step — the adjacent pair
with the largest jump. That's your new [lo, hi] interval.[lo, hi]. Enumerate evenly-spaced
first-parent commits that actually touched code (filter to src/*,
deps/*, or the relevant subtrees — skip pure CI / docs / vendoring
commits, since they cannot move performance). Build them, run the
interleaved bench [lo, p1, p2, …, hi], and inspect adjacent deltas.
Start at N_RUNS=3 and escalate per the CV table in step 1.[lo, hi] is one commit, or when the
remaining interval is below the noise floor.A single endpoint number like "+30% slower" can hide several smaller steps that may compound through the same code path. Always print the full per-tag / per-commit ladder, not just endpoint deltas — a smooth slope versus a single cliff have very different fixes.
When a known regression dominates the gap and a fix is already in flight, apply that fix on top of every historical commit you benchmark. This neutralises the known cost so the ladder surfaces only the residual regressions that arrived alongside it. Without this technique, a large dominant cost can mask smaller but still real per-commit jumps.
Build the fix-on-commit variants by:
fix-on-<sha> branch off <sha>.Use a code injector (a small script that locates the target function by
signature regex, balances braces to find its body, and inserts the fix at a
known anchor) rather than a static git apply patch when:
--ignore-whitespace won't save you).Keep the injector idempotent (detect its own marker before injecting) so reruns are safe across rebuilds.
Once a single commit is suspected:
[<commit>^, <commit>] only. Here it's worth
spending more iterations than the bisect ladder used — start at N_RUNS=5
and escalate to 8 if max CV > 2.5 %. The delta should match the step size
found during bisection to within 2 × max CV.<bad> with just that commit reverted)
and confirm it returns to <lo> performance. This guards against picking up an
innocent commit that sits next to the real offender via a noisy run.Report only what the data supports:
Do not round, do not collapse multi-step regressions into a single cause, and do not skip the confirmation step.
End with: