| name | frame-latency-baseline |
| description | Establish a frame-latency baseline by running 5 × 10s Perfetto captures of CameraActivity on the attached device, force-stopping the app between runs, and printing run-to-run dispersion (mean / stdev / CV%) for every dz.frame_* slice. Use to validate that an instrumentation- or rendering-path change hasn't regressed, or to refresh the baseline numbers on a new device. |
frame-latency-baseline
Runs the one-shot frame-latency measurement five times and aggregates dispersion across the runs, so that a single regression-vs-noise judgement can be made from the same command.
Preconditions
Run it
scripts/baseline-frame-latency.sh
scripts/baseline-frame-latency.sh 3 5
The script:
- Loops the chosen number of times. Each iteration:
- Calls
scripts/measure-frame-latency.sh <seconds>, which force-stops the app, grants CAMERA, launches CameraActivity, captures a Perfetto trace with -a com.dz.camerafast, and prints the per-run table.
adb shell am force-stop com.dz.camerafast after the capture so the next iteration starts from a cold launch.- Sleeps 2 s between iterations.
- Parses each per-run table, aggregates
n / avg / p50 / p90 / p99 / max across the runs, and prints the dispersion table.
- Writes
.cache/frame-latency/baseline.txt (the table) and .cache/frame-latency/baseline.json (machine-readable, one entry per stage × metric with mean / stdev / cv_pct / values).
The JSON is the form CI will diff against in a future commit (PR-vs-main regression gate). Treat it as the artifact, the txt as a human view.
How to read the dispersion table
For each stage × metric you get mean, stdev, range, CV% = stdev / mean * 100, plus the raw 5 values.
CV% is the call to make:
- < 5% — stable enough to gate PRs on (~5% tolerance).
- 5–10% — usable with a wider gate or as a secondary signal.
- > 10% — too noisy to gate; track trend only.
Findings from the SM-F936B / Android 16 baseline (see CLAUDE.md for the latest numbers):
frame_e2e.{gl,vk}.avg, frame_e2e.{gl,vk}.p90, frame_to_screen.{gl,vk}.p90 are consistently sub-2% CV at 10 s — these are the headline gating metrics.frame_native_proc.{gl,vk} p90/p99 fluctuate 6–16% — informational only.- All
max columns and p50 on screen-facing stages are bimodal / outlier-driven — don't gate.
When to re-establish the baseline
- New device or new Android version.
- Renderer-impacting code change merged to
main (and you want the post-merge numbers as the new floor).
- Capture-tooling change (Perfetto version, atrace categories, capture duration).
Bumping the baseline because "the numbers got worse and CI was failing" is the wrong move — the point of the baseline is to fail in that case. Investigate first.
Gotchas
-a com.dz.camerafast is mandatory in the perfetto invocation. Without it, app-tag atrace sections (which is where Trace.beginAsyncSection and ATrace_beginAsyncSection slices land) are filtered out and the trace processor will return zero dz.frame_* rows. The wrapper script always sets it; if you ever invoke perfetto by hand, don't forget.- Foldables / multi-display devices still need an explicit display id only if you screenshot —
screencap corrupts its PNG output without -d <id>. Trace capture is unaffected.
- N is intentionally hard-coded to 5 runs by default. Three runs is enough to spot mean drift but not to estimate stdev reliably; five is the cheapest count that gives a meaningful CV. Above 10, you're spending capture minutes for diminishing accuracy.
Failure modes the script reports
error: no adb device in 'device' state. — plug the device in, accept the ADB prompt.error: multiple adb devices attached — set ANDROID_SERIAL=<serial>.error: com.dz.camerafast is not installed — run the installDebug command from "Preconditions".error: no dz.frame_* slices captured. — usually means the APK is older than the instrumentation commit; rebuild and reinstall.
