| name | qa-regression-agent |
| description | Guards Spectra pixel output via golden tests and unit suite. Use when golden fails, updating baselines, adding golden coverage for new series/UI, or before merge on rendering changes. |
QA Regression Agent
Detect and prevent rendering regressions by maintaining golden image baselines and keeping the full unit test suite green. This agent owns the pixel-correctness layer of QA.
Required Context
Before starting any task, read:
plans/QA_results.md — known rendering regressions and their statustests/golden/ — golden baseline PNGs and test source files
Workflow
1. Build (full build including golden tests)
cmake -B build -G Ninja -DSPECTRA_BUILD_GOLDEN_TESTS=ON -DSPECTRA_BUILD_QA_AGENT=ON
cmake --build build -j$(nproc)
2. Run unit tests first (fast gate)
ctest --test-dir build --output-on-failure --exclude-regex "golden|gpu"
All non-GPU unit tests must pass before touching golden images.
3. Run golden image tests
ctest --test-dir build -L golden --output-on-failure
./build/tests/golden_image_tests
./build/tests/golden_image_tests_phase2
./build/tests/golden_image_tests_phase3
./build/tests/golden_image_tests_3d
./build/tests/golden_image_tests_3d_phase3
4. Triage failures
For each failing golden test:
- Identify which screenshot name failed and what changed visually.
- Check if the change was intentional (theme fix, new feature) or unintentional (regression).
- For unintentional regressions: find the commit that broke it, fix the root cause.
- For intentional changes: regenerate the golden after verifying the new output looks correct.
5. Regenerate golden baselines (intentional changes only)
./build/tests/golden_image_tests --gtest_filter='*' --update-golden
./build/tests/golden_image_tests_3d --gtest_filter='*' --update-golden
./build/tests/golden_image_tests --gtest_filter='GoldenTest.LinePlot' --update-golden
Never regenerate goldens to paper over a rendering regression. Always verify the new output visually first.
6. Add new golden baselines for new features
When a new series type, colormap, or UI state is added, add a corresponding golden test:
- Write the test in the appropriate
tests/golden/golden_test_*.cpp file.
- Run once with
--update-golden to create the baseline.
- Verify the baseline visually.
- Commit both the test and the baseline PNG.
7. Run design-review to cross-check
./build/tests/spectra_qa_agent --seed 42 --design-review --no-fuzz --no-scenarios \
--output-dir /tmp/spectra_qa_regression_check
Compare design-review screenshots against golden baselines to catch visual regressions not covered by the golden suite.
8. Verify and document
cmake --build build -j$(nproc)
ctest --test-dir build --output-on-failure
Document any baseline updates in plans/QA_results.md with before/after descriptions.
Golden Test Coverage Map
| Suite | File | What it covers |
|---|
golden_image_tests | tests/golden/golden_test.cpp | Core 2D plots: line, scatter, bar, subplots, legend, grid |
golden_image_tests_phase2 | tests/golden/golden_test_phase2.cpp | UI chrome: inspector, timeline, split views, themes |
golden_image_tests_phase3 | tests/golden/golden_test_phase3.cpp | Advanced: docking, multi-window, crosshair, overlays |
golden_image_tests_3d | tests/golden/golden_test_3d.cpp | 3D: surface, scatter, line3D, camera angles |
golden_image_tests_3d_phase3 | tests/golden/golden_test_3d_phase3.cpp | 3D colormaps, orthographic projection, transparency |
Features needing golden coverage (from recent commits)
| Feature | Status | Action needed |
|---|
| 3D surface colormaps (Viridis/Plasma/Inferno/Magma/Jet/Coolwarm/Grayscale) | ⚠️ Check | Verify golden_test_3d_phase3.cpp covers all 7 colormap types |
| Series clipboard paste result | ⚠️ Add | Paste a series and golden the result layout |
| Figure serialization roundtrip | ⚠️ Add | Save+load and golden the re-rendered figure |
| Line culling pan/zoom | ✅ Covered | Renderer correctness covered by existing line plot goldens |
Issue-to-File Map
| Issue type | Primary file |
|---|
| Golden test infrastructure | tests/golden/golden_test.cpp + helpers |
| 3D golden tests | tests/golden/golden_test_3d.cpp |
| Golden baseline PNGs | tests/golden/baseline/ |
| Golden test output (failures) | tests/golden/output/ |
| Renderer producing wrong output | src/render/renderer.cpp |
| 3D surface colormap shader | src/gpu/shaders/surface3d.frag |
| Theme color changes affecting goldens | src/ui/theme/theme.cpp |
| Axes/grid rendering changes | src/render/renderer.cpp |
| Text rendering quality | src/render/text_renderer.cpp |
Registered Commands Used in Regression Scenarios
| Scenario command | Purpose |
|---|
view.toggle_grid | Golden with/without grid |
view.toggle_legend | Golden with/without legend |
view.toggle_3d | Golden 2D→3D transition |
file.save_figure + file.load_figure | Serialization roundtrip golden |
series.paste | Post-paste layout golden |
Performance Targets
| Metric | Target |
|---|
| Non-GPU unit tests | 100% pass |
| Golden tests (all suites) | 0 failures |
| New feature golden coverage | 100% for new series/colormap types |
Session notes
Update REPORT.md and plans/QA_*.md; extend references/ if you discover new patterns.
Session notes
Update REPORT.md and plans/QA_*.md; extend references/ if you discover new patterns.
Self-Improvement — YYYY-MM-DD
Improvement: <one sentence describing what was added/changed>
Motivation:
Change: <file(s) edited OR new golden test / new coverage row added>
Next gap:
### How to pick an improvement
1. **If regressions were found:** Turn the root cause into a new targeted golden test that would have caught it in isolation. Ask: "What single-feature golden would have failed before the fix?"
2. **If no regressions were found:** Coverage is incomplete. Pick from the Improvement Backlog below, add the golden test, and document the new baseline.
### Improvement Backlog (consume one per session, add new ones as discovered)
| ID | Improvement | How to implement |
|---|---|---|
| REG-I1 | Golden for series clipboard paste result — post-paste layout must be stable | Add golden: copy series A, paste into figure, golden the result axes state |
| REG-I2 | Golden for figure serialization roundtrip — saved+loaded figure must render identically | Add golden: create figure, save as `.spectra`, load, render, diff against original |
| REG-I3 | Golden for all 7 colormap types on SurfaceSeries | Add 7 goldens (one per `ColormapType`) in `golden_test_3d_phase3.cpp` |
| REG-I4 | Golden for deep-zoom precision (camera-relative rendering at 10^10 zoom) | Add golden: set view range to 1e-10, render 1K-point line, verify points still visible |
| REG-I5 | Golden for shared cursor across linked subplots | Add golden: 2 linked axes, hover at x=5.0, capture both crosshair lines positioned correctly |
| REG-I6 | Golden for orthographic 3D projection (currently listed as ⚠️ Check) | Verify `26_3d_orthographic` baseline is present and tight (±2 pixel tolerance) |
| REG-I7 | Golden for inspector panel with series stats — number formatting stability | Add golden: inspector open on 10K-point series, assert stat text positions don't jump |
| REG-I8 | Tighten existing 2D golden pixel tolerance from default to ±1 pixel | Audit tolerance values in `golden_test.cpp` run_golden_test calls; tighten where default is too loose |
| REG-I9 | Golden for light theme (separate from dark theme goldens) | Add a light-theme variant of `01_default_single_line` golden to catch theme-switching regressions |
| REG-I10 | Golden for tab context menu visual state (right-click menu open) | Capture screenshot 40 `40_tab_context_menu` as a golden baseline |
---
## Live Report
Append findings to [REPORT.md](REPORT.md) at the end of every session.
### Report update procedure
After every run, open `REPORT.md` and:
1. Add a new `## Session YYYY-MM-DD HH:MM` block at the top (newest first).
2. Fill in: unit test results (pass/fail count), golden suite results per suite, failures investigated (ID + root cause + intentional/regression), baselines regenerated, new goldens added, self-updates made.
3. Update the `## Current Status` block at the very top.
4. Never delete old session blocks.
---
## MCP
[spectra-mcp](../spectra-mcp/SKILL.md).
## Guardrails
- Never regenerate goldens without visually inspecting the new output first.
- Never delete a golden test — mark it `DISABLED_` if temporarily blocked.
- Run the full unit test suite before touching golden files.
- If a golden fails after a theme change, check `plans/QA_design_review.md` for the intentionality.
- Add new goldens in the same session that introduces the new feature.
- Document all baseline updates (file, reason, before/after) in `plans/QA_results.md`.
- Self-updates to this file require all tests passing — never update speculatively.
