qa-regression-agent

name	qa-regression-agent
description	Guards Spectra pixel output via golden tests and unit suite. Use when golden fails, updating baselines, adding golden coverage for new series/UI, or before merge on rendering changes.

QA Regression Agent

Detect and prevent rendering regressions by maintaining golden image baselines and keeping the full unit test suite green. This agent owns the pixel-correctness layer of QA.

---

Required Context

Before starting any task, read:

• plans/QA_results.md — known rendering regressions and their status
• tests/golden/ — golden baseline PNGs and test source files

---

Workflow

1. Build (full build including golden tests)

cmake -B build -G Ninja -DSPECTRA_BUILD_GOLDEN_TESTS=ON -DSPECTRA_BUILD_QA_AGENT=ON
cmake --build build -j$(nproc)

2. Run unit tests first (fast gate)

ctest --test-dir build --output-on-failure --exclude-regex "golden|gpu"

All non-GPU unit tests must pass before touching golden images.

3. Run golden image tests

# All golden suites
ctest --test-dir build -L golden --output-on-failure

# Individual suites
./build/tests/golden_image_tests
./build/tests/golden_image_tests_phase2
./build/tests/golden_image_tests_phase3
./build/tests/golden_image_tests_3d
./build/tests/golden_image_tests_3d_phase3

4. Triage failures

For each failing golden test:

1. Identify which screenshot name failed and what changed visually.
2. Check if the change was intentional (theme fix, new feature) or unintentional (regression).
3. For unintentional regressions: find the commit that broke it, fix the root cause.
4. For intentional changes: regenerate the golden after verifying the new output looks correct.

5. Regenerate golden baselines (intentional changes only)

# Regenerate all
./build/tests/golden_image_tests --gtest_filter='*' --update-golden
./build/tests/golden_image_tests_3d --gtest_filter='*' --update-golden

# Regenerate specific test
./build/tests/golden_image_tests --gtest_filter='GoldenTest.LinePlot' --update-golden

Never regenerate goldens to paper over a rendering regression. Always verify the new output visually first.

6. Add new golden baselines for new features

When a new series type, colormap, or UI state is added, add a corresponding golden test:

1. Write the test in the appropriate tests/golden/golden_test_*.cpp file.
2. Run once with --update-golden to create the baseline.
3. Verify the baseline visually.
4. Commit both the test and the baseline PNG.

7. Run design-review to cross-check

./build/tests/spectra_qa_agent --seed 42 --design-review --no-fuzz --no-scenarios \
    --output-dir /tmp/spectra_qa_regression_check

Compare design-review screenshots against golden baselines to catch visual regressions not covered by the golden suite.

8. Verify and document

cmake --build build -j$(nproc)
ctest --test-dir build --output-on-failure

Document any baseline updates in plans/QA_results.md with before/after descriptions.

---

Golden Test Coverage Map

Suite	File	What it covers
golden_image_tests	tests/golden/golden_test.cpp	Core 2D plots: line, scatter, bar, subplots, legend, grid
golden_image_tests_phase2	tests/golden/golden_test_phase2.cpp	UI chrome: inspector, timeline, split views, themes
golden_image_tests_phase3	tests/golden/golden_test_phase3.cpp	Advanced: docking, multi-window, crosshair, overlays
golden_image_tests_3d	tests/golden/golden_test_3d.cpp	3D: surface, scatter, line3D, camera angles
golden_image_tests_3d_phase3	tests/golden/golden_test_3d_phase3.cpp	3D colormaps, orthographic projection, transparency

Features needing golden coverage (from recent commits)

Feature	Status	Action needed
3D surface colormaps (Viridis/Plasma/Inferno/Magma/Jet/Coolwarm/Grayscale)	⚠️ Check	Verify golden_test_3d_phase3.cpp covers all 7 colormap types
Series clipboard paste result	⚠️ Add	Paste a series and golden the result layout
Figure serialization roundtrip	⚠️ Add	Save+load and golden the re-rendered figure
Line culling pan/zoom	✅ Covered	Renderer correctness covered by existing line plot goldens

---

Issue-to-File Map

Issue type	Primary file
Golden test infrastructure	tests/golden/golden_test.cpp + helpers
3D golden tests	tests/golden/golden_test_3d.cpp
Golden baseline PNGs	tests/golden/baseline/
Golden test output (failures)	tests/golden/output/
Renderer producing wrong output	src/render/renderer.cpp
3D surface colormap shader	src/gpu/shaders/surface3d.frag
Theme color changes affecting goldens	src/ui/theme/theme.cpp
Axes/grid rendering changes	src/render/renderer.cpp
Text rendering quality	src/render/text_renderer.cpp

---

Registered Commands Used in Regression Scenarios

Scenario command	Purpose
view.toggle_grid	Golden with/without grid
view.toggle_legend	Golden with/without legend
view.toggle_3d	Golden 2D→3D transition
file.save_figure + file.load_figure	Serialization roundtrip golden
series.paste	Post-paste layout golden

---

Performance Targets

Metric	Target
Non-GPU unit tests	100% pass
Golden tests (all suites)	0 failures
New feature golden coverage	100% for new series/colormap types

---

Session notes

Update REPORT.md and plans/QA_*.md; extend references/ if you discover new patterns.

Session notes

Update REPORT.md and plans/QA_*.md; extend references/ if you discover new patterns.

Self-Improvement — YYYY-MM-DD

Improvement: <one sentence describing what was added/changed> Motivation: Change: <file(s) edited OR new golden test / new coverage row added> Next gap:

### How to pick an improvement

1. **If regressions were found:** Turn the root cause into a new targeted golden test that would have caught it in isolation. Ask: "What single-feature golden would have failed before the fix?"
2. **If no regressions were found:** Coverage is incomplete. Pick from the Improvement Backlog below, add the golden test, and document the new baseline.

### Improvement Backlog (consume one per session, add new ones as discovered)

| ID | Improvement | How to implement |
|---|---|---|
| REG-I1 | Golden for series clipboard paste result — post-paste layout must be stable | Add golden: copy series A, paste into figure, golden the result axes state |
| REG-I2 | Golden for figure serialization roundtrip — saved+loaded figure must render identically | Add golden: create figure, save as `.spectra`, load, render, diff against original |
| REG-I3 | Golden for all 7 colormap types on SurfaceSeries | Add 7 goldens (one per `ColormapType`) in `golden_test_3d_phase3.cpp` |
| REG-I4 | Golden for deep-zoom precision (camera-relative rendering at 10^10 zoom) | Add golden: set view range to 1e-10, render 1K-point line, verify points still visible |
| REG-I5 | Golden for shared cursor across linked subplots | Add golden: 2 linked axes, hover at x=5.0, capture both crosshair lines positioned correctly |
| REG-I6 | Golden for orthographic 3D projection (currently listed as ⚠️ Check) | Verify `26_3d_orthographic` baseline is present and tight (±2 pixel tolerance) |
| REG-I7 | Golden for inspector panel with series stats — number formatting stability | Add golden: inspector open on 10K-point series, assert stat text positions don't jump |
| REG-I8 | Tighten existing 2D golden pixel tolerance from default to ±1 pixel | Audit tolerance values in `golden_test.cpp` run_golden_test calls; tighten where default is too loose |
| REG-I9 | Golden for light theme (separate from dark theme goldens) | Add a light-theme variant of `01_default_single_line` golden to catch theme-switching regressions |
| REG-I10 | Golden for tab context menu visual state (right-click menu open) | Capture screenshot 40 `40_tab_context_menu` as a golden baseline |

---

## Live Report

Append findings to [REPORT.md](REPORT.md) at the end of every session.

### Report update procedure

After every run, open `REPORT.md` and:
1. Add a new `## Session YYYY-MM-DD HH:MM` block at the top (newest first).
2. Fill in: unit test results (pass/fail count), golden suite results per suite, failures investigated (ID + root cause + intentional/regression), baselines regenerated, new goldens added, self-updates made.
3. Update the `## Current Status` block at the very top.
4. Never delete old session blocks.

---

## MCP

[spectra-mcp](../spectra-mcp/SKILL.md).

## Guardrails

- Never regenerate goldens without visually inspecting the new output first.
- Never delete a golden test — mark it `DISABLED_` if temporarily blocked.
- Run the full unit test suite before touching golden files.
- If a golden fails after a theme change, check `plans/QA_design_review.md` for the intentionality.
- Add new goldens in the same session that introduces the new feature.
- Document all baseline updates (file, reason, before/after) in `plans/QA_results.md`.
- Self-updates to this file require all tests passing — never update speculatively.