| name | quality-report-update |
| description | Repeatable weekly run that refreshes an already-bootstrapped quality dashboard. Runs the per-repo pipeline (analyzers → collect → generate.py) through the execution wrapper setup chose, rotates This-week → Last-week and appends History, recomputes grades/RAG/trends, validates the dashboard is internally consistent, then narrates what changed week-over-week — leading with regressions and anything newly red. Degraded tools become labelled limitations, never fabricated or stale-carried numbers. Use when the user says 'update the quality report', 'regenerate the quality dashboard', 'run the weekly quality report', 'refresh quality metrics', or asks how this week's quality compares to last. Requires quality-report-setup to have run first; if no dashboard/generator exists, redirect there. Designed to survive a context reset — the setup skill and template/README.md are the source of truth for the contract. |
Quality Report Update — weekly run
Refreshes the dashboard that quality-report-setup stood up. Run repeatedly (typically
weekly). This skill does not build analyzers or rewrite the generator — it drives the
pipeline setup already wired, checks the result is trustworthy, and turns the numbers into a
read a human can act on.
Relationship to setup: quality-report-setup is the one-time bootstrap — it inspects the
stack, builds the per-repo analyzers + collect step, and installs the shipped generate.py.
This skill is the many-times companion (the same split as e2e-setup → e2e-retrofit). It
consumes what setup produced and never bootstraps. If setup hasn't run, stop and redirect
(Step 0).
Read reference/metrics.md in the setup skill if you need to re-confirm what a metric means
or how its grade/RAG thresholds work — the same definitions govern interpretation here. The
generator's input contract lives in the setup skill's template/README.md (the metrics.json
shape). You shouldn't need to touch either unless a metric reads wrong.
Conventions (carried over from setup — they still bind)
The three setup invariants are exactly the failure modes that turn a re-run into a
plausible-but-wrong dashboard. They apply every run, not just at bootstrap:
- Run every tool through the one wrapper setup chose. Direct, or inside the container /
compose service — whichever Step 1 of setup recorded. Don't half-run some analyzers on the host
and some in the container; an artifact the generator can't open reads as "metric missing".
- Paths stay repo-relative, identical everywhere. The new-code lens intersects
git diff
paths with each artifact's paths. If a tool upgrade or a moved work-dir changed a path prefix,
the intersection silently empties and new-code coverage/maintainability/a11y read 0%.
Re-run the cross-check (below) before trusting the new-code column.
- Degraded metric → label it; never fake or carry it. If a tool fails or is unavailable this
week, show its value as
n/a with the reason and list it as a limitation. Do not silently
reuse last week's number, and do not blend against missing branch data. A loud caveat beats
a stale or invented figure.
Step 0 — Preflight: is the repo set up, and can it run? (gate)
Confirm setup has run and the pipeline can execute. Do not run analyzers yet.
- Dashboard + generator present? Look for the dashboard (e.g.
QUALITY_REPORT.md), the
copied-in generator (quality/generate.py or wherever setup put it), the collect step, and
static-sections.md with a documented regenerate command.
- Missing → setup never ran (or only partially). Stop and tell the user to run
quality-report-setup first. Don't try to reconstruct the analyzers here — that's setup's job.
- Present → continue.
- Git usable — working tree state known; default branch resolvable (
origin/HEAD, else the
branch setup recorded). Re-check for a shallow clone / <7 days history (affects the BASE
resolution for the new-code slice — Step 3).
- Execution wrapper reachable — if setup recorded a container/compose service, confirm it's
up (or can be brought up) so analyzer commands route the same way they did at bootstrap.
- Env wiring intact — any tokens/paths the analyzers or scanner need are present.
Emit a one-paragraph readiness note (what's wired, what looks degraded this week, the regenerate
command you'll run) and proceed. If something is degraded but the run can still produce a partial
dashboard, say which metrics will be n/a and why — then continue rather than blocking.
Step 1 — Confirm the run window (light; proceed on defaults)
Set the report window. Default and proceed unless the user has said otherwise:
date — week ending today (default: today).weekStart — date − 7 days.base / head — head = current default-branch tip; base = the first-parent
default-branch commit just before the 7-day window. These feed the new-code diff.
Only ask if the intent is genuinely ambiguous (e.g. the user wants to (re)generate a specific
past week, or a window other than 7 days). Idempotency to rely on: re-running the same
date overwrites This-week and its History row and leaves Last-week untouched; a new date
rotates This → Last and appends a History row. So a same-day re-run after a fix is safe.
Step 2 — Run the pipeline
Run the single regenerate command setup documented — analyzers → collect → generate.py —
through the recorded execution wrapper. Typically:
python3 quality/generate.py --input metrics.json --out QUALITY_REPORT.md --static quality/static-sections.md
(preceded by the analyzer + collect steps the regenerate command chains). Capture each stage's
exit status and output. Surface every analyzer that failed or produced an empty artifact —
those drive the Step 3 degradation handling. Don't swallow a non-zero exit and let the collect
step emit a stale or zero value.
Step 3 — Handle a degraded run honestly
A weekly run hits real-world breakage that bootstrap didn't. For each problem, label — don't
paper over:
- A tool failed this run (flaky scanner, coverage run errored, a11y lint crashed) → that
metric is
n/a (tool failed this run: <reason>) for the week and a listed limitation. Never
carry last week's value forward as if it were fresh.
- New-code window can't resolve (shallow clone,
<7 days, branch doesn't descend from
default) → try git fetch --unshallow / deepen; if it still can't, render the new-code lens
n/a (new-code window unavailable: <reason>) rather than diffing against the wrong base.
- Path cross-check — take one file you know changed this week and confirm its path string is
byte-identical in the
git diff and in each metric's artifact. If a tool upgrade moved a prefix,
fix the normalization before trusting the new-code column (otherwise it silently reads 0%).
- Branch data regressed (a coverage tool stopped emitting branches) → report line-only and
label it; don't blend against zero branches.
Step 4 — Validate the dashboard
Same internal-consistency bar setup's verify step holds — re-check it every run, because the
inputs changed:
- Every wired metric has a non-empty, in-range value; grades/RAG match the
reference/metrics.md
thresholds; backing numbers reconcile with the headline values.
- Rotation happened: for a new
date, last run's This-week moved to Last-week and a History
row was appended; for a same-date re-run, This-week + its History row were overwritten and
Last-week is untouched.
- Trends computed off state: week-over-week arrows reflect the
*.state.json sidecar's prior
values (confirm the state file exists and was read — a deleted/!gitignored state file silently
zeroes all trends).
- Limitations for any
n/a metric are shown with a reason, not hidden.
Fix wiring and re-run (same date — it's idempotent) until the dashboard is internally
consistent. A clean, consistent generation is the gate for reporting it.
Step 5 — Narrate the week (the value over a bare command)
This is why the run is a skill and not just a cron line. Summarize, regressions first:
- Headline RAG — how many 🟢/🟡/🔴 this week; call out anything that newly went red or
dropped a grade since last week.
- Movements — the metrics that improved or regressed week-over-week, with the delta and the
likely driver if visible (e.g. "api coverage 58% → 49% 🔴, new endpoints landed un-tested").
- New-code vs whole-repo — where the Clean-as-You-Code lens diverges from the whole-repo
figure (new code dragging an otherwise-green metric down is the early-warning signal).
- Limitations this week — which metrics were
n/a and why, so a reader doesn't misread a gap
as a pass.
Keep it tight and actionable — a few lines a tech lead can scan, not a re-print of the table.
Step 6 — Commit / hand off
- If the dashboard is tracked (setup committed it rather than gitignoring), commit the
refreshed
QUALITY_REPORT.md on the user's behalf with a dated message
(e.g. Quality report — week ending YYYY-MM-DD). The *.state.json sidecar and intermediate
artifacts stay gitignored — don't commit them.
- If the dashboard is gitignored (a personal/working artifact), don't commit; just report
where it is.
- Confirm outward actions before pushing, per normal git etiquette.
- Close with the one-paragraph narration from Step 5 and the dashboard path.
Done-when checklist
