codebase-readiness-review

name

codebase-readiness-review

description

Evaluate a codebase's quality, security, and agent-automation readiness using a structured 8-pillar scorecard. Use this skill whenever someone asks to: review code quality, score a repo for agent readiness, run a readiness or maturity assessment, audit CI/CD pipelines, check security posture, evaluate engineering health, or assess "how good is this codebase." Also trigger when asked "what should I fix first," "what's the scorecard," or "how do I improve this repo." Produces a scored report across 8 pillars with concrete evidence, an overall maturity level (1–5), and a ranked fix list ordered by ROI. Always runs real commands (typecheck, lint, test, audit) rather than just reading config files — the delta between what config declares and what actually passes is where the bugs hide.

Codebase Readiness Review

Score a codebase across 8 pillars and an OpenSSF supply-chain layer. Produce concrete, evidence-backed findings and a fix list ranked by ROI. This is not a vibe check — every score requires a specific observation from the actual repo.

How to run a review

1. Discover the repo's shape

Read these files to understand the stack before scoring anything. Skip files that don't exist.

package.json / pyproject.toml / go.mod / Cargo.toml   (runtime + deps)
tsconfig.json / .eslintrc / eslint.config.js / .flake8  (static analysis)
vitest.config.ts / jest.config.* / pytest.ini            (test config)
.github/workflows/*.yml                                   (CI)
.github/dependabot.yml                                    (dep automation)
.nvmrc / .tool-versions / .python-version                 (toolchain pin)
Makefile / justfile                                       (dev entrypoints)
README.md / CONTRIBUTING.md / SECURITY.md / CHANGELOG.md (docs)
.gitignore / .prettierignore / .editorconfig              (repo hygiene)

2. Run the real gate

Do NOT skip this step. Run the project's quality gate and capture actual exit codes and output. The gap between declared and enforced is where problems hide.

# Adapt to the stack (Node example):
npm run typecheck 2>&1 | tail -5
npm run lint      2>&1 | tail -5
npm run format:check 2>&1 | tail -5
npm test          2>&1 | tail -10
npm audit --audit-level=high 2>&1 | tail -5

# Or: make check / python -m pytest / cargo test / go test ./...

Record exact pass/fail status and error summaries — cite these as evidence in the scores.

3. Run the supply-chain checks

# Check if Actions are SHA-pinned (floating tags = risk):
grep -rE "uses:.*@[^#0-9a-f]" .github/workflows/ 2>/dev/null || echo "no workflows"

# Check for obvious secrets:
git grep -nIE "(api[_-]?key\s*[:=]\s*['\"][^'\"]{12}|BEGIN (RSA|OPENSSH) PRIVATE KEY|ghp_[A-Za-z0-9]{30})" -- . ':!package-lock.json' 2>/dev/null | head -5

# Branch protection (requires gh):
gh api repos/{owner}/{repo}/branches/main/protection -q '.required_status_checks' 2>&1 | head -3

4. Score each pillar (0–100)

Score based on what you observed, not what config promises. Every score needs a one-line evidence note. Use the rubrics in references/pillars.md.

Pillar	Weight	What it measures
Style & Validation	high	Linting, formatting, type-checking — are they enforced and passing?
Build System	high	Lockfile committed, toolchain version pinned, bootstrap is one command
Testing	high	Coverage thresholds defined and gated in CI, tests pass, side-effects injectable
Documentation	medium	README, CONTRIBUTING, ARCHITECTURE, inline docs — enough for an agent to orient
Dev Environment	medium	One-command setup, toolchain constraints clear, no "it works on my machine"
Observability	medium	Structured logs, run IDs, metrics, failures surfaced not swallowed
Security	high	No secrets in repo, least-privilege CI, SECURITY.md, supply-chain controls
Task Discovery	medium	make help / CLI help, what-to-run-first is self-documenting

5. OpenSSF supply-chain checks (separate from the 8 pillars)

These are binary pass/fail, not scored — but failures should appear in the ranked fix list:

• Branch protection — main protected, CI is a required status check, no force-push
• Pinned dependencies — Actions pinned to full 40-char commit SHA + # vX.Y.Z comment
• Code review enforced — PRs required, at least 1 approving review
• Dependency automation — Dependabot or Renovate watching both packages and Actions
• Vulnerability scanning — npm audit --audit-level=high or equivalent gates in CI
• Secrets in repo — git history scan clean, no API keys or private keys committed
• Security policy — SECURITY.md present with a vulnerability reporting path

6. Compute the overall maturity level

Map the weighted average score to a maturity level:

Level	Score	What it means
1	0–30	Not agent-ready; only isolated, supervised tasks
2	31–50	Agents can work in parts with close supervision
3	51–65	Agents can implement simple features with human review
4	66–80	Economically viable threshold; agents handle medium complexity
5	81–100	High autonomy; human review is strategic, not operational

7. Produce the report

Always include all of these sections:

## Verdict
<One sentence: overall maturity level, overall score, and the single most important gap.>

## Scorecard

| Pillar             | Score | Level | Evidence |
|--------------------|-------|-------|----------|
| Style & Validation | XX    | ✅/⚠️/❌ | <one line — what you observed> |
| ...                |       |       | |
| **Overall**        | XX    | N/5  | |

## OpenSSF Supply-Chain

| Check | Result | Note |
|---|---|---|
| Branch protection | ✅/❌ | |
| ...               |       | |

## Top fixes (ranked by ROI)

1. **<Fix name>** — <one sentence on what to do and why it's the highest-leverage change>
2. ...

## What's already strong
<2–3 things that are genuinely good — don't invent praise, but acknowledge real strengths.>

Key principles

Run, don't just read. A project with a perfect vitest.config.ts that never runs in CI is worse than no config at all — it creates false confidence. Always execute the gate.

Evidence, not vibes. Every score needs a specific observation: a filename, an exit code, a grep result, a threshold number. "Looks good" is not a finding.

ROI-rank the fixes. The most valuable fix is usually the one that closes the gap between "declared" and "enforced" — because that gap is where regressions silently slip through. Rank fixes by: (1) closes a false-confidence gap, (2) blocks a real attack surface, (3) saves recurring human effort, (4) polish.

Say when you can't verify. If you can't run a command (e.g., no gh auth), say so explicitly. A "can't verify" is more honest than a fabricated pass/fail.