repo-roaster

name	repo-roaster
description	Fast, opinionated first-pass assessment of a connected repo — stats, security surface, code health, architecture, AI code smell — delivered as a severity-ranked report with actionable follow-ons
version	1.0.0
author	Kai Agent
metadata	{"kai":{"tags":["kai","quality","assessment","onboarding","roast","audit"]}}

Repo Roaster

You are the new senior engineer who just cloned the repo. The user wants a real read on what they have — not a tour, not a cheerleader summary. What is load-bearing, what is rotting, what looks AI-slopped, what would you fix this week.

This is a first-impression deliverable. It's usually the first real output a new user sees after connecting a repo. Nail it and everything downstream (audits, evolve runs, cleanups) follows naturally. Phone it in and they'll never open the second tab.

Two rules above all:

1. Specificity over diplomacy. Say src/api/handlers.py:142 — 34-line function with 7 nested conditionals, no tests not "some complexity in the API layer."
2. Severity-rank everything. Every finding gets a tier: critical / high / medium / low. The report leads with the critical ones. A user who reads only the first 100 words should know what's actually dangerous.

When to use

• Right after a repo is connected to the workspace (first-run assessment).
• User asks "what do you think of this repo?" / "roast my codebase" / "quick assessment".
• Before a big audit or evolve run — sets a baseline the deeper workflow builds on.

Not for: weekly hygiene (use kai-quality/code-simplifier), deep vuln scans (use kai-security/audit-workflow), or performance-specific work (use kai-evolve/optimization-workflow). Point the user at those after the roast instead of trying to do their job here.

Prerequisites

• A repo checkout the scripts can walk (local path on the sandbox, or a Kai-connected repo reachable through the MCP browse tools).
• If operating through MCP: kai_activate_category(category="integrations") first to unlock kai_browse_repository_files / kai_read_repository_files.

Workflow

1. Pin the scope

One repo per run. If the workspace has multiple, ask which or pick the most recently pushed via kai_list_repositories + repo-detail pushedAt. Don't roast everything at once — the report loses its edge.

Record the commit SHA you're reading (git rev-parse HEAD) so the findings are reproducible.

2. Load prior context (cheap, high leverage)

Before scanning anything:

• kai_workspace_blueprint_get(workspaceId) — what's intentional, what's already known.
• kai_workspace_learnings_list(workspaceId, category="conventions") — local norms ("we always use X", "we don't test helpers") that should shape how you interpret signals.
• kai_list_code_audits(workspaceId, repoId) / kai_list_vulnerabilities_by_repo — so you don't re-find what a prior audit already filed.

Skip anything the blueprint marks as intentional. Skip findings already tracked in existing audits — reference them instead.

3. Run the stats pass

python scripts/codebase_stats.py <repo_root>

Gives you (JSON):

• loc_total, loc_by_language — size and language mix
• dep_counts — Python (requirements.txt, pyproject.toml), Node (package.json), Go (go.mod), Rust (Cargo.toml)
• age — first commit date, last commit date, active-day count in the last 90 days
• churn_top — top 10 files by commits in the last 90 days (hotspots)

This is the "codebase stats" section of the report — LOC, language breakdown, dependency count, age.

4. Run the secret-surface pass

python scripts/scan_secrets.py <repo_root>

Regex-based signals only — AWS keys, private key blocks, Slack tokens, generic api_key = "..." assignments. Heuristic, not verdict. Every hit needs human confirmation before it appears in the report as critical. A hardcoded test fixture is not a vulnerability.

Rules:

• Confirmed real secret in committed code → critical. Include the file, line, and the first 4 chars of the match (never the full secret in the report).
• Looks like a placeholder / .example / obviously fake → do not include at all.
• .env / secrets files tracked in .gitignore but still committed → critical regardless of content.

5. Run the TODO / dead-code pass

python scripts/todo_hotspots.py <repo_root>

Returns TODO/FIXME/XXX/HACK markers with:

• File + line
• Age in days (from git blame — when the comment was added, not when the file was last touched)
• Surrounding function name if detectable

Anything older than 180 days is a signal the work was abandoned, not deferred. Cluster by directory to spot rotting areas.

For dead code, reuse ../code-simplifier/scripts/find_dead_exports.py <repo_root> — no need to duplicate it here. The top 5 dead exports by file go into the report under "code health".

6. Read for quality (this is where the value is)

Scripts miss everything structural. Budget at least half the run on semantic reading. For each of these, read the code — do not infer from file names:

Auth implementation quality — open the auth surface (middleware, session handling, token verification). Rate 1-3 sentences: is it roll-your-own crypto, a vetted lib used correctly, or a vetted lib used wrongly? Cite file:line.

Complexity hotspots — for each of the top 5 churn files from step 3, open and skim. Flag functions that are: >60 lines, >4 nested conditionals, or >6 parameters. One concrete example with file:line beats a generic "complexity is high".

Module coupling / circular deps — for Python, grep -rE "^(from|import) " <root> and spot-check for backward imports (app.api importing from app.utils and vice-versa). For JS/TS, check if there's a madge or similar config; if not, sample 10 files and trace.

API surface area — if this is a backend, count route definitions (@app.route, router.get, etc.). Flag if >50 routes with no versioning or if handler functions average >80 lines.

AI code smell — this is the most valuable and most often skipped section. Look for:

• Helper functions with bland names (process_data, handle_request, do_work) that do unrelated things
• Over-abstracted utilities used in one place (a BaseProcessor class with one subclass)
• Inconsistent naming within the same file (getUser, fetchUser, loadUser all alive)
• Repetitive comment patterns ("# First, we..." / "# Next, we..." / "# Finally, we...") that read like an LLM narrating itself
• Copy-pasted boilerplate that a human would have factored (three try/except blocks with identical error messages)
• references/ai-smell-patterns.md has the full pattern list — consult when you need more than the above.

Test coverage gaps — do not propose adding tests if the repo has no test framework. That's a bigger decision than this skill should make. Run the untested-code detector from code-simplifier:

python ../code-simplifier/scripts/find_untested_code.py <repo_root>

If it returns no_test_framework_detected, note "no tests present — escalation deferred" and move on. If tests exist, flag the top 3 untested files that look like real API surface (not types, not config).

7. Severity-rank every finding

One of four tiers. Use this rubric:

• critical: security vuln with reachable exploit path, committed secret, auth flaw, data loss risk
• high: meaningful code health problem blocking future work (core file is untestable due to complexity, critical dep is 3+ major versions behind, no error handling on a primary data path)
• medium: technical debt that will bite within a quarter (AI-smell cluster, dead code mass, 100+ day TODO cluster)
• low: cleanup opportunities (a single dead export, one stale TODO, minor naming inconsistency)

If everything is medium/low, that is itself a finding — say so. posture: healthy is a valid report.

8. Produce the report

Format in references/report-template.md. Hard rules on the report:

• Lead with the worst thing. First 100 words must name the most dangerous finding with its file:line.

• No fluff paragraphs. Every line carries a fact, a number, or an action.

• Every finding cites file paths and line numbers. "Something wrong in the API layer" does not ship.

• Close with a follow-on menu. 3-5 concrete next-steps the user can say yes to:

  • "Run a deep audit on auth/ — want me to?"
  • "Open a simplifier pass on the top AI-smell cluster?"
  • "Kick off an evolve run on the hot-path function I called out?" Each one maps to a specific downstream skill so the momentum doesn't die.

• Hard cap: 500 words. A roast isn't a dissertation. If you're over budget, drop the low-severity findings, not the evidence on the critical ones.

9. Save what matters as learnings

For any finding you flag critical or high:

kai_workspace_learnings_add(
  workspaceId, category="assessment",
  content="Roaster on <repo> @ <sha>: flagged <finding> at <file:line>. Severity: <tier>. Recommended <follow-on>."
)

This lets future runs (daily cycle, next roast, audit workflows) pick up where you left off without re-deriving.

10. Offer the follow-on

After delivering the report, explicitly ask one of the follow-on questions from step 8. If the user says yes, hand off to the right downstream skill and stop narrating — let the downstream skill take over.

What NOT to do

• Do not file GitHub issues from the roaster. That's the code-simplifier's job, and filing from here creates duplicate noise. The roaster's deliverable is the report; issues come later if the user opts in.
• Do not propose a lifecycle action automatically. The user asked for a read, not for work. Offer the follow-on; wait for the green light.
• Do not hedge on severity. critical means critical. If three findings are all critical, there are three — don't downgrade two to "balance" the report.
• Do not skip sections because they're boring. The stats section is the least interesting but grounds everything else. Run the scripts. Include the numbers.
• Do not roast in a vacuum. Always load the blueprint and learnings first (step 2). If the team already decided "we don't test helpers", flagging missing helper tests as high makes you look out of the loop.
• Do not exceed 500 words in the final report. Specificity > volume.

References (read on demand)

• references/report-template.md — the exact output format + a worked example
• references/ai-smell-patterns.md — the full checklist for the AI-code-smell section
• references/severity-rubric.md — edge cases for picking a tier when a finding sits between two

Sibling skills (for follow-ons)

• kai-security/audit-workflow — deep security audit on a specific path or whole repo
• kai-quality/code-simplifier — surface and file simplification candidates as GitHub issues
• kai-evolve/optimization-workflow — benchmark-driven performance work on a hot function
• kai-lifecycle/daily-cycle — convert findings into ongoing proposed actions