| name | ai-augmentation |
| description | Use to add AI/LLM features to a modernized web system — define concrete use cases, pilot them, and add evals. For this domain: message-draft generation, send anomaly detection, and natural-language operations-status summarization. Feasibility and cost of in-house LLMs are first-class concerns. |
AI Augmentation
Attach AI where it removes repetitive manual work — not for novelty. Start from
concrete use cases with acceptance criteria, pilot small, and guard outputs.
When to use
- After core UX/backend modernization is underway (needs stable events/APIs).
- Prefer invoking through
renovate-orchestrator. If invoked directly, run the
state check below first.
First action: verify renovation state
Read plan.md and confirm gates.grilling: passed. Design mode (LLM-envelope
check, use-case specs, prompt design) may proceed after grilling. Code mode
(shipping an AI feature) requires the harness green for every seam the feature
touches, plus the test and review gates.
Method
- Confirm the LLM envelope first. Check which LLMs are usable in the
in-house/enterprise environment and their cost/latency limits. This constrains
every use case — do it before designing prompts.
- Define use cases with acceptance criteria (from
requirements-grilling).
Domain candidates:
- Message draft generation — auto-draft campaign messages for author→approve flow.
- Send anomaly detection — flag failure/delay spikes from the realtime status
stream and surface likely cause.
- NL operations summary — natural-language summary of system/send status for the
ops dashboard, replacing manual VM checks and operator inquiries.
- Pilot thin, behind flags. Ship one use case end-to-end before expanding.
- Guard the plumbing. Validate/scheme-check outputs, add fallbacks for LLM
failure/timeout, keep a human-in-the-loop for approval-affecting actions. Never let
an AI action bypass the existing review/approve gate.
- Evaluate quality. Add an eval set (golden inputs + rubric) for each use case;
track accuracy/usefulness before scale-out. (Plumbing tests live in
test-authoring.)
Artifacts
docs/renovation/07-ai/ — LLM-envelope findings (models, cost, limits), use-case
specs, prompt designs, guardrails, eval results.
Staged goals (30 / 60 / 100)
- 30: 2-3 AI use cases defined + technical feasibility confirmed.
- 60: 2 pilots implemented (message draft generation, NL ops summary).
- 100: expanded set (draft generation, anomaly detection, ops summary) in production.
Gates
- Requires: harness green; consumes stable events/APIs from
strangler-backend.
- Produces code → must pass
test-authoring and strict-code-review.
Transition
Feed measured AI impact into impact-measurement.
Output template and helper script
- Korean template: OUTPUT_TEMPLATE.md
- Scaffold script:
scripts/create-ai-doc.sh <repo-root> <target-name>
