grill-for-unknowns

# Docs + Unknowns Grill ## Overview The core idea is: - **The map** = the prompt, plan, assumptions, skills, prior context, docs excerpts, and the agent's current mental model. - **The territory** = the real codebase, product constraints, APIs, docs, user taste, deployment environment, and failure modes. - **Unknowns** = the gap between the map and the territory. This skill combines docs-grounded grilling, one-question-at-a-time interviewing, domain modeling, and a four-quadrant unknowns pass. Grilling here means few, evidence-priced questions — not relentless interrogation; not asking about a non-material topic is correct behavior. The goal is to discover the few answers that would materially change the plan (see the **Material** criterion below) — and to write down the shared understanding as it forms. ## When to Use Use when: - The user says not to rush implementation, asks for a stronger plan, or wants a rigorous planning pass before orchestrating implementation work. - The task depends on unfamiliar docs, APIs, libraries, platform behavior, or source conventions. - The user has a vague product/design desire and likely has **unknown knowns**: they will know good/bad when they see it, but cannot fully specify it upfront. - The agent is about to spawn subagents or a long-running coding agent and needs a better launch packet. - A previous attempt failed or is stuck because the agent made assumptions, overfit to generic best practices, or missed real codebase constraints. - Reviewing a plan/spec/PR where you need to pressure-test hidden assumptions before merge. Do **not** use when: - The task is trivial, mechanical, or already has unambiguous acceptance criteria. - The user explicitly wants immediate execution and the risk of wrong assumptions is low. - You can verify the right answer directly with a single tool call and no interview is needed. ## Operating Mode Stay in **Explore** or **Plan** mode until the unknowns that could change the implementation are resolved or explicitly accepted as assumptions. The grill has a defined end: it is over when the unknowns ledger is empty — every material unknown resolved, defaulted, or explicitly accepted. Announce the remaining count as it shrinks (e.g., "2 material unknowns left") so the user can see the end approaching. Default sequence: 1. **Restate the map** — summarize the user's request, the intended outcome, and what is already known. 2. **Read the territory** — inspect the relevant docs/source/tests/config before grilling. Do not rely on vibes if docs or code are available. 3. **Open a grill session ledger** — use `templates/grill-session.md` when the session is complex enough to need a durable working doc. 4. **Build the unknowns ledger** — classify per the Unknowns Taxonomy below. 5. **Build the domain ledger** — identify fuzzy terms, overloaded concepts, vocabulary conflicts, and context boundaries. Use `references/domain-modeling-add-on.md` for `CONTEXT.md` / ADR rules. 6. **Grill one decision at a time** — follow the grill procedure below. 7. **Propose defaults** — for low-risk unknowns, choose a sensible default and label it as an assumption instead of blocking. 8. **Persist shared understanding** — update `CONTEXT.md` for crystallized domain terms and offer ADRs when the Domain Modeling criteria are met. 9. **Create or revise the plan** — see Implementation Plan Requirements below. 10. **Ask for confirmation before build** — do not enact the plan until the user confirms shared understanding, unless they explicitly authorize proceeding with labeled assumptions. 11. **During implementation** — keep implementation notes for deviations and newly discovered unknowns. 12. **Post-implementation** — produce an explainer and quiz/review checklist so the user understands what changed. ## Unknowns Taxonomy Use this table explicitly in the output when the task is ambiguous enough to justify it. | Type | Meaning | How to expose it | Example | | --- | --- | --- | --- | | Known knowns | Requirements already stated or proven by docs/source | Restate and cite | "Use Stripe Connect; webhook endpoint already exists." | | Known unknowns | The user/agent knows a decision is unresolved | Ask targeted questions or choose labeled defaults | "Should refunds sync one-way or two-way?" | | Unknown knowns | The user would recognize the right result when shown, but has not verbalized the criterion | Prototype, sketches, examples, references | "This dashboard feels too enterprise; make it more operator-like." | | Unknown unknowns | Constraints or possibilities nobody has considered yet | Blindspot pass over docs/source/tests/internet; ask experts; search prior art | "The API rate limit makes this sync architecture impossible." | ## Docs-Grounded Grill Procedure ### 1. Gather evidence first Before asking the user to decide, inspect available ground truth: - Official docs for libraries/platforms/APIs. - Local source files, routes, models, schemas, migrations, tests, and config. - Existing project conventions and similar implementations. - Error logs, CI failures, issue comments, PR diffs, or previous implementation notes. - Reference implementations the user points to, even if in another language. Fetch missing-but-retrievable docs; if docs cannot be accessed, say so and mark the claim as unverified. ### 2. Convert evidence into pressure-test questions Good grill questions have all three properties: - **Material** — the answer could change architecture, scope, UX, data model, security, permissions, or acceptance criteria. - **Grounded** — the question points to docs/source behavior or a concrete uncertainty, not generic preference fishing. - **Answerable** — the user can choose from options, approve a default, or supply a reference. Bad grill questions: - Obvious preferences that a competent agent can default. - Exhaustive questionnaires before any research. - Asking the user to answer things the code/docs can answer. - Asking the user to verbalize taste they can only recognize when shown ("what does modern mean to you?") — route those to prototypes and references instead. - Open-ended "anything else?" questions with no context. ### 3. Ask one material question at a time when blocked If an answer is required to proceed, ask one question, explain why it matters, and give a recommended default. Walk the design tree branch-by-branch — do not dump the whole tree on the user at once. Template: ```md Blocking question: <question> Why it matters: <what changes if answer A vs B> Evidence: <doc/source/test/reference citation> Recommended answer: <default + rationale> If you don't care: I'll proceed with <default>. ``` If multiple questions are useful but not blocking, keep them in the grill queue and ask the next unresolved material decision first. Budget and exit rules: - Default budget: ~5 blocking questions per session. Going beyond it requires asking the user whether to continue. - **Fatigue valve**: if the user's answers turn short or impatient (one-word replies, "just pick"), stop interviewing — convert the remaining unknowns to labeled defaults and present them as one batch for veto. - Once no blocking questions remain, do not keep asking one at a time: present the residual low-risk unknowns as a single assumptions list for veto. ## Domain Modeling: Shared Language and ADRs Grilling must also maintain shared language. During the grill, challenge fuzzy or overloaded terms immediately, compare the user's terms against existing `CONTEXT.md`, code identifiers, docs, and product copy, and update `CONTEXT.md` when a term crystallizes (glossary only — no plans, scratchpads, or ADR content). Offer an ADR only when the decision is (1) hard to reverse, (2) surprising without context, and (3) the result of a real trade-off; otherwise record it in the session/implementation notes. See `references/domain-modeling-add-on.md` for file layout, formats, and examples. ## Finding Unknown Unknowns: Blindspot Pass Run a blindspot pass when the user is entering an unfamiliar domain, unfamiliar part of the codebase, or high-stakes integration: search the relevant docs/source/tests — including documented limits and known failure modes of load-bearing dependencies — for unknown unknowns that could materially change the plan, explain them in plain language, rank by implementation risk, and suggest how to resolve each one cheaply. Output shape: ```md ## Blindspot Pass ### Highest-risk unknown unknowns 1. <unknown> - Why it matters: - Evidence: - Cheap resolution: - Decision owner: user / agent / docs / prototype ### Likely safe assumptions - <assumption> — why safe, how to verify later ### Questions worth asking now 1. <one material question> ``` ## Unknown Knowns: Brainstorms, Prototypes, and References When the user will recognize the right answer visually or behaviorally but cannot fully specify it: - Build cheap prototypes before wiring real systems — e.g., a single-file mock with fake data showing 3 distinct directions. - Offer multiple directions with meaningful contrast, not tiny variations. - Ask the user to react to examples, screenshots, demos, or reference source — e.g., 2-3 similar in-repo modules plus one external reference, then ask which behavior to match. - Capture the user's reactions as explicit criteria — and when quality can't be checked by a test, distill them into a short rubric that becomes the verification gate. ## Implementation Plan Requirements When producing the plan, lead with the decisions most likely to change: 1. **Decision surface** — data model, type interfaces, permissions, user-facing flows, API semantics, migration strategy. 2. **Evidence** — docs/source references that justify the plan. 3. **Open questions** — only material unknowns, ranked by risk. 4. **Resolved assumptions** — low-risk defaults the agent will use unless corrected. 5. **Prototype/reference artifacts** — links or paths if relevant. 6. **Implementation steps** — bite-sized, ordered, with verification gates. 7. **Deviation policy** — what the implementer should do if the territory contradicts the map. ## During Implementation: Notes and Deviations For complex work, create a temporary implementation notes file such as `implementation-notes.md` or include an equivalent section in the final report. Use `templates/implementation-notes.md` for the minimum sections: plan snapshot, decisions made, deviations, new unknowns, and verification. Default deviation policy: - If the issue is low-risk and local, choose the conservative option, log it, and continue. - If the issue changes architecture, data migration, security, cost, or user-facing behavior, stop and ask. - If docs contradict the plan, trust the docs/source over the original map and update the plan. ## Post-Implementation: Explain, Pitch, Quiz After implementation, help the user and reviewers understand the territory discovered during the work. Deliver: - What changed and why. - Which unknowns were resolved. - Which assumptions remain. - Docs/source evidence for important behavior. - Verification results from real commands/tests. - A short quiz/checklist if the user needs to understand before merge — every quiz item must be answerable from the report itself. ## Subagent / Coding-Agent Launch Packet Before spawning a subagent or external coding agent, prepare a launch packet from `templates/launch-packet.md`. It covers: goal, map, territory to inspect first, the four unknowns categories, deviation policy, and verification gates. If using multiple subagents, split roles: - **Docs scout** — reads official docs/source and returns constraints. - **Codebase scout** — maps existing patterns and tests. - **Prototype scout** — creates cheap visual/API alternatives to expose unknown knowns. - **Implementer** — edits only after the plan is stable enough. - **Reviewer** — grills the diff against the launch packet and docs. ## Calibration: Over- vs Under-Constraining - **Too specific**, and the agent follows instructions even when a pivot is better. Define the goal, constraints, and stop/continue rules; leave room for implementation judgment. - **Too vague**, and the agent defaults to generic best practices that may not fit the product/codebase. Provide references, docs, taste examples, and acceptance criteria. ## Verification Checklist Before moving from planning to implementation: - [ ] Relevant docs/source/tests/config were inspected, or the lack of access is stated. - [ ] Known knowns, known unknowns, unknown knowns, and suspected unknown unknowns are listed. - [ ] Blocking questions are material and include recommended defaults. - [ ] Low-risk unknowns are converted into labeled assumptions rather than blocking progress. - [ ] The plan leads with likely-to-change decisions, not mechanical steps. - [ ] Deviation policy is explicit for long-running/subagent work. - [ ] Verification gates are defined before implementation begins. Before finalizing implementation: - [ ] Deviations and newly discovered unknowns were logged. - [ ] Tests/checks/manual verification were actually run and reported. - [ ] Remaining assumptions are visible. - [ ] The user/reviewer gets an explainer sufficient to understand the change. --- Adapted from Matt Pocock's `grilling` + `domain-modeling` skills and Thariq's "Finding Your Unknowns" article — see `README.md` for full attribution.