kb-with-receipts

# kb-with-receipts — Sourced Knowledge Base Builder A battle-tested method for "growing" an assistant that answers questions about your product/operations **and cites where every answer came from**. Distilled from a real internal deployment (anonymized) that took ~300 scattered source files to a **97%-correct, zero-hallucination, fully-traceable** Q&A knowledge base. This Skill is the **method**, not a specific knowledge base. Use it to build your own. --- ## The core idea (read this first) > **Maintain ONE clean set of structured Markdown (the Single Source of Truth), and feed it to TWO outputs.** 1. **A no-code Q&A tool** (e.g. NotebookLM) — for non-technical teammates to ask questions by typing. Picked because it gives inline, click-back-to-source citations, has zero learning curve, and costs ~nothing. 2. **A reusable Claude Skill** (`SKILL.md` + `references/`) — for advanced queries and automation. Update only the SSOT → both outputs stay in sync. Never maintain two copies of the truth. **Four non-negotiable principles** (the source of the quality): - **Extractive, not abstractive** — move and reorganize text; never paraphrase facts. Numbers, amounts, dates, rules, field names, status codes are copied **verbatim**. Abstractive rewrites read nicely but hallucinate; for ops/spec knowledge, factual consistency beats fluency. - **Per-section source tags** — every section starts with `> Source: {original-filename} | {date}`. This both improves retrieval (contextual retrieval) and lets the Q&A tool surface real provenance. - **Time-ordering as conflict resolution** — when a topic has multiple versions, the **newest** wins; older versions only backfill gaps the new one didn't cover, with the evolution explicitly annotated. - **Faithful-to-source ≠ correct** — source docs may contain drafts, cross-contaminated requirements, or unimplemented "vision" items. Verbatim consolidation drags these in. Only a domain expert sampling answers can catch them → recycle every catch into the regression test bank. --- ## The pipeline at a glance Build it like training a new librarian: give clean material, then test, correct, and fill gaps. | Stage | What | Reference | |---|---|---| | 0 | **Inventory & taxonomy** — list every source, flag empties/drafts/dupes, bucket topics mutually-exclusively | `references/02-technical-pipeline.md` | | 1 | **Consolidate text** (parallel agents) — dedupe, time-order, extractive move, source-tag each section | `references/02-technical-pipeline.md` | | 2 | **Videos → text** — scene-change frames + transcript → timestamped illustrated steps | `references/02-technical-pipeline.md` | | 3 | **Design files → text** — REST API render of screens → read visually → write text specs | `references/02-technical-pipeline.md` | | 4 | **Publish to the Q&A tool** — programmatic upload, split by audience | `references/05-qa-tool-gotchas.md` | | 5 | **QA loop** — ask the most likely questions, grade vs. source, fix, re-test | `references/03-qa-and-audit.md` | | 6 | **Integrity audit** — three-way cross-check to kill "source has it, KB missing it" gaps | `references/03-qa-and-audit.md` | | ∞ | **Monthly update** — re-export, auto-refine, sync; human only arbitrates conflicts & samples | `references/04-monthly-update-sop.md` | For the plain-language "why" and the tool-selection rationale, read `references/01-method-overview.md`. --- ## Routing table — which reference to load for which job | If the user wants to… | Read | |---|---| | Understand the approach / decide if it fits | `references/01-method-overview.md` | | Actually condense raw docs/design/video into a KB | `references/02-technical-pipeline.md` | | Set up the QA loop and integrity audit | `references/03-qa-and-audit.md` | | Keep the KB fresh every month | `references/04-monthly-update-sop.md` | | Avoid the Q&A tool's silent quality killers | `references/05-qa-tool-gotchas.md` | | **Resolve "two files say different things"** contradictions | `references/06-consensus-pages.md` | | **Audit the whole KB adversarially** before shipping | `references/07-adversarial-audit.md` | | **Turn the KB into a chat bot + a correction loop** | `references/08-correction-flywheel.md` | | **Serve non-technical and technical users from one KB** | `references/09-audience-adaptive-answers.md` | | **Answer "does our database support feature X?"** | `references/10-schema-grounding.md` | | **Let an agent safely edit the KB (verified)** | `references/11-maker-verifier-loop.md` | | **Give the KB a chat-app (LINE) face — only after Q&A works** | `references/12-notebooklm-line-bot.md` | --- ## When you apply this Skill, do this 1. **Start with the overview** (`references/01-method-overview.md`) to confirm the approach fits the user's situation (do they need citations? non-technical audience? low maintenance?). If a fully-custom RAG or an enterprise search platform is genuinely warranted, say so. 2. **Inventory before building** — never start consolidating before Stage 0 taxonomy is done; it's what prevents the same file being processed twice. 3. **Enforce the four principles** on every consolidated file. The single most common failure is silently paraphrasing a number. 4. **Resolve contradictions with consensus pages** (`references/06-consensus-pages.md`) instead of silently picking one side — add an authoritative concept page and backlink the sources. 5. **Always run the QA loop, the integrity audit, and the adversarial audit** before declaring done (`references/03` + `references/07`) — "it answers" is not "it answers correctly and completely." 6. **Mind the Q&A-tool gotchas** (`references/05-qa-tool-gotchas.md`) — e.g. NotebookLM silently drops fenced code blocks; run `scripts/make_nlm_upload.py` on any file containing code/SQL/JSON before uploading. 7. **Hand off a monthly update SOP and a correction flywheel** (`references/04` + `references/08`) — this is a living asset, not a one-shot deliverable. 8. **If an agent will edit the KB unattended**, run it as a maker/verifier loop (`references/11-maker-verifier-loop.md`) with a checksum safety net. 9. **Offer a chat-app (LINE) face only as a progressive, opt-in step — never pitch it up front.** The gate has three parts, in order: (a) **first confirm the KB is actually live on NotebookLM and answering correctly** (the QA loop passed — a bot on a hallucinating KB just distributes the hallucination faster); (b) **then ask the user** whether they want to put it on a chat app such as LINE; (c) **only if the user says yes, load `references/12-notebooklm-line-bot.md`** and proceed. Do not push reference 12, the bot, or the template before asking — surfacing it unsolicited is a violation of this gate. ## Reusable artifacts in this template - `scripts/make_nlm_upload.py` — converts SSOT Markdown's fenced code blocks to plain-text-marked form so the Q&A tool can index SQL/commands verbatim. (Critical: see gotchas.) - `references/03-qa-and-audit.md` — the QA grading rubric (✅/⚠️/❌) and the four-layer integrity audit. - `references/04-monthly-update-sop.md` — the repeatable monthly refresh + CI-style coverage diff. - `examples/` — a fully fictional before/after showing scattered, contradictory docs consolidated into a sourced SSOT. - `templates/line-bot-service/` — a de-identified, deployable LINE-bot service (FastAPI + the `notebooklm` CLI) that gives your KB a chat-app face. Ship only after the progressive gate above; see `references/12-notebooklm-line-bot.md`. ## Methodology sources (index) - Anthropic — *Contextual Retrieval*: prepend per-chunk context/source to cut retrieval failures (~35% in Anthropic's tests). Basis for the per-section source tags. — https://www.anthropic.com/news/contextual-retrieval - Anthropic — *Agent Skills best practices*: progressive disclosure (lean `SKILL.md` router + `references/` loaded on demand, avoid deep nesting). Basis for this template's structure. — https://platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices - RAG chunking practice: Markdown is the most LLM-native format; heading-aware splitting beats fixed-length; extractive beats abstractive for lower hallucination.