kg-builder

name	kg-builder
description	Designs and builds knowledge graphs from documents — ontology modeling with domain/range constraints, entity/relation/event extraction, entity resolution, provenance and supersession, and GraphRAG serving. Use when asked to "build a knowledge graph", "design an ontology", "extract entities and relations", "deduplicate entities", "entity resolution", "add GraphRAG", or "graph memory for an agent". NOT for multi-agent task graphs or agent orchestration, use task-decomposer.
metadata	{"version":"1.0.0","category":"data","tags":["knowledge-graph","ontology","entity-resolution","graphrag","provenance"],"difficulty":"advanced","phase":"build"}

KG Builder

A knowledge graph is a product with a schema, not a pile of triples. Quality comes from pipeline order: model the domain before extracting, validate during extraction, fuse before storing, and attach provenance to every fact from the first write.

This skill covers the full build — value test, ontology, extraction, quality gate, entity resolution, serving, and maintenance — plus the boundary question that decides whether the result is trustworthy: which stages are deterministic code and which are LLM judgment.

Scope note. This is about knowledge graphs — what an agent remembers. It is not about task graphs, agent orchestration, or multi-agent topology.

Reference Files

File	Contents	Load When
references/ontology-design.md	Competency questions, entity/relation types, domain/range, storage choice	Phase 1
references/extraction.md	Source routing, NER/RE/EE prompt patterns, validation, failure modes	Phase 2
references/fusion.md	Blocking, matching layers, merge policy, threshold bands	Phase 3
references/serving.md	GraphRAG retrieval, path queries, community summaries, query layer	Phase 4
references/provenance-and-supersession.md	Claim model, append-only updates, contradiction handling, audit trail	Phase 1 and Phase 4

The deterministic / LLM boundary

Decide this before writing code. Code owns control flow, identity, validation, and merges. The model gets contained judgments behind a typed interface, each with a measured baseline.

Stage	Deterministic (code)	LLM judgment (measure it)
Source routing	format detection, structured mapping	—
Entity extraction	span capture, type validation, dictionary matching	"what entities are in this text"
Relation extraction	domain/range enforcement, endpoint checks	"which relation does this sentence assert"
Quality gate	sampling, scoring, thresholds	—
Blocking	key generation, candidate pairing	—
Matching	string/attribute/structure scoring	ambiguous middle band only
Merge	canonical selection, edge union, lineage	— (never let a model own a merge)
Serving	traversal, subgraph selection, serialization	the agent's own reasoning

Measure every LLM surface against a prompt-only baseline before trusting it. This is not theoretical caution. In a pre-registered real-model evaluation of an LLM-adjudicated dedup and contradiction loop, the loop trailed a plain prompt-only baseline by 0.28–0.33 on detection and safety across every provider cell tested. Adjudication that is not measured is decoration.

Workflow

Phase 1 — Design (do not skip)

1. Value test. A graph pays off when queries are multi-hop ("who worked with X on projects using Y"), when entities recur across documents, or when the relationships are the data. If every query is a single-hop lookup or an aggregation, use a table and stop here. Write the kill criterion down before continuing.
2. Competency questions. Write the 10–20 questions the graph must answer. These are the ontology's spec and its test suite. Anything you cannot path through the finished schema is a missing type or relation.
3. Ontology. 5–15 entity types, 10–30 relation types, each relation with explicit domain and range. Precise verb names (ACQUIRED, DEPENDS_ON) — never RELATED_TO. Keep it in ontology.yaml as the single source of truth; every extraction prompt embeds it verbatim.
4. Storage and identity. Choose property graph (default), RDF/OWL (interop, description-logic reasoning), or typed edges in SQLite (<50K nodes). Decide now how time and provenance attach to every fact — retrofitting provenance after fusion is effectively impossible.

Validate the schema before extracting anything:

uv run scripts/validate_ontology.py ontology.yaml

Phase 2 — Extract

5. Route by source type. Structured sources (databases, CSVs, APIs) map column → type in deterministic code with no model involved. Semi-structured sources (HTML tables, infoboxes) get per-layout parsers. Only unstructured text enters the LLM pipeline. Running NLP over already-structured data is the classic waste.
6. Entities. Dictionaries and exact rules first for closed vocabularies — free, deterministic, perfect precision. LLM extraction for open text, with the ontology in the prompt. Always capture surface form, canonical guess, type, source pointer, and confidence.
7. Relations. Extract only between entities that already passed step 6; never let relation extraction invent endpoints. Constrain output to the ontology's relation list and validate domain/range in code. Require a verbatim evidence quote that asserts the relation — co-occurrence is not assertion ("Musk discussed Twitter" is not OWNS).
8. Events. For dynamic domains, extract events as first-class nodes (trigger + typed arguments + time), never flattened into pairwise edges — flattening loses which acquisition happened at which price.

Phase 3 — Consolidate

9. Quality gate. Sample 50 items and score entity precision and relation precision before fusing anything. Target ≥90% precision. Fix the prompt or the rules, then re-run — never hand-patch the output. Recall improves with more passes; bad precision poisons the graph permanently.
10. Fusion. Blocking → matching → merge. Blocking avoids O(n²) comparison; matching scores string, attribute, and structural evidence (two J. Smith nodes sharing three coauthors and an affiliation are one person; identical names with disjoint neighborhoods are not); merge policy is deterministic code that keeps the canonical name, unions aliases and edges, preserves conflicting values with provenance, and records merged_from for undo. An erroneous merge is far more damaging than a missed one — it silently fuses two entities' entire edge sets. Auto-merge only above the high band; queue the middle for review.

Check the blocking strategy against labeled pairs before running it at scale:

uv run scripts/blocking_report.py candidates.jsonl --labels matches.jsonl

Phase 4 — Serve and maintain

11. Serving. Entity-link the query, expand 1–2 hops, serialize the subgraph as compact (head)-[REL {time, source}]->(tail) lines grouped by head. For multi-hop questions retrieve paths between the query's entities, not neighborhoods around each — the path is the answer skeleton. Cluster and pre-summarize for "what are the themes" questions.
12. Maintenance. New facts supersede rather than overwrite: keep the prior claim with status, supersedes, and validity interval. When a new fact contradicts a stored one, keep both with time and provenance and prefer the newer at retrieval. Re-run fusion periodically — unmaintained memory graphs rot exactly like unfused extractions.

Output

Deliver these artifacts, in this order:

Artifact	Contents
competency.md	The 10–20 questions, each marked answerable or blocked
ontology.yaml	Entity types, relation types with domain/range, event argument schemas
extraction/	Per-source-type prompts and deterministic mappings
quality-report.md	Sampled entity and relation precision, with sample size and method
fusion-report.md	Blocking reduction ratio, pair recall, merge counts per band
The graph	Nodes and edges, every one carrying source, extracted_at, confidence

Report precision as a sampled estimate with its sample size. A precision number without a stated sampling method is a vibe.

Working rules

• Schema first, always. Extraction without an ontology produces a word cloud with arrows. If the user resists schema design, induce a minimal 5-type ontology from three sample documents and show it for approval — never skip to extraction.
• Provenance on every fact. source, extracted_at, confidence. Non-negotiable; fusion and trust both depend on it.
• Pilot before scale. Run 10 documents through all four phases first. The pilot exposes ontology gaps at 1% of the cost.
• Never auto-accept an induced schema. LLM-proposed ontologies overfit their sample documents. Prune to the minimal set that answers the competency questions.
• The LLM is stage machinery, not the pipeline. It slots into extraction and the ambiguous matching band. The surrounding schema, validation, and merge policy are what make the output a knowledge graph rather than a transcript.

Errors and troubleshooting

Symptom	Cause	Fix
Graph full of Concept/Thing nodes	Extracted without an ontology	Phase 1 first, then re-extract
Same person appears as four nodes	No canonical-form rule; fusion skipped	Define the rule in ontology.yaml; run Phase 3
Confident but wrong relations	Co-occurrence treated as assertion	Require evidence quotes; enforce domain/range in code
Events flattened into edge soup	No event argument schema	Promote events to first-class nodes with typed arguments
Precision collapses as sources grow	One prompt drifting across document types	Per-source-type prompts; run the quality gate per source
Fusion merges two real entities	Threshold too low; no structural layer	Raise the auto-merge band; add neighborhood comparison; undo via merged_from
Multi-hop answers are wrong but fluent	Unfused duplicates break paths	Re-run fusion; paths cannot cross duplicate boundaries
GraphRAG returns noise	Hop expansion too wide	Cap at 2 hops, or re-rank; retrieve paths, not neighborhoods
Retrieval is stale after updates	Facts overwritten instead of superseded	Adopt the claim model in references/provenance-and-supersession.md