| name | general-graph-extraction |
| description | Use this skill whenever a user wants to extract entities and relationships from non-financial documents to build a queryable knowledge graph. Triggers include: codebases and technical documentation, internal wikis, product specs, README files, meeting notes, support tickets, HR or policy documents, onboarding materials, knowledge bases, Notion/Confluence exports, API docs, and general text corpora. Also trigger when the user says "map dependencies in my codebase", "graph this doc", "extract knowledge from this wiki", "connect these notes", "build a graph from my repo", or "index this for querying". Use the financial-graph-extraction skill instead when the document is a financial filing, earnings report, or analyst document. Output must follow the GraphRAG ingestion schema: document_id + entities + relationships + chunks, with chunk-based source_id values.
|
General Graph Extraction Skill
Agent Runtime Contract
Use this skill when the raw input file is in to_be_extracted/ and the content is not primarily financial or academic literature.
Execution contract:
- Call
get_server_status() before starting extraction.
- If overall is
degraded, explain what is degraded, what still works, and ask whether to proceed or fix first.
- Read one raw source file from
to_be_extracted/ at a time unless the user explicitly asks for a combined extraction.
- For the best graph quality, prefer
.md and .txt source files. If the source is PDF or heavily formatted export text, assume there is no built-in repo parser or OCR layer and be conservative about layout noise.
- Write exactly one extraction JSON for that source file to
extractions/{source_filename}_extracted.json.
- Validate entity, relationship, and chunk integrity before ingestion.
- If the extraction has duplicates, orphaned relationships, or consistency issues, run reconciliation before ingestion.
- After a clean extraction is written, call
ingest_from_file.
What This Skill Does
This skill guides an agent to read a document or codebase and produce a structured JSON extraction — entities and relationships — that can be ingested into a knowledge graph via the ingest_from_file MCP tool.
Unlike financial extraction (which enforces strict numerical precision), general extraction prioritizes semantic richness and connection density — the goal is to enable natural-language queries like:
- "What does the AuthService depend on?"
- "Which modules are owned by the payments team?"
- "What broke when we changed the database schema?"
- "Who are the experts on the recommendation engine?"
The goal is not to graph every noun. The goal is to build a graph with high-signal, reusable structure:
- who owns what
- what depends on what
- what decisions were made
- what actions, risks, blockers, and systems matter
- what evidence supports those links
General Reading Strategy
Read the source in layers instead of linearly dumping entities:
Pass 1 — Identify document subtype and signal
Classify the source before extracting:
- codebase or technical docs
- internal wiki or knowledge base
- meeting notes or ticket
- policy or process document
- product spec or decision record
During this pass, decide:
- what the main units of knowledge are
- what should count as entities versus plain context
- which sections are likely high signal
- whether the source is clean enough for dense extraction
Pass 2 — Extract structure first
Prioritize:
- named systems, modules, APIs, tools, teams, people, processes, decisions, projects
- explicit ownership
- explicit dependencies
- procedural order
- blockers, action items, and operational constraints
Pass 3 — Add only useful semantic detail
Add broader concepts only when they help downstream queries:
- architecture patterns
- operational policies
- domain concepts
- known issues
If the text is noisy, repetitive, or weakly structured, reduce extraction density instead of inflating the graph.
Extraction Output Format (Repo-Compatible)
Write to extractions/{source_filename}_extracted.json.
This repo requires document_id, entities, relationships, and chunks.
{
"document_id": "backend_readme",
"file_path": "README.md",
"timestamp": 1727740800,
"entities": [...],
"relationships": [...],
"chunks": [...]
}
Chunks (Required)
Chunks are the evidence base. Every entity and relationship must point to a chunk via source_id.
{
"chunk_id": "chunk_001",
"content": "The OrderService is owned by the Payments team and calls InventoryService.",
"chunk_order_index": 1,
"file_path": "README.md"
}
**Chunk size limits (required):**
- Target 400-800 characters per chunk (or ~200-300 tokens if you have a tokenizer).
- Hard cap 1,000 characters per chunk. If a paragraph/table is longer, split it.
Domain Subtypes and Entity Profiles
Choose the right entity profile based on what you're processing:
1. Codebase / Technical Docs
| Entity Type | Examples |
|---|
MODULE | auth_service, payment_processor |
FUNCTION | validate_token(), create_order() |
CLASS | UserRepository, InvoiceBuilder |
API_ENDPOINT | POST /api/v2/orders |
DATABASE_TABLE | users, transactions |
CONFIG_KEY | MAX_RETRY_COUNT, DB_URL |
DEPENDENCY | numpy, FastAPI, Redis |
TEAM | Platform Team, Payments Squad |
CONCEPT | idempotency, circuit breaker |
Relevant relationship keywords: IMPORTS, CALLS, OWNS, DEPENDS_ON, DOCUMENTS, IMPLEMENTS, EXTENDS, STORES_IN, EXPOSES
2. Internal Wiki / Knowledge Base
| Entity Type | Examples |
|---|
PERSON | Sanjay Elango (author/owner) |
TEAM | Backend Infrastructure |
PROCESS | Deployment Pipeline, Oncall Rotation |
TOOL | Datadog, PagerDuty, Terraform |
CONCEPT | SLA definition, retry policy |
DOCUMENT | Runbook: DB failover |
DECISION | ADR-042: Use Postgres over MySQL |
Relevant relationship keywords: OWNS, WRITTEN_BY, REFERENCES, SUPERSEDES, IMPLEMENTS, USES, DESCRIBES, DECIDES
3. Meeting Notes / Tickets
| Entity Type | Examples |
|---|
PERSON | Named attendees/assignees |
ACTION_ITEM | "Fix token expiry bug by Friday" |
DECISION | "We'll use Redis for session storage" |
BLOCKER | "Waiting on security review" |
PROJECT | Project Phoenix |
DATE | 2024-10-15 (target date) |
Relevant relationship keywords: ASSIGNED_TO, BLOCKS, RELATES_TO, DECIDED_IN, DUE_BY
Universal Entity Schema (Required Fields)
{
"entity_name": "auth_service",
"entity_type": "MODULE",
"description": "Authentication service that issues and validates JWTs; owner=platform_team; path=services/auth/.",
"source_id": "chunk_002",
"file_path": "services/auth/README.md"
}
Rules:
entity_name: unique, snake_case, no spacesentity_type: pick from the relevant profile listdescription: include concrete facts and properties from the text (owner, path, constraints)source_id: must match a real chunk_id in chunks- prefer one strong entity per real concept, not many aliases for the same thing
Universal Relationship Schema (Required Fields)
{
"src_id": "order_service",
"tgt_id": "payment_processor",
"description": "OrderService calls PaymentProcessor to charge the customer during checkout.",
"keywords": "CALLS,context=checkout_flow",
"source_id": "chunk_001",
"weight": 1.0,
"file_path": "README.md"
}
Rules:
src_id and tgt_id must be valid entity_name valueskeywords should include the relationship type (e.g., CALLS, OWNS, DEPENDS_ON)- Add
context=... or source_section=... in keywords when useful
- Directed relationships: source → target; for bidirectional add both directions
- only create a relationship when the source text gives direct support for it or the implication is extremely clear from local context
What To Ignore
Do not extract:
- repeated headers, navigation text, sidebars, export boilerplate, template labels
- generic filler like "this document describes" unless it carries a real relationship
- every file path, import, or config key in bulk when they add no query value
- every attendee name in notes if they are not tied to a decision, action, or ownership signal
- duplicate concepts restated in multiple sections
Prefer signal over completeness. A smaller, cleaner graph is better than a bloated one.
Extraction Quality Principles
For Codebases
- Trace dependencies: what imports what, what calls what
- Map ownership: which team/person owns which module
- Surface configuration: key env vars and their consuming modules
- Capture architectural patterns mentioned (MVC, event-driven, etc.)
- Note TODOs/FIXMEs as
KNOWN_ISSUE entities if they're significant
- Distinguish between implementation details and stable concepts; prefer stable concepts unless the user clearly wants code-level granularity
For Documentation
- Every named concept becomes an entity
- Every "see also", "refer to", "documented in" becomes a
REFERENCES relationship
- Every "owned by", "maintained by" becomes an
OWNS relationship
- Procedural steps → sequence of
PROCESS entities linked by PRECEDES
- Collapse repeated explanations into one canonical entity with the clearest supporting chunk
For Notes/Tickets
- Every named person is an entity
- Every task or action item is an entity
- Capture blockers as explicit
BLOCKER entities with BLOCKS relationships
- Favor decisions, owners, due dates, and blockers over conversational filler
Extraction Density vs. Precision Trade-off
| Dimension | Financial Extraction | General Extraction |
|---|
| Precision | Exact (numbers must match) | Semantic (meaning over literal match) |
| Coverage | Deep in financial semantics | Broad across any domain |
| Ambiguity | Resolve strictly (no inference) | Light inference OK if clearly implied |
| Missing values | Leave blank, don't guess | Infer from context if confident |
In general extraction, if a README says "the auth module handles login and token refresh", it's reasonable to extract a CONCEPT entity token_refresh and link it to the auth module via IMPLEMENTS even if not explicitly stated that way.
But do not stretch this into speculation. If the text is ambiguous, prefer fewer entities and relationships.
Better Chunking For General Sources
Use chunk boundaries that preserve meaning:
- one architectural paragraph
- one decision paragraph
- one action-item block
- one API section
- one process step group
Avoid chunks that mix unrelated topics just to hit a length target. Semantic coherence matters more than uniform size.
Better Graph Outcomes
Optimize extraction for the questions users actually ask later:
- ownership: who owns this system, process, or document
- dependency: what calls, uses, imports, or relies on what
- decision history: what changed, why, and what it replaced
- operations: what blocks what, what must happen first, what is risky
- expertise: who is associated with which subsystem or domain
If an extracted entity or relationship is unlikely to help answer one of those question types, think twice before adding it.
Step-by-Step Agent Workflow
1. CALL get_server_status() and surface degraded modes before continuing
2. READ the full document (or repo files in scope)
3. IDENTIFY domain subtype (codebase, wiki, notes, etc.)
4. CHUNK the text into evidence blocks (sections, paragraphs, or tables)
- Assign chunk_id like chunk_001, chunk_002...
5. SCAN for all named entities — people, modules, tools, concepts, processes
6. EXTRACT entities with required fields and chunk-based source_id
7. EXTRACT relationships by asking: "how does X relate to Y in this text?"
8. VALIDATE:
- No entity_name duplicates
- All relationships reference existing entity_name values
- Every entity/relationship source_id matches a chunk_id
- Output path matches the current source file name
9. WRITE to extractions/{filename}_extracted.json
10. CALL ingest_from_file MCP tool
11. CALL list_pending_summaries() and resolve any flagged entities/relationships
via submit_summary(...) before finishing — Preciso never compresses
descriptions itself, so this step is the only way descriptions
exceeding raw_tail_size ever get summarized
Example: Codebase README Extraction
Input text:
"The OrderService module is owned by the Payments team. It calls InventoryService to check stock before calling PaymentProcessor to charge the customer. It stores order records in the orders table of the PostgreSQL database."
Output:
{
"document_id": "backend_readme",
"file_path": "README.md",
"timestamp": 1727740800,
"chunks": [
{
"chunk_id": "chunk_001",
"content": "The OrderService module is owned by the Payments team. It calls InventoryService to check stock before calling PaymentProcessor to charge the customer. It stores order records in the orders table of the PostgreSQL database.",
"chunk_order_index": 1,
"file_path": "README.md"
}
],
"entities": [
{
"entity_name": "order_service",
"entity_type": "MODULE",
"description": "OrderService module owned by Payments team.",
"source_id": "chunk_001",
"file_path": "README.md"
},
{
"entity_name": "inventory_service",
"entity_type": "MODULE",
"description": "InventoryService called to check stock before charging.",
"source_id": "chunk_001",
"file_path": "README.md"
},
{
"entity_name": "payment_processor",
"entity_type": "MODULE",
"description": "PaymentProcessor used to charge the customer.",
"source_id": "chunk_001",
"file_path": "README.md"
},
{
"entity_name": "payments_team",
"entity_type": "TEAM",
"description": "Payments team owns OrderService.",
"source_id": "chunk_001",
"file_path": "README.md"
},
{
"entity_name": "orders_table",
"entity_type": "DATABASE_TABLE",
"description": "orders table in PostgreSQL storing order records.",
"source_id": "chunk_001",
"file_path": "README.md"
}
],
"relationships": [
{
"src_id": "payments_team",
"tgt_id": "order_service",
"description": "Payments team owns OrderService.",
"keywords": "OWNS",
"source_id": "chunk_001",
"weight": 1.0,
"file_path": "README.md"
},
{
"src_id": "order_service",
"tgt_id": "inventory_service",
"description": "OrderService calls InventoryService to check stock.",
"keywords": "CALLS,context=stock_check",
"source_id": "chunk_001",
"weight": 1.0,
"file_path": "README.md"
},
{
"src_id": "order_service",
"tgt_id": "payment_processor",
"description": "OrderService calls PaymentProcessor to charge the customer.",
"keywords": "CALLS,context=checkout_flow",
"source_id": "chunk_001",
"weight": 1.0,
"file_path": "README.md"
},
{
"src_id": "order_service",
"tgt_id": "orders_table",
"description": "OrderService stores order records in orders table.",
"keywords": "STORES_IN,db=PostgreSQL",
"source_id": "chunk_001",
"weight": 1.0,
"file_path": "README.md"
}
]
}
After Extraction
- Run
ingest_from_file("extractions/{filename}_extracted.json") via MCP
- If ingestion fails, use
reingest_from_file(...) to replay without re-extracting
- Call
list_pending_summaries(). Preciso never compresses descriptions with an LLM — if any entity/relationship is flagged (its raw description history exceeded raw_tail_size), write a concise summary of prior_summary + old_descriptions and call submit_summary(...) for each one before considering the job done
- Sample queries after ingestion:
"What does OrderService depend on?""Which tables does the Payments team write to?""Who owns InventoryService?"
Large Document Handling
When to use subagents
Never for extraction.
Always extract the full document yourself.
Your entity registry prevents name drift.
Spawn reconciliation subagents only when:
- Extraction produces more than 150 entities
- Extraction produces more than 300 relationships
- You want a quality check before ingestion
How to spawn reconciliation subagents
After writing your extraction file:
Step 1: Spawn 3 reconciliation subagents in parallel.
Give each subagent the extraction JSON.
Subagent 1 gets: entities list only
Task: find entity name variants and duplicates
Output: extractions/{filename}_patch_entities.json
Subagent 2 gets: relationships list only
Task: find duplicate relationships and conflicts
Output: extractions/{filename}_patch_relationships.json
Subagent 3 gets: entities + relationships
Task: find broken references (src_id or tgt_id
not in entities list)
Output: extractions/{filename}_patch_orphans.json
Step 2: Wait for all 3 subagents to finish.
Step 3: Call ingest_with_reconciliation([
"extractions/{filename}_extracted.json",
"extractions/{filename}_patch_entities.json",
"extractions/{filename}_patch_relationships.json",
"extractions/{filename}_patch_orphans.json"
])
Reconciliation subagent instructions
When you are a reconciliation subagent:
- You are cleaning data, not extracting from documents
- Be conservative: only flag what is clearly wrong
- For entity merges: only merge if you are certain
they refer to the same real-world entity
- For temporal entities (FY2024, Q3 2024):
NEVER merge even if semantically similar
Different periods = different entities always
- Write your patch file and stop
- Do not call any ingest tool
