تشغيل أي مهارة في Manus بنقرة واحدة

creel-architecture

النجوم٠

التفرعات٠

آخر تحديث١٨ يونيو ٢٠٢٦ في ٠٥:٢٩

Use when working anywhere inside the creel package internals — the source-to-graph extraction engine. Covers the architecture map (layers/modules), the 15 design decisions D1–D15, the key Protocol interfaces, the canonical JSON contract, the two-layer (grammar vs bindings) join, and where each choice is justified in the research. Trigger when adding/changing core modules, wiring the extract() facade, the graph model, the spec/grammar layer, or canonical JSON; or when a design question needs the authoritative answer.

التثبيت

التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.

تشغيل في Manus

المصدر

thorwhalen

thorwhalen/creel

فتح مستودع GitHub عرض مستودعات المنشئ

تنزيل

تشغيل في Manus

المهن ذات الصلةSOC

استنادا إلى تصنيف SOC المهني

مطوّرو البرمجياتمهن الحاسوب والرياضيات·SOC 15-1252

SKILL.md

readonly

name	creel-architecture
description	Use when working anywhere inside the creel package internals — the source-to-graph extraction engine. Covers the architecture map (layers/modules), the 15 design decisions D1–D15, the key Protocol interfaces, the canonical JSON contract, the two-layer (grammar vs bindings) join, and where each choice is justified in the research. Trigger when adding/changing core modules, wiring the extract() facade, the graph model, the spec/grammar layer, or canonical JSON; or when a design question needs the authoritative answer.
metadata	{"audience":"developers"}

creel architecture

creel = extract(sources, graph_spec, extractors) -> graph. A typed Labeled Property Graph (LPG) is the single source of truth; everything downstream is a projection. Authoritative design: misc/docs/research/00-synthesis-and-design-implications.md (decisions D1–D15). Roadmap: misc/docs/design/ROADMAP.md. Decision log: misc/docs/design/DECISIONS.md — which now includes D-OP7 (ingestion layer, report R13), D-OP8 (extraction granularity = hybrid class-cluster passes, report R14), D-OP9 (bidirectional traceability + human annotation/coding as a first-class contract; 5 additive extensions A1–A5, report R15), and the 2026-06-17 resolutions of the open questions (notably #14 entity resolution = full cascade, required because the real consumer corpus is very messy).

Module map (single-package spike; uv-workspace split at v0.4 — D-OP3)

creel/
  facade.py          # extract() — the only public entry point
  ingest/            # INGESTION layer: files -> Source(s) (D-OP7, R13) [planned]
                     #   route-by-format + quality-gate; Docling default; markdown +
                     #   provenance sidecar (page/cell/char-span/bbox). See creel-ingestion.
  spec/              # GRAMMAR layer = graph-definition / SSOT (D1,D3,D7)
    model.py         #   GraphSpec, NodeType, EdgeType, AttrSchema (frozen dataclasses)
    validate.py      #   validate an instance graph against a GraphSpec
    linkml.py        #   [semantic] LinkML <-> GraphSpec; generate JSON Schema/Pydantic
  bindings.py        # EXTRACTION/VERIFICATION metadata layer (keyed by taxonomy path)
  join.py            # pure equijoin join(spec, bindings) -> ResolvedPlan (D7)
  graph/
    model.py         #   Graph: networkx.MultiDiGraph wrapper, stable IDs on nodes+edges (D1)
    canonical.py     #   to/from creel canonical JSON ($schema, version, sorted) (D4)
  extract/
    protocol.py      #   Extractor Protocol, ExtractionContext, Extraction (D5)
    pattern.py       #   regex / Callable  (the trivial default; brings the facade up)
    query.py         #   [query] DuckDB SQL (tables) + Mongo-filter/JMESPath (json)
    llm.py           #   [llm] schema-as-extractor via constrained structured output
    registry.py      #   decorator registry + entry points (creel.extractors) (D12)
    cache.py         #   Cache Protocol (no-op default), deterministic key (D11)
  verify/            # the EVALUATION-time dual (see creel-eval skill) (D9)
    protocol.py kinds.py rubric.py
  evidence.py        # provenance + grounding selectors + confidence (sidecar) (D8)
  view/projections.py# to_dot/to_mermaid/to_table/... (NOT renderers) (D15)
  render.py          # GraphRenderer Protocol + AnnotatedGraph contract (no concrete renderers)
  export/            # [lazy] networkx, cytoscape, dot, mermaid, rdf_star, neo4j_cypher (D2)

The core contracts (keep these stable; see synthesis §"Key Python interfaces")

extract(sources, graph_spec, extractors=None, *, verifiers=None, cache=None, on_missing_binding="schema_as_extractor") -> Graph — keyword-only past arg 3.
Extractor Protocol: extract(ctx: ExtractionContext) -> Extraction.
Verifier Protocol: verify(actual, expected, *, context) -> {score∈[0,1], passed, reason, details}.
Cache / GraphRenderer Protocols. All runtime_checkable; callables, not inheritance trees.

Non-negotiables (= the load-bearing decisions)

Edges are first-class with their own id + attrs (D1). Parallel edges between the same endpoints must stay distinguishable — test it.
Grammar and bindings are physically separate, joined by id (D7). Never fuse "what the graph is" with "how to extract it" into one object.
Grammar enforces shape; ranges/faithfulness are a verify pass (D6).
Canonical JSON is deterministic (sorted keys, id-sorted arrays, stable edge ids) — round-trip byte-identity is a tested invariant (D4).
Core stays tiny; new mechanisms enter via Protocol seams + registry/entry points, never by adding a pipeline framework to the spine (D10, D11, D12).

When the synthesis doesn't answer a question

Make the smallest defensible choice consistent with the load-bearing decisions, implement it, and append a D-OP* entry to misc/docs/design/DECISIONS.md (decision + rationale + rejected alternatives). Don't leave design choices implicit in code.

المزيد من هذا المستودع

نفس المستودع

creel-ai

thorwhalen/creel

Use a real LLM with creel — wire an injected LLM client, run AI-powered graph extraction, and judge/resolve with a model. Use when the user wants schema-as-extractor extraction, to inject an LLM client (services={'llm': aix_client()} or anthropic_client()), the ('llm', {}) binding, an LLM judge for llm_rubric verification, LLM entity resolution, self-consistency policy, or to test LLM code with a fake client. Triggers on "use a real LLM with creel", "extract with an LLM / GPT / Claude", "schema-as-extractor", "inject an LLM client", "aix_client / anthropic_client", "services={'llm':...}", "the ('llm',{}) binding", "LLM as a judge", "LLM entity resolution", "self-consistency voting", "ExtractionPolicy", "AI-powered graph extraction", "fake LLM client for tests". The LLM is a swappable strategy — no provider SDK lives in creel's core.

2026-07-240

creel-bindings

thorwhalen/creel

Write creel bindings: map each grammar element to an extractor strategy. Use when you need to choose an extractor strategy for a node-type or edge-type, decide pattern vs query vs LLM extraction, write regex_node/regex_edge over prose, or table_map/sql/json_query over structured sources. Covers the binding mental model ({element_id: (strategy, params)} or a bare callable), id templates and casts, endpoint templates for edges, exclude_groups, cluster bindings (cluster_llm), registering a custom extractor strategy, and how unbound elements behave (on_missing_binding). Trigger on: map grammar elements to extractors, route prose to regex and tables to query, bind a funds/measures edge, derive node ids, id_template/id_from, register_extractor a custom strategy, cluster_llm one-pass extraction, on_missing_binding, fix a binding error.

2026-07-240

creel-evaluation

thorwhalen/creel

Evaluate / verify a creel extraction — the answer to "is this extraction correct?" is a pluggable Verifier, NOT a hardcoded == . Use when you need to score extraction quality, compare an actual graph to an expected_graph, build a test corpus, or pick the right comparison per attribute. Covers the verifier-kind taxonomy (exact, normalized, numeric_tolerance, set_match, schema_constraint, semantic_similarity, composite), graph_match (decomposable partial credit with a structured mismatch report), and llm_rubric (a verifier defined by a natural-language criterion, graded by an injected LLM judge). Keywords: pluggable Verifier not equality, graph_match partial credit, numeric_tolerance, semantic similarity, natural-language LLM rubric verifier, score extraction quality.

2026-07-240

creel-extract

thorwhalen/creel

Extract a typed property graph from messy sources with creel. Use when the user wants to turn prose, tables, JSON, or PDFs/documents into a clean, typed graph — i.e. run extract(), build a property graph, go sources->graph, use the creel facade, or get started with creel. Triggers on "extract a graph from documents", "build a knowledge/property graph with creel", "creel quickstart", "how do I call extract()", "sources to graph", "turn this prose/table/JSON into a typed graph", "ingest a folder of files/PDFs into a graph", "the one-call creel workflow". This is the ENTRY/orchestrator skill: it covers the end-to-end extract(sources, graph_spec, extractors) -> graph call, its keyword args (services, on_missing_binding, resolve, cache), and how to read the returned Graph (nodes, edges, evidence, report, validate_graph, to_canonical_json). Hand off to creel-grammar, creel-bindings, creel-ai, creel-evaluation, creel-projections for the deeper pieces.

2026-07-240

creel-grammar

thorwhalen/creel

Use when you need to define a creel grammar (a GraphSpec) — the typed schema for the graph you want to extract. Covers declaring node-types and edge-types with AttrSchema attributes; first-class edges that carry their own typed attributes (a funds edge with amount/currency); attribute options (required, range, enum, minimum/maximum, pattern, multivalued, description); reusable EnumDef value-sets and named ranges; inheritance via is_a/abstract/mixins; validating a grammar (validate_spec) and an instance graph (validate_graph); and optionally authoring in LinkML to generate JSON-Schema or Pydantic. Trigger on "define a creel grammar", "declare node-types and edge-types", "typed attributes on edges", "enums/ranges/inheritance for graph extraction", "schema for graph extraction", "LinkML authoring", or "validate a spec".

2026-07-240

creel-projections

thorwhalen/creel

Project/transform a creel graph into everything downstream — persist, query, view, export, annotate are all projections of the one graph, never engine features. Use after you have a Graph and want to clean it up or get it OUT. Covers entity resolution (merge duplicate/same-entity nodes via Normalize/Registry/LLM/Cascade resolvers, the resolve= facade arg, or resolve_graph); the reify/unreify toggle (attributed edge ⇄ relation node when it goes n-ary, losslessly); view projections (to_node_edge_records/to_table/to_dot/to_mermaid/to_cytoscape); export adapters (to_jgf/to_graphml/to_cypher/to_turtle — GraphML, JGF, parameterized Cypher for Neo4j, RDF-star Turtle); the annotate/render contract; and trace (per-attribute grounding, TraceIndex reverse lookup, reanchor to source span). Triggers on "merge duplicate nodes", "entity resolution", "reify edge to node", "view as Mermaid/DOT/Cytoscape", "export to GraphML/Cypher/RDF/Turtle", "annotate a graph", "trace a value to its source span", "downstream rendering".

2026-07-240

name	creel-architecture
description	Use when working anywhere inside the creel package internals — the source-to-graph extraction engine. Covers the architecture map (layers/modules), the 15 design decisions D1–D15, the key Protocol interfaces, the canonical JSON contract, the two-layer (grammar vs bindings) join, and where each choice is justified in the research. Trigger when adding/changing core modules, wiring the extract() facade, the graph model, the spec/grammar layer, or canonical JSON; or when a design question needs the authoritative answer.
metadata	{"audience":"developers"}

creel architecture

Module map (single-package spike; uv-workspace split at v0.4 — D-OP3)

creel/
  facade.py          # extract() — the only public entry point
  ingest/            # INGESTION layer: files -> Source(s) (D-OP7, R13) [planned]
                     #   route-by-format + quality-gate; Docling default; markdown +
                     #   provenance sidecar (page/cell/char-span/bbox). See creel-ingestion.
  spec/              # GRAMMAR layer = graph-definition / SSOT (D1,D3,D7)
    model.py         #   GraphSpec, NodeType, EdgeType, AttrSchema (frozen dataclasses)
    validate.py      #   validate an instance graph against a GraphSpec
    linkml.py        #   [semantic] LinkML <-> GraphSpec; generate JSON Schema/Pydantic
  bindings.py        # EXTRACTION/VERIFICATION metadata layer (keyed by taxonomy path)
  join.py            # pure equijoin join(spec, bindings) -> ResolvedPlan (D7)
  graph/
    model.py         #   Graph: networkx.MultiDiGraph wrapper, stable IDs on nodes+edges (D1)
    canonical.py     #   to/from creel canonical JSON ($schema, version, sorted) (D4)
  extract/
    protocol.py      #   Extractor Protocol, ExtractionContext, Extraction (D5)
    pattern.py       #   regex / Callable  (the trivial default; brings the facade up)
    query.py         #   [query] DuckDB SQL (tables) + Mongo-filter/JMESPath (json)
    llm.py           #   [llm] schema-as-extractor via constrained structured output
    registry.py      #   decorator registry + entry points (creel.extractors) (D12)
    cache.py         #   Cache Protocol (no-op default), deterministic key (D11)
  verify/            # the EVALUATION-time dual (see creel-eval skill) (D9)
    protocol.py kinds.py rubric.py
  evidence.py        # provenance + grounding selectors + confidence (sidecar) (D8)
  view/projections.py# to_dot/to_mermaid/to_table/... (NOT renderers) (D15)
  render.py          # GraphRenderer Protocol + AnnotatedGraph contract (no concrete renderers)
  export/            # [lazy] networkx, cytoscape, dot, mermaid, rdf_star, neo4j_cypher (D2)

The core contracts (keep these stable; see synthesis §"Key Python interfaces")

extract(sources, graph_spec, extractors=None, *, verifiers=None, cache=None, on_missing_binding="schema_as_extractor") -> Graph — keyword-only past arg 3.
Extractor Protocol: extract(ctx: ExtractionContext) -> Extraction.
Verifier Protocol: verify(actual, expected, *, context) -> {score∈[0,1], passed, reason, details}.
Cache / GraphRenderer Protocols. All runtime_checkable; callables, not inheritance trees.

Non-negotiables (= the load-bearing decisions)

Edges are first-class with their own id + attrs (D1). Parallel edges between the same endpoints must stay distinguishable — test it.
Grammar and bindings are physically separate, joined by id (D7). Never fuse "what the graph is" with "how to extract it" into one object.
Grammar enforces shape; ranges/faithfulness are a verify pass (D6).
Canonical JSON is deterministic (sorted keys, id-sorted arrays, stable edge ids) — round-trip byte-identity is a tested invariant (D4).
Core stays tiny; new mechanisms enter via Protocol seams + registry/entry points, never by adding a pipeline framework to the spine (D10, D11, D12).