thorwhalen

Use this skill whenever the user wants to add, refactor, or audit the agent runtime — the loop that drives multi-step tool-using LLM conversations, optionally with durable execution for long-running, resumable agents. Triggers include "add an agent loop", "make my agent multi-step", "stop the agent from looping forever", "resume an agent after a crash", "run an agent in the background", "agent handoff to sub-agent", "add Pydantic AI / LangGraph / OpenAI Agents SDK", "wrap my agent in Temporal / DBOS / Inngest / Restate", "stream agent state to the UI", "cancel a running agent", "human-in-the-loop approval mid-run", or any task on the agent orchestration or durable execution layers. Read BEFORE writing a custom agent loop.

Use this skill whenever the user wants to add, audit, refactor, or maintain an embedded AI assistant (chat + agentic) inside their own application. Triggers include phrases like "add an AI assistant to my app", "wire up a chatbot", "embed Claude/GPT into my UI", "make my app operable by AI", "audit my AI assistant integration", "what should I add next for my AI assistant", or any task that touches multiple of {chat UI, system prompts, skills, MCP tools, agent runtime, command dispatch → MCP, multi-tenancy, observability, billing} together. This is the entry-point skill — it audits the codebase, identifies the layer in question, and routes to the focused sub-skills (ai-assistant-chat-ui, ai-assistant-prompts-skills, ai-assistant-command-mcp, ai-assistant-agent-runtime).

Use this skill whenever the user wants to add, modify, or audit the chat UI / streaming / wire-protocol layer of an embedded AI assistant. Triggers include "build a chat UI", "add streaming responses", "render tool calls in the chat", "add thread CRUD", "show markdown in the assistant", "branch conversations", "make the AI response copyable", "switch from polling to streaming", "embed Claude into my React app", "set up assistant-ui", "use Vercel AI SDK in my app", "expose chat over SSE from FastAPI", or any task on the frontend chat layer of a React + Vite + shadcn application. Read this skill BEFORE writing any new chat component, runtime hook, or SSE endpoint.

Use this skill whenever the user wants to expose application operations to an AI assistant via tools/function-calling/MCP. Triggers include "make my app operable by AI", "expose my commands to Claude/GPT", "build an MCP server for my app", "bridge command dispatch to MCP", "annotate tools for AI", "add tool approval flow", "scope tools per user", "stop the AI from calling destructive operations without confirmation", "compose multiple MCP servers", "use py2mcp", "use FastMCP", "use the MCP TypeScript SDK", or any task that wires application operations to the agent's tool surface. Look for command dispatch (any of: @command decorator, CommandSpec, acture, wrapex, py2mcp, a registry of typed handlers). If absent, prefer adding one first.

Use this skill whenever the user wants to add, refactor, manage, or audit system prompts, user-editable prompts, or "skills" (Anthropic-style) in an AI assistant. Triggers include "set up a prompt registry", "version my system prompts", "add a skill to my assistant", "make prompts editable by non-engineers", "auto-select the right skill", "write a SKILL.md", "integrate Langfuse for prompt management", "add prompt evals to CI", "manage prompt versions", "load context dynamically per request", "store prompts in git", or any task involving the SSOT for instructions an LLM consumes. Read this skill BEFORE creating any prompts/ or skills/ directory or wiring up a prompt store.

Use when inspecting or moving between the skill/agent layers with coact — auditing what extras an agent adds over its skill, estimating the token cost of fanning out a fleet before you spend it, enumerating a project's reusable AI assets, or harvesting an agent back into a skill. Triggers on "diff this skill and agent", "what does this agent add", "how much will a fleet cost", "is multi-agent worth it here", "list the skills/agents/tools in this project", "inventory", or "turn this agent back into a skill".

Use when turning an existing .claude/skills/ skill into a .claude/agents/ agent definition with coact — i.e. COMPLETE. Triggers on "complete a skill into an agent", "make an agent from this skill", "what extras would coact add", "fill in the coact frontmatter block", or wanting a dry-run/provenance preview before generating an agent. Covers complete(), plan_completion(), the additive coact frontmatter, and the no-LLM default.

Use when developing, debugging, reviewing, or extending the coact package itself (this repo) — adding emit targets or realize backends, touching COMPLETE/REALIZE, the additive coact frontmatter, the policy, or the LLM/synthesis paths. Covers the architecture, the invariants that must not be broken, the open-closed extension points, and the testing/tooling gotchas specific to this repo.

Use a real LLM with creel — wire an injected LLM client, run AI-powered graph extraction, and judge/resolve with a model. Use when the user wants schema-as-extractor extraction, to inject an LLM client (services={'llm': aix_client()} or anthropic_client()), the ('llm', {}) binding, an LLM judge for llm_rubric verification, LLM entity resolution, self-consistency policy, or to test LLM code with a fake client. Triggers on "use a real LLM with creel", "extract with an LLM / GPT / Claude", "schema-as-extractor", "inject an LLM client", "aix_client / anthropic_client", "services={'llm':...}", "the ('llm',{}) binding", "LLM as a judge", "LLM entity resolution", "self-consistency voting", "ExtractionPolicy", "AI-powered graph extraction", "fake LLM client for tests". The LLM is a swappable strategy — no provider SDK lives in creel's core.

Write creel bindings: map each grammar element to an extractor strategy. Use when you need to choose an extractor strategy for a node-type or edge-type, decide pattern vs query vs LLM extraction, write regex_node/regex_edge over prose, or table_map/sql/json_query over structured sources. Covers the binding mental model ({element_id: (strategy, params)} or a bare callable), id templates and casts, endpoint templates for edges, exclude_groups, cluster bindings (cluster_llm), registering a custom extractor strategy, and how unbound elements behave (on_missing_binding). Trigger on: map grammar elements to extractors, route prose to regex and tables to query, bind a funds/measures edge, derive node ids, id_template/id_from, register_extractor a custom strategy, cluster_llm one-pass extraction, on_missing_binding, fix a binding error.

Evaluate / verify a creel extraction — the answer to "is this extraction correct?" is a pluggable Verifier, NOT a hardcoded == . Use when you need to score extraction quality, compare an actual graph to an expected_graph, build a test corpus, or pick the right comparison per attribute. Covers the verifier-kind taxonomy (exact, normalized, numeric_tolerance, set_match, schema_constraint, semantic_similarity, composite), graph_match (decomposable partial credit with a structured mismatch report), and llm_rubric (a verifier defined by a natural-language criterion, graded by an injected LLM judge). Keywords: pluggable Verifier not equality, graph_match partial credit, numeric_tolerance, semantic similarity, natural-language LLM rubric verifier, score extraction quality.

Extract a typed property graph from messy sources with creel. Use when the user wants to turn prose, tables, JSON, or PDFs/documents into a clean, typed graph — i.e. run extract(), build a property graph, go sources->graph, use the creel facade, or get started with creel. Triggers on "extract a graph from documents", "build a knowledge/property graph with creel", "creel quickstart", "how do I call extract()", "sources to graph", "turn this prose/table/JSON into a typed graph", "ingest a folder of files/PDFs into a graph", "the one-call creel workflow". This is the ENTRY/orchestrator skill: it covers the end-to-end extract(sources, graph_spec, extractors) -> graph call, its keyword args (services, on_missing_binding, resolve, cache), and how to read the returned Graph (nodes, edges, evidence, report, validate_graph, to_canonical_json). Hand off to creel-grammar, creel-bindings, creel-ai, creel-evaluation, creel-projections for the deeper pieces.

Use when you need to define a creel grammar (a GraphSpec) — the typed schema for the graph you want to extract. Covers declaring node-types and edge-types with AttrSchema attributes; first-class edges that carry their own typed attributes (a funds edge with amount/currency); attribute options (required, range, enum, minimum/maximum, pattern, multivalued, description); reusable EnumDef value-sets and named ranges; inheritance via is_a/abstract/mixins; validating a grammar (validate_spec) and an instance graph (validate_graph); and optionally authoring in LinkML to generate JSON-Schema or Pydantic. Trigger on "define a creel grammar", "declare node-types and edge-types", "typed attributes on edges", "enums/ranges/inheritance for graph extraction", "schema for graph extraction", "LinkML authoring", or "validate a spec".

Project/transform a creel graph into everything downstream — persist, query, view, export, annotate are all projections of the one graph, never engine features. Use after you have a Graph and want to clean it up or get it OUT. Covers entity resolution (merge duplicate/same-entity nodes via Normalize/Registry/LLM/Cascade resolvers, the resolve= facade arg, or resolve_graph); the reify/unreify toggle (attributed edge ⇄ relation node when it goes n-ary, losslessly); view projections (to_node_edge_records/to_table/to_dot/to_mermaid/to_cytoscape); export adapters (to_jgf/to_graphml/to_cypher/to_turtle — GraphML, JGF, parameterized Cypher for Neo4j, RDF-star Turtle); the annotate/render contract; and trace (per-attribute grounding, TraceIndex reverse lookup, reanchor to source span). Triggers on "merge duplicate nodes", "entity resolution", "reify edge to node", "view as Mermaid/DOT/Cytoscape", "export to GraphML/Cypher/RDF/Turtle", "annotate a graph", "trace a value to its source span", "downstream rendering".

Use when building or extending creel's evaluation/verification subsystem, or when writing tests for extraction. Covers the Verifier Protocol, the verifier-kind taxonomy (exact/normalized/numeric_tolerance/set_match/graph_match/schema_constraint/semantic_similarity/llm_rubric/composite), how an LLM-instruction (G-Eval) verifier is specified and run, the test-corpus layout ({sources, expected_graph, verifier_overrides?}), per-element→per-type→per-graph score roll-up, and the rule that comparisons are pluggable verifiers — NOT hardcoded equality. Trigger on work in creel/verify/, the eval runner, the RBM corpus, or any extraction test.

Use when working anywhere inside the creel package internals — the source-to-graph extraction engine. Covers the architecture map (layers/modules), the 15 design decisions D1–D15, the key Protocol interfaces, the canonical JSON contract, the two-layer (grammar vs bindings) join, and where each choice is justified in the research. Trigger when adding/changing core modules, wiring the extract() facade, the graph model, the spec/grammar layer, or canonical JSON; or when a design question needs the authoritative answer.

Use when the user wants to curate a folder of images into a smaller training set — for character LoRAs, product LoRAs, style LoRAs, or any "pick K best from N candidates" image-selection task. Triggers on "curate these images", "pick K for LoRA training", "choose images for character LoRA", "filter my photo dump", "select diverse training images", "build a training set". Also use to drive lookbook over MCP/HTTP from an agent.

Use when developing or modifying the lookbook package — extending the curation pipeline, adding scorers/selectors/embedders, working with the manifest, swapping storage backends. Triggers on lookbook architecture questions, "how do I add a scorer", "where does the manifest live", "what's the registry pattern", or any work inside /Users/thorwhalen/Dropbox/py/proj/t/lookbook/.

Use after a lookbook curation run to interpret the result, explain why specific images were dropped or kept, query the manifest for provenance, or read cluster-coverage diagnoses. Triggers on "why was this image rejected", "what's missing from my training set", "explain the curation report", "this image was dropped — why", "what coverage gaps does my set have".

Use when the user wants to customize a lookbook curation pipeline beyond what the built-in recipes / profiles provide — overriding filter thresholds, swapping selectors, adjusting quotas, building a one-off recipe inline. Triggers on "different defaults", "custom scorer weights", "skip aesthetic scoring", "build my own pipeline", "tweak the funnel", "override the threshold", "what-if reweighting".

Use when working on lookbook's HTTP surface — adding a new endpoint, debugging a route, configuring the server (host/port/data-root), or wiring a frontend/agent client. Triggers on lookbook.http questions, "add an endpoint", "new HTTP route", "lookbook serve", "FastAPI app", or work in lookbook/http.py.

Use when adding a new subject profile to lookbook (product, scene, style, brand, font, …). A profile is a YAML file that bundles scorers, embedders, filters, a selector, and constraints into a named recipe. Triggers on "add a profile for X", "support product LoRAs", "scene profile", "style LoRA recipe", "new subject type", or work in lookbook/profiles/.

Use when adding a new selector to lookbook — wrapping an apricot submodular function, building a quota/constraint selector, integrating DPPy, or implementing a custom set-selection algorithm. Triggers on "add a selector", "wrap a submodular function", "quota-aware selection", "DPP selector", "custom set selection".

Use when adding a new per-image metric (scorer) to lookbook — wrapping a model from pyiqa/insightface/CLIP, adding technical metrics like noise or compression detection, or any "score one image" annotation producer. Triggers on "add a scorer", "wrap pyiqa model X", "add a metric for Y", "score images by Z".

Find and add references to documents. Use this skill whenever a user provides text (usually markdown) and wants references, citations, or sources added to it. This includes requests like "add references to this document", "find citations for this", "source this article", "add links to back up the claims", "reference this for publication", "find supporting evidence", or any variation where the user has written content and wants it backed by credible sources. Also use when the user asks to generate a references section, find sources for specific claims, or validate statements with citations. Trigger even for indirect requests like "make this more credible", "this needs sources", or "prepare this for submission". Works for blog posts, academic papers, grant proposals, reports, and any written content that benefits from citations.

Acquire PDFs of academic references from a document. Parses reference sections, tries direct URLs and arxiv, falls back to libgen search with smart query strategies. Logs all attempts and produces references.md + missed_references.md.

Download one or more web resources (URLs, markdown links, reference-style citations, mixed prose) and save them as Markdown (default), PDF, or original bytes. Use when the user asks to download, fetch, save, archive, or grab web pages, blog posts, docs, specs, or any URL — especially for non-academic content where the acquire-references skill doesn't apply. Triggers on "download these URLs", "save this page", "grab this article", "fetch the docs", "archive these links".

Assess how well a draft article fits a set of target journals and recommend the best venue. Compares the article's scope, tone, length, audience, and contribution type against journal profiles. Use when deciding where to submit an article.

Reformat and adapt a draft article to meet the specific requirements of a target journal (IEEE Software, CACM, IEEE TSE, etc.). Adjusts structure, word count, abstract, references, and required elements. Use when you have a polished draft and need to produce a journal-ready version.

Prepare all materials for journal submission and generate a step-by-step submission guide. Creates a submission checklist, cover letter, author metadata, and submission instructions specific to the target journal. Use when an article is ready and you need to actually submit it.

Research a technical or academic topic in depth to support article writing. Finds key papers, surveys the state of the art, identifies competing approaches, and produces a research brief with citations. Use when preparing to write an article and needing to understand the landscape and find references.

Review a draft article for academic/technical journal submission. Gives structured feedback on clarity, contribution, structure, novelty, and journal fit. Use when you have a draft article and want an expert critical review before submission.

Use when building the agent / "copilot" layer of the hearing meeting-transcription project — the thing that consumes a transcript and produces feedback (summaries, action items, decisions, follow-up research in batch; running notes, suggested questions, relevant docs, fact-checks live). Triggers on agent layer, meeting copilot, post-meeting summary, action items, decision extraction, follow-up research, live notes, suggested questions, real-time RAG, fact-check, queue consumer, asyncio.Queue of utterances, finalized segments, FrameProcessor, Pipecat, LiveKit Agents, RealTimeAgentPipeline, non-blocking transcript intercept, backpressure / decoupling the LLM from capture, feeding me/them + diarization labels into the prompt, vector store over meeting context, LlamaIndex, Qdrant, web search during a meeting, deep-research follow-up, SimpleLLMAgent, pluggable LLM backend, batch-vs-live unification. The agent defaults to Claude but stays pluggable.

Use when designing, wiring, or modifying the hearing project's pipeline architecture — the SSOT for HOW the four layers (audio capture, STT, diarization/speaker-ID, agents) compose by dependency injection. Triggers on capture source, channel split, mic vs system audio, STT engine facade, transcribe(audio)->segments, streaming transcription, diarization interface, speaker label, TranscriptSegment, agent consumer, pipeline assembly, batch vs live, recipe-style composition, swap/substitute a component, Protocol/dataclass contracts, asyncio.Queue between stages, backpressure/decoupling, progressive disclosure of the pipeline, and any "how do the hearing-* skills fit together" or "what's the shared data model" question. This is the contract every other hearing-* skill (hearing-audio-capture, hearing-stt, hearing-diarization, hearing-agents) implements against.

Use when capturing audio for meeting transcription on macOS (and the iOS limits) — getting BOTH the microphone AND system audio (the other participants coming out of your speakers) into a Python pipeline. Triggers on system audio capture, loopback, BlackHole, Aggregate Device, Multi-Output Device, Audio MIDI Setup, Core Audio taps, AudioHardwareCreateProcessTap, AudioTee, ScreenCaptureKit, SCStream, PyObjC SCK -3805, sounddevice InputStream, soundcard, PyAudio, ffmpeg avfoundation, channel slicing / mapping, "record what I hear", "capture the meeting audio", separate mic vs system channels, the "me vs them" channel trick, virtual audio driver, sample-rate / clock / drift mismatch, screen-recording permission for audio, and iOS/iPhone call/app audio recording restrictions (ReplayKit, iOS 18.1 call recording).

Use when working on speaker attribution in the "hearing" meeting-transcription library — turning a transcript into a who-spoke-when, speaker-labeled (diarized) transcript, separating remote participants, and deciding which library to run. Triggers on diarization, speaker diarization, who spoke when, speaker labels, Speaker 1/Speaker 2, speaker turns, speaker segmentation, speaker attribution, "me vs them", local vs remote speaker, channel energy / channel trick, dual VAD, speaker identification, voice fingerprinting, voice enrollment, speaker embedding, pyannote.audio, pyannote community-1, speaker-diarization-3.1, NVIDIA NeMo Sortformer/MSDD, WhisperX diarize, SpeechBrain ECAPA/x-vector, DER (diarization error rate), or feeding speaker labels into the agent prompt. Distinguish from raw transcription (that is hearing-stt) and from audio capture/channel layout (that is hearing-audio-capture).

Use when building or designing the TypeScript/React frontend for the "hearing" meeting-transcription project — the transcript viewer, the live AI-copilot overlay, the agent-feedback panel (notes, suggested questions, surfaced docs, fact-checks), or the meeting/project-context UI. Covers schema-driven UI with zodal (defineCollection, Zod v4, content-metadata bifurcation routing transcript metadata to a DB and audio/recording content to object storage), command-dispatch with acture (one command → palette + hotkeys + AI tool + MCP + tests), how the Python backend (argh CLI + HTTP layer) feeds the UI, and how live finalized utterances stream to the client. Triggers on transcript view, copilot overlay, agent panel, feedback panel, suggested questions, surfaced docs, fact-check, meeting collection, segment schema, speaker/me-vs-them labels, zodal collection, acture command, bifurcation, lk.transcription, SSE/WebSocket transcript stream, novice-friendly frontend, declarative/schema UI for hearing. Explain JS/TS deci

Use when building the streaming / live / real-time milestone of the hearing meeting-transcription library — turning the batch (record-then-transcribe) path into a low-latency loop that emits transcript segments as people speak. Triggers on streaming STT, real-time transcription, live captions, RealtimeSTT, whisper_streaming, SimulStreaming, WhisperLive, OpenAI Realtime API (gpt-realtime-whisper), Pipecat, LiveKit Agents, VAD / voice activity detection, silero-vad, webrtcvad, utterance / turn finalization, partial vs final segments, word-level timestamps for keyword triggers, feed_audio, asyncio.Queue decoupling, backpressure between capture/STT/diarization/agent stages, latency budget (VAD + STT + LLM TTFT + tool/RAG + transport), semantic turn detection, mid-sentence pause handling, and any task that adds a streaming capture/sink without changing the STT or agent interfaces.

Use when choosing, wiring, swapping, or benchmarking a speech-to-text (STT/ASR) engine behind the hearing project's transcription facade — the pluggable layer that turns audio into segments so the rest of the architecture never names a specific engine. Triggers on transcribe(audio)->segments, ASR engine selection, Whisper / whisper.cpp / faster-whisper / WhisperX / distil-whisper, NVIDIA Parakeet-TDT / Canary, Moonshine, Apple-Silicon MLX paths (lightning-whisper-mlx, mlx-audio, Lightning-SimulWhisper), cloud STT APIs (OpenAI gpt-4o-transcribe / -mini / gpt-realtime-whisper / gpt-4o-transcribe-diarize, Deepgram Nova-3, AssemblyAI Universal, Google Chirp, Azure Speech, Speechmatics, Rev), per-minute STT pricing, local-vs-cloud / on-device decision, WER tolerance, concurrency, self-hosting break-even (~100k min/month), CTranslate2 / int8 quantization, model-name versioning and the ~June-2026 OpenAI transcribe retirements, and "which engine should I use for meeting transcription". Diarization (who-spoke-when) is

Skill-building tooling for the skill package. Build AI agent skills for any Python package by analyzing its code, tests, and docs. Use this skill when the user wants to create skills for a package, analyze a codebase to suggest what skills would be useful, generate SKILL.md files from existing code/tests/docs, or when someone says things like "make skills for this package", "what skills should this project have", "create a skill from this module", or "generate skills". Also trigger when the user points at a package and asks for AI-powered interfaces to it.

Skill documentation tooling for the skill package. Document a project's AI agent skills in the README and other formats. Use this skill when the user wants to add skills documentation to a README, explain what skills a project offers, write a "Skills" section, generate skill installation instructions, or when asked things like "document the skills", "add skills to the README", "how should I present skills to users", or "write skill usage examples". Also trigger when updating a README for a package that has skills in .claude/skills/ or {pkg}/data/skills/.

Ship a Python package's AI agent skills with `pip install` — and decide WHICH skills to ship. Use when the user asks how to include skills in their package distribution, make skills available after `pip install`, ship SKILL.md files with their package, set up `{pkg}/data/skills/`, configure package-data, add a skills installer/convenience function, or says "how do users get my skills after pip install", "ship skills with my package", "distribute skills", "package skills", or "enable skills in my library". Also trigger when working on pyproject.toml and the project has skills to distribute. The overall layout decision (where real files live, the `.claude/skills/` symlink bridge, spec compliance) is owned by **skill-package-setup**; this skill covers the pip-distribution mechanics and audience classification.

Skill management tooling for the skill package. Search, list, inspect, and manage AI agent skills across agents and projects. Use this skill when the user wants to find skills, see what skills are installed, check where skills are linked, inspect a skill's contents, install or uninstall skills, compare skills across agents (Claude Code, Cursor, Copilot), or says things like "what skills do I have", "find a skill for X", "show me the skill", "where are my skills", "install this skill", "remove that skill", "list skills", or "search skills". Trigger on any skill inventory, discovery, or housekeeping request.

Canonical, agent-agnostic policy for WHERE a package's AI agent skills live and HOW they are distributed so they're discoverable by `gh skill` (and any agent host — Copilot, Cursor, Codex, Gemini, Claude) AND, where wanted, shipped with `pip install`. Use this skill to set up skills for a new package, migrate an existing repo to the canonical layout, consolidate skills scattered across `.claude/skills/` / `skills/` / `{pkg}/data/skills/`, make every SKILL.md spec-clean, or answer "where should my package's skills live". Also the authority other skills defer to (e.g. wads-skillify). Triggers on "set up skills", "where should skills live", "make skills gh-skill installable", "migrate/restructure the skills layout", "skillify this repo". Maintainer skill.

Skill maintenance tooling for the skill package. Keep AI agent skills in sync with their source package code. Use this skill when a package's code, API, or behavior has changed and its skills may be outdated, when the user says "update the skills", "skills are stale", "sync skills with code", "refresh skills", or when you notice that skill instructions reference functions/classes/patterns that no longer exist in the codebase. Also use after major refactors, API changes, or version bumps.

How to build and extend ek's AGENT/ASSISTANT evaluation instance (ek/agents/) — the second concrete instance after ek.ocr. Covers the episode object (Episode/Trajectory/Step/Cost/RunProvenance as Layer B) and the task/tool grammar (ToolSpec/TaskSpec building a GraphGrammar as Layer A, where FieldSpec.importance is the cost of a wrong tool argument), the agent metrics (TaskSuccessMetric with an injected Checker oracle, CostPerSuccessMetric implementing Cost-of-Pass E[cost]/P(success), ToolCallMetric doing BFCL AST match over a multiset of calls, TrajectoryMetric doing a Needleman-Wunsch sequence edit distance), the reliability estimators (pass@k capability vs pass^k reliability — pure functions, NOT Metrics), the LLM-as-judge seams (JudgeSignal reference-free vs JudgeMetric reference-based vs pairwise_judge, plus judge_validation reusing Krippendorff/Cohen IAA), the agent harness (run_suite with k trials, and the variance-aware agent_regression_gate using Wilson/bootstrap intervals), and the duck-typed bridges

How to add a reference-based (offline) metric to ek's score() side — the output-type to metric to library decision table (CER/WER via jiwer, ANLS*/anls_star, span-F1 via seqeval/nervaluate, TEDS/GriTS for tables, cost-weighted typed-graph GED via networkx/apted/zss, coref via metametric), the Metric Protocol __call__(pred, gold, *, grammar=None) -> float, the Report/Score decomposition object, how to wrap a library behind the Protocol and register it, and the flagship cost-weighted typed-graph must-build whose weights come from Layer-A importance. Use when implementing, wrapping, registering, choosing, or debugging an offline/reference-based metric, scoring pred vs gold, slot/entity F1, table TEDS, graph edit distance, ANLS, WER/CER, canonicalization-before-scoring, match schemes, or per-slice/per-path TP/FP/FN. For online/reference-free confidence, calibration, and decisions use the estimate_quality side instead.

Licensing guardrail for ek developers: consult BEFORE adding ANY dependency. Covers the license-tiering policy (lean permissive core vs opt-in extras), the register of 6 license landmines (GPL Levenshtein/python-Levenshtein, LGPL TorchCP, non-commercial surya-ocr, GPL Potato, proprietary Prodigy) with their permissive replacements, scanner-invisible traps where license terms live in repo files not PyPI metadata (zss BSD-3, mistralai namespace pkg, PubTabNet CDLA), the rule that nothing copyleft or non-commercial is ever a default, and the CI license gate (pip-licenses/reuse) that fails the build on GPL/AGPL/non-commercial in core..hitl. Use when picking a metric/calibration/validation/OCR/agreement/HITL backend, adding to pyproject extras, choosing an edit-distance or conformal or annotation library, hitting a "GPL"/"non-commercial"/"RAIL"/"copyleft" license, deciding which extra a dep belongs in, or whether a dep needs import-only quarantine.

How to add a reference-free quality-estimation component to ek's online estimate_quality() side: a new Signal (deterministic verifier / intrinsic-confidence / logprob / agreement-ROVER / auxiliary-QE / LLM-judge), a Calibrator (Platt/Temperature/Isotonic), or a DecisionPolicy/SelectivePolicy (ConformalGate / RiskControlGate / CostSensitiveGate). Use when building or extending the signal->calibrate->decide pipeline, wiring a Strategy Protocol into the registry, adding confidence/uncertainty/agreement signals, ROVER voting, conformal prediction (MAPIE/crepes/TorchCP), calibration (netcal/sklearn), ECE/D-ECE, risk-coverage curves, or selective prediction / accept-flag-block gating for extraction or OCR quality. Not for offline reference-based metrics (that is the score()/add-metric side).

Master orientation map for developers BUILDING ek (a framework for building Knowledge Evaluation systems that evaluate information-extraction outputs; OCR is the noisiest special case). Read this FIRST before touching any ek component. Covers the two-need 2x2 (reference-based vs reference-free, instance vs system), the two-layer data model (Layer A GraphGrammar schema/cost SSOT + Layer B AnnotatedExtraction metadata), the facades score()/evaluate()/estimate_quality(), the strategy-Protocol + registry + dependency-injection + requires_extra open-closed pattern, the module layout (base, registry, stores, metrics, canonicalize, qe/rover, validate, harness, ocr, tools, __main__), the config2py.AppData + dol persistence model, the must-builds (two-layer object, cost-weighted typed-graph distance, ROVER), and a which-report-for-which-component table. Use when deciding where code goes, what a component is named, how facades share the Layer-A object, how strategies plug in, or which ek_0N report to read.

Build and maintain ek's OCR evaluation layer on top of the ocracy package — the concrete instance of ek's reference-based score() and reference-free estimate_quality() facades applied to image->OcrResult engines. Use when working on the ek[ocr] optional extra, wiring or running the OCR offline benchmark (CER/WER/ANLS per-slice over a gold corpus), mapping OcrResult/TextBlock into AnnotatedExtraction/FieldEstimate, deciding which ocracy backend to evaluate, handling null-safe VLM engines (Claude/GPT-4o/Mistral/Mathpix returning text-only, blocks=[], confidence=None), reading per-engine capability profiles (who emits real vs model-guessed geometry, who is calibrated, who emits TEDS-able tables), gating remote engines that make paid API calls, dealing with the missing normalized table type, or anything touching ek<->ocracy. Enforces the dependency-direction rule: ek depends on ocracy via an extra; never ocracy->ek; ek core depends only on the OcrResult shape so it evaluates any image->OcrResult callable.

How to render ov's Markdown report sections and produce the single agent-consumable synopsis. Load this for the final step of study-web-app — turning analyzed findings into reports and a deduplicated synopsis.json + SYNOPSIS.md.

Study, audit, reverse-engineer, or review a web app or website end-to-end with the `ov` tool. Use whenever the user says things like "go study this app", "audit this site's UX", "reconstruct this frontend", "what's this site built with", "review our web app", "analyze the accessibility of this page", or hands you a URL to investigate. You (the host agent) are the manager: you plan the journey, call ov's deterministic tools, reason over the grounded evidence, and assemble the report. ov supplies the hands (drive) and eyes (observe + analyze); you supply the control loop.

How to run ov's deterministic architecture-reconstruction analyzers and turn the scored facts into a rebuild blueprint (reconstruct) or an architecture audit (review). Load this for the software-design lens: framework/rendering/state/routing detection, source-map recovery, API surface synthesis, dependency/CVE flags.

How to run ov's deterministic UX + accessibility analyzers, assemble a grounded evidence bundle, and add narrative UX judgment as the host. Load this for the UX lens: accessibility audit, contrast, Core Web Vitals, form friction, Nielsen heuristics, and severity-ranked usability findings with fixes.

How to drive a web app with ov's operate primitives — read affordances, choose and execute the next action, log per-step intent, and detect no-progress. Load this when you (the host) are driving a journey: crawling, pursuing a goal (signup/checkout), or replaying a script, and need the perceive→decide→act→record loop.

How to run an `ov` capture session against a web app — launch the browser, choose which probes to enable, and inspect the resulting artifact store. Load this when you are capturing a target with ov (step 1 of study-web-app), choosing probes, or looking at what was captured.

Behavioural policy for running unattended. Use when you have been launched in headless / auto-permission mode (e.g. by `ge run-roadmap` or `ge run-triage`) and there is no user available to answer questions. Converts every clarifying question into a logged decision so progress continues.

Triage and resolve a cross-repo issue backlog. Use when launched by `ge run-triage` against multiple repos — Phase A classifies and orders issues into a `TriageBacklog`; Phase B executes them one at a time (failing test → confirm red → fix → confirm green → open PR, then stop). Stops at PR; merging is a separate later phase.

Execute a roadmap of tasks end-to-end using GitHub as durable memory. Use when launched against a roadmap issue (typically by `ge run-roadmap`) — take the next todo task, do it, verify it, mark it done, log decisions, and repeat. Survives context exhaustion because each work unit rehydrates from GitHub.

Use when asked to work on a GitHub issue, PR, or discussion. Resolves flexible input (URL, folder, number), confirms with the user, prepares context, and guides the workflow.

Use when you need to prepare, read, or refresh full context for a GitHub issue, PR, or discussion — fetching data, downloading media, and writing structured context documents.

Use when you need to check whether a GitHub issue or PR is still relevant, stale, or already resolved — without preparing full context or downloading media.

How to add a reference-based (offline) metric to ke's score() side — the output-type to metric to library decision table (CER/WER via jiwer, ANLS*/anls_star, span-F1 via seqeval/nervaluate, TEDS/GriTS for tables, cost-weighted typed-graph GED via networkx/apted/zss, coref via metametric), the Metric Protocol __call__(pred, gold, *, grammar=None) -> float, the Report/Score decomposition object, how to wrap a library behind the Protocol and register it, and the flagship cost-weighted typed-graph must-build whose weights come from Layer-A importance. Use when implementing, wrapping, registering, choosing, or debugging an offline/reference-based metric, scoring pred vs gold, slot/entity F1, table TEDS, graph edit distance, ANLS, WER/CER, canonicalization-before-scoring, match schemes, or per-slice/per-path TP/FP/FN. For online/reference-free confidence, calibration, and decisions use the estimate_quality side instead.

How to add a reference-free quality-estimation component to ke's online estimate_quality() side: a new Signal (deterministic verifier / intrinsic-confidence / logprob / agreement-ROVER / auxiliary-QE / LLM-judge), a Calibrator (Platt/Temperature/Isotonic), or a DecisionPolicy/SelectivePolicy (ConformalGate / RiskControlGate / CostSensitiveGate). Use when building or extending the signal->calibrate->decide pipeline, wiring a Strategy Protocol into the registry, adding confidence/uncertainty/agreement signals, ROVER voting, conformal prediction (MAPIE/crepes/TorchCP), calibration (netcal/sklearn), ECE/D-ECE, risk-coverage curves, or selective prediction / accept-flag-block gating for extraction or OCR quality. Not for offline reference-based metrics (that is the score()/add-metric side).

Master orientation map for developers BUILDING ke (a framework for building Knowledge Evaluation systems that evaluate information-extraction outputs; OCR is the noisiest special case). Read this FIRST before touching any ke component. Covers the two-need 2x2 (reference-based vs reference-free, instance vs system), the two-layer data model (Layer A GraphGrammar schema/cost SSOT + Layer B AnnotatedExtraction metadata), the facades score()/evaluate()/estimate_quality(), the strategy-Protocol + registry + dependency-injection + requires_extra open-closed pattern, the module layout (base, registry, stores, metrics, canonicalize, qe/rover, validate, harness, ocr, tools, __main__), the config2py.AppData + dol persistence model, the must-builds (two-layer object, cost-weighted typed-graph distance, ROVER), and a which-report-for-which-component table. Use when deciding where code goes, what a component is named, how facades share the Layer-A object, how strategies plug in, or which ke_0N report to read.

Licensing guardrail for ke developers: consult BEFORE adding ANY dependency. Covers the license-tiering policy (lean permissive core vs opt-in extras), the register of 6 license landmines (GPL Levenshtein/python-Levenshtein, LGPL TorchCP, non-commercial surya-ocr, GPL Potato, proprietary Prodigy) with their permissive replacements, scanner-invisible traps where license terms live in repo files not PyPI metadata (zss BSD-3, mistralai namespace pkg, PubTabNet CDLA), the rule that nothing copyleft or non-commercial is ever a default, and the CI license gate (pip-licenses/reuse) that fails the build on GPL/AGPL/non-commercial in core..hitl. Use when picking a metric/calibration/validation/OCR/agreement/HITL backend, adding to pyproject extras, choosing an edit-distance or conformal or annotation library, hitting a "GPL"/"non-commercial"/"RAIL"/"copyleft" license, deciding which extra a dep belongs in, or whether a dep needs import-only quarantine.

Build and maintain ke's OCR evaluation layer on top of the ocracy package — the concrete instance of ke's reference-based score() and reference-free estimate_quality() facades applied to image->OcrResult engines. Use when working on the ke[ocr] optional extra, wiring or running the OCR offline benchmark (CER/WER/ANLS per-slice over a gold corpus), mapping OcrResult/TextBlock into AnnotatedExtraction/FieldEstimate, deciding which ocracy backend to evaluate, handling null-safe VLM engines (Claude/GPT-4o/Mistral/Mathpix returning text-only, blocks=[], confidence=None), reading per-engine capability profiles (who emits real vs model-guessed geometry, who is calibrated, who emits TEDS-able tables), gating remote engines that make paid API calls, dealing with the missing normalized table type, or anything touching ke<->ocracy. Enforces the dependency-direction rule: ke depends on ocracy via an extra; never ocracy->ke; ke core depends only on the OcrResult shape so it evaluates any image->OcrResult callable.

Use for AUDIO editing with the `mixing` package: trim/crop a clip, fade in or fade out, join/concatenate clips (with optional crossfade), mix music under a voice or overlay one track on another, normalize loudness, convert to mono, resample, peek at raw samples, align two recordings of the same performance, or split a long recording (concert, DJ mix, radio show, podcast) into segments. Trigger on phrasings like "trim audio", "fade out the end", "join these audio clips", "add background music under the narration", "make it louder / normalize", "convert to mono", "align two recordings", "find the offset between these takes", "split a concert into songs", "separate speech from music". For video audio (replace/normalize a video's track) use mixing-video; this skill is audio-file → audio-file.

Use when re-voicing or translating audio/video with text-to-speech via the `mixing` package: turning text into spoken audio, synthesizing narration, translating subtitles, or producing a foreign-language dub of a video from its SRT. Trigger on requests like "dub this video", "re-voice this in another language", "text to speech / TTS this", "synthesize narration", "translate these subtitles", "make a French version of this clip", or "read this script in voice X". For transcribing speech → SRT in the first place, see mixing-transcript; for the `output` protocol and ffmpeg, see the mixing router skill.

Use the `mixing` Python package for video and audio editing tasks: slicing audio/video, fades, cropping, looping, changing speed, replacing or mixing a video's audio, normalizing levels, Ken Burns pan/zoom, concatenating clips, thumbnails, burning in subtitles, speech-to-text + filler removal, chapter detection, and TTS dubbing/translation. Trigger whenever someone wants to edit, transform, transcribe, dub, or assemble audio/video files with Python — e.g. "trim this clip", "add music to a video", "remove the ums", "make a thumbnail", "turn this audio into a podcast", "dub this in French". Start here to pick the right tool and the right sub-skill (mixing-audio / mixing-video / mixing-transcript / mixing-dubbing).

Transcribe speech to text, remove filler words (ums/uhs), build SRT subtitles or clean prose, and detect chapter markers — using the `mixing` package's `transcript` and `chapters` modules (ElevenLabs Scribe under the hood). Trigger whenever someone wants to "transcribe this audio/video", "get an SRT/captions", "remove the ums and uhs", "clean up the fillers", "make chapters/timestamps", "detect topic shifts", or "get word-level timestamps" from a media file. For re-voicing/translation use mixing-dubbing; for plain audio/video edits use mixing-audio / mixing-video. Read the `mixing` router skill first for the `output` egress protocol and ffmpeg/key setup.

Use for VIDEO editing with the `mixing` Python package: trim/crop a clip, loop it, speed it up or down, replace or mix in a new audio track, normalize audio levels, pan/zoom a still image (Ken Burns), build a film from panels, concatenate clips, grab/extract frames, make a YouTube-style thumbnail, burn in subtitles, or resize for shorts/tiktok/square. Trigger on phrasings like "trim this video", "loop the intro", "speed up / slow down this clip", "replace the audio with this music", "add a pan-zoom over this photo", "make a thumbnail", "burn the subtitles in", "resize for YouTube Shorts", or "stitch these clips together". For audio-only work use mixing-audio; for transcription / filler removal use mixing-transcript; for TTS re-voicing use mixing-dubbing.

Turn songs (.wav/.mp3) into publishable YouTube music videos with the `yb` package: render a still-cover, Ken Burns, or audio-reactive video, derive the thumbnail, loudness-normalize a whole album, verify the result against what YouTube actually accepts, and publish. Use when the user wants to publish a song or album to YouTube, make a music video or visualizer from an audio file, add cover art to a track for upload, or check a rendered music video before shipping it.

Prepare and publish a video to YouTube with the `yb` package: transcribe, derive title/description/keywords, detect chapters, render a thumbnail, upload (with captions), and edit already-published videos (metadata, subtitles, chapters, thumbnail). Use when the user wants to publish/upload a video to YouTube, prepare YouTube metadata/chapters/subtitles, replace or add subtitles on an existing video, or update a published video's title/ description/tags/thumbnail. For first-time OAuth setup, use yb-setup first.

Prepare a podcast episode with the `yb` package: show notes, chapter markers (ID3 chapters embedded in the MP3 + a Podlove PSC sidecar), an optional cover-over-audio video (static or Ken Burns), and an RSS episode item. Use when the user wants to turn audio (or a video's audio) into a publishable podcast episode, generate show notes / podcast chapters, embed chapters in an MP3, make a cover video for an audio episode, or build a podcast RSS feed.

Set up YouTube publishing for the `yb` package: install the right extras, create a Google OAuth client (gcloud optional — the console path works), wire up the credentials, run the one-time consent, and test the connection. Use when the user wants to publish/upload to YouTube for the first time, hits an OAuth or "client secrets" error, sees "accessNotConfigured", needs to make or replace a YouTube OAuth client, or asks how to authenticate yb with YouTube.