SkillAlchemy

SkillAlchemy — 一念落地，万象成形。输入任意想法或蒸馏目标，输出可安装的 SKILL.md。 内部编排 Lens（看清问题）和 LEAP（执行蒸馏/融合）。用户唯一入口。 Use when 用户说「蒸馏」「生成 skill」「融合」「我想做 X 但不知道从哪下手」。

LEAP — 落地执行引擎。内含两条管线：A 分支蒸馏（从 raw data 提取 skill）、 B 分支融合（多 skill 编织为一个）。被 SkillAlchemy 编排器调用。 Use when 编排器判断需要蒸馏或融合时。

Screens biomedical / life-science papers for signs of data fabrication, image manipulation, and statistical anomalies, using the detection techniques distilled from the field's canonical exposure platforms (PubPeer, Data Colada, Science Integrity Digest, For Better Science) and tools (ImageTwin/Proofig, statcheck, GRIM/GRIMMER, Problematic Paper Screener, Seek & Blastn). Use when asked to check a paper/figure for image duplication, blot splicing, impossible statistics, paper-mill or tortured-phrase signals, research integrity, or "is this data faked"; or when a user shares a figure, Western blot, supplementary dataset, or DOI and asks whether it looks manipulated. Reports observable anomalies as questions for clarification — it never accuses anyone of fraud.

Lens — 给你的问题加一层认知镜片。输入任意任务描述，输出增强版 description， 发现「你不知道自己不知道」的隐性维度、前置条件和认知路线。 Use when 用户说「帮我想想」「分析一下」「生成 skill」「蒸馏」「融合」 或输入看起来太简单需要展开。

Cross-framework enhancement overlay for choosing a multi-agent topology BEFORE writing any agent. A binary-question rubric — is single-agent + tools enough? do agents need to know about each other? does the output need one voice? — maps the answer to single-agent / supervisor / swarm / sequential / hierarchical. Activates when a coder agent is tempted to "split the work into roles" or reaches for a multi-agent framework. Encodes the *selection rubric* that the per-framework skills assume but never surface. Search keywords: when to use multi-agent, single vs multi agent, do I need multiple agents, supervisor vs swarm, multi-agent vs single agent, agent team design.

SOP for terminal-based, git-native AI pair programming with Aider (git work-tree + tree-sitter repo-map + edit-format + human-in-loop REPL). Use when editing code in an existing git repo via an LLM, when you need to converge a change to 2-5 files, pick an edit format that fits the model, run architect+editor mode, or wire an auto-test loop.

Universal discipline for any LM-driven loop — agent retries, plan-act-observe, multi-agent handoffs, optimiser passes, test-fix cycles. Encodes the one rule every framework documents quietly and every team relearns expensively: the LM in the loop is NEVER a reliable terminator. Termination must be provided by an explicit counter + exit predicate + stagnation signal + escalation path that live OUTSIDE the LM's control. This is a tool- level, framework-agnostic skill. It maps onto LangGraph (recursion_limit + state counter + interrupt), CrewAI (max_iter + max_rpm + human_input), Claude / OpenAI SDKs (max_iterations + tool_use_budget), DSPy (declared evaluation budget), Aider (REPL + explicit retry cap), and AutoGen (max_consecutive_auto_reply). Search keywords: infinite loop, recursion limit, recursion_limit, GraphRecursionError, max iterations, max_iter, agent stuck, agent won't stop, runaway agent, ReAct loop not terminating, agent repeating itself.

Decision rubric for when an LM agent should write-and-run code (Program-of-Thought / code interpreter) versus reason in natural language: classify each step as deterministic- computable (emit + execute code, feed the result back) vs judgment (stay in prose). Use when designing or debugging an agent step that does arithmetic/parsing/data transforms, when prose reasoning hallucinates a computation (under-coding), or when a sandbox round- trip is wasted on a judgment task (over-coding). Search keywords: code interpreter, agent does math wrong, calculator hallucination, when to run code vs reason, program of thought, PoT, tool vs reasoning.

Coder-agent working-file budget discipline: keep the editable working set (files you /add into writable context) under ~25k tokens, separate "read" from "edit", delegate breadth to a read-only repo-map, and drop files once edited. Use when an LLM coder-agent edits multiple files, when the working set must stay focused, or when the model starts editing the wrong file / missing targets because too much context dilutes attention. Search keywords: context window full, agent edits wrong file, too much context, /add /drop files, working file budget, context dilution, lost in the middle.

SOP for writing, loading, and evolving a project-level convention file (CONVENTIONS.md / CLAUDE.md / .cursor/rules / .clinerules / AGENTS.md) so that a coder-agent reliably respects your codebase's style choices every session. Tool-agnostic; covers the four load mechanics (read-only attachment, ancestor-walk auto-load, glob-scoped rules, agent backstory) and the conflict resolution between pinned conventions and the existing code.

Split a multi-call LM workflow by cognitive load, not by accuracy: let one strong model make the few reasoning decisions and a cheap model do the many mechanical executions (Aider architect+editor, DSPy optimizer-LM vs task-LM, vLLM speculative draft+target, LangGraph supervisor+worker are the same shape). Use when designing or cost-optimizing a pipeline that calls an LM many times, when deciding which steps need a strong reasoner vs a cheap executor, or when adding an escalation valve for when the cheap tier degrades. Search keywords: reduce LLM cost, cheaper model, lower token cost, model cascade, route to cheap model, strong model plus cheap model, LLM cost optimization.

SOP for building multi-agent systems with CrewAI — role-based collaboration, sequential/hierarchical processes, Flows, memory, delegation. Use when modeling agent teams with clear roles and task pipelines.

SOP for building LLM applications on Dify — visual workflow + chatflow + agent + RAG knowledge base + plugin marketplace + observability, self-hostable. Use when shipping LLM apps fast with a "no-code to pro-code" gradient, especially when non-engineers need to co-author the flow.

Build and govern a 50-200 example domain-specific held-out benchmark sampled from real traffic. Distinct from public benchmarks (MMLU/HumanEval/GSM8K via lm-evaluation-harness) which measure GENERAL capability. Only a held-out domain set predicts whether THIS system works on YOUR data. Collect real examples, label, hold out (never train/prompt on it), size 50-200, version it, refresh on drift.

Operating SOP for DSPy (Stanford NLP) — the declarative framework for "programming, not prompting" language models. Activate when the user says any of: "use DSPy", "compile a prompt", "optimize prompts/programs", "MIPRO/MIPROv2", "BootstrapFewShot", "GEPA", "Signatures + Modules", "teleprompter", "auto-tune prompts for a different LM", or whenever a brittle hand-crafted prompt pipeline needs to be turned into a *compiled*, measurable, swappable program. Do NOT activate for one-shot prompt tweaks, no-metric exploratory work, or pipelines where prompts must remain human-authored verbatim — use raw prompting or LangChain templates instead.

Neutral, framework-agnostic decision tree for project kickoff: "which agent / RAG / LLM framework should I reach for?" Synthesizes the ecosystem sections of 7 landmark-project SOPs (LangGraph, LlamaIndex, DSPy, CrewAI, vLLM, Aider, Dify) into one layered rubric. Core stance: frameworks are LAYERS, not competitors — a real project usually combines DSPy (compile) + LlamaIndex (retrieve) + LangGraph (orchestrate) + vLLM (serve), and you choose ONE per layer, not one to rule all. Use when starting any LLM/agent/RAG project, or whenever the "which framework?" question is asked. Deliberately neutral — unlike vendor docs and the LangChain-biased `framework-selection` on skill.sh, this skill has no horse in the race.

Decision protocol for wrapping a REST / GraphQL / RPC API as a tool an LLM agent can call. The load-bearing premise: the *tool surface* is an LM-friendly subset of the *API surface* — one tool per user intent, not one per endpoint. Activates when a coder agent must expose an external HTTP API to a model (function calling, tool_use, MCP, LangChain `@tool`, CrewAI `BaseTool`). Encodes the *what to surface, how to name, how to shape, how to fail* — not any single framework's API. ~80% of agent tools in production are HTTP wrappers; this is the SOP for getting them right.

Enhancement-overlay SOP for adding sparse (BM25 / keyword) retrieval alongside dense (embedding) retrieval. Activate when a calling agent is building, reviewing, or debugging a retrieval pipeline whose corpus contains exact-match tokens — identifiers, error codes, SKUs, API/function names, proper nouns, citations, rare jargon — that pure dense embedding silently misses. Encodes the single decision rule (**hybrid is traffic-driven, not theoretical: add sparse only when the query share that depends on exact tokens is non-trivial**), the wiring of QueryFusionRetriever-style fusion (RRF vs alpha-weighted), and per-query-type alpha tuning. Frame the work as recovering lexical identity that dense pooling destroys, not as "add keyword search for completeness". Cross-links [[llamaindex]].

Re-ingest-correctness SOP for production RAG. Activate when a calling agent builds, reviews, or debugs an ingestion pipeline that runs more than once over a changing corpus — scheduled re-index, incremental updates, CI re-ingest, or a "retrieval has duplicates / shows deleted docs" bug. Encodes the rule — **ingestion must be idempotent: a document's content hash decides insert/update/skip, so re-running over unchanged docs is a no-op** — plus the docstore + doc-hash upsert machinery (LlamaIndex `IngestionPipeline` + `DocstoreStrategy`), the delete-propagation problem, and cross-framework equivalents (LangChain `index()` + `RecordManager`, manual hash table). ENHANCE overlay over [[llamaindex]]: the IngestionPipeline exists in the base skill but the re-ingest-correctness contract is not surfaced.

Decision protocol for building, debugging, and operating LangGraph-based agent systems. Activates when a coder agent is asked to design a stateful LLM workflow, add human-in-the-loop, choose a multi-agent pattern (supervisor / swarm / hierarchical), pick a checkpoint backend, or migrate a fragile chain into a durable graph. LangGraph is positioned by its maintainers as a "low-level orchestration framework for building, managing, and deploying long-running, stateful agents" — this skill encodes the *when* and *why*, not the API.

Operating-system distillation of LlamaIndex — the leading RAG / document-agent framework. Activate when the calling agent must build, debug, harden, or evaluate a Retrieval-Augmented Generation pipeline over unstructured/private data, decide between RAG primitives (Index types, retrievers, query engines, routers, agents), or pick LlamaIndex vs LangChain / Haystack / raw vector store for a coding task. Encodes the 5-layer mental model (Documents → Nodes → Indices → Retrievers → Query Engines / Response Synthesizers), the canonical RAG bootstrap SOP from baseline `VectorStoreIndex` through hybrid + reranker + eval-loop hardening, the official 13-failure-mode checklist, and 5 dilemma cases distilled from docs, GitHub issues, and 2025 production post-mortems.

Enhancement overlay — version the WHOLE deployable LLM-app artifact as one bundle: prompts + compiled programs + model snapshot pins + retrieval config + eval-set version, versioned together so a deploy is reproducible and rollback is atomic. Activate when preparing to deploy an LLM app, when asking "what exactly is running in prod right now?", when a deploy must be reproducible months later, or when an incident needs a clean rollback. The core reframe: an LLM app artifact is NOT an ML model — it is a manifest over many independently-mutable parts, not one weights file. Do NOT activate for one-off prompt edits with no deploy, for a single-component demo, or where a vendor owns the whole prompt lifecycle. For versioning ONE compiled prompt use [[agentsop-per-model-artifacts]]; for the CI comparison mechanism use [[agentsop-regression-gate]]. Search keywords: prompt versioning, reproducible deploy, what is running in prod, rollback LLM app, model pinning, prompt registry, version prompts and config.

Cross-engine decision rubric for self-hosting or recommending an LLM serving stack. Picks among vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, and MLX as a function of (hardware × workload × constraint), not "which is fastest". Activates whenever a coder-agent must choose, defend, or migrate a serving runtime.

Decision protocol for making side-effectful agent tools idempotent — so when an LLM tool call is retried (timeout, framework resume, user re-run, model duplicate emit), the second call is a no-op instead of a double-send. The load-bearing premise: the LM cannot promise it'll call exactly once; the tool must promise the second call is safe. Framework-agnostic — applies to LangGraph node bodies that re-run on resume, MCP tools, OpenAI tool-calling retries, CrewAI delegated tool invocations, and direct HTTP wrappers. Search keywords: duplicate email sent, charged twice, exactly-once, idempotency key, tool called twice, retry side effect, double-send, at-least-once delivery.

Decision protocol for the map-reduce / dynamic fan-out pattern in LM pipelines — "given list L, run f(item) for each item in parallel, then combine". Activates when the coder agent is about to process N items with N LM calls (per-doc summarize, per-query retrieve, per-candidate rank, parallel tool fan-out). Encodes the *when*, *how many at once*, *what to do when one fails*, and *how to reduce* — not the API of any single framework. Cross-framework: LangGraph `Send`, CrewAI parallel tasks / Flow, `asyncio.gather`, `ThreadPoolExecutor`, LlamaIndex batch retrieval.

Decomposed, multi-criteria metric design for LLM pipelines. The metric IS the model — change the metric and the optimizer changes behavior. Decompose by default; bool during compile, float during eval; calibrate against human; mitigate judge bias. Search keywords: LLM-as-judge, llm as judge, eval metric, evaluation score, scoring function, rubric, RAGAS, G-Eval, judge bias, verbosity bias, how to evaluate LLM output.

ENHANCE overlay on [[dspy]] — the upfront rubric for choosing a reasoning SHAPE (Predict / ChainOfThought / ReAct / ProgramOfThought) BEFORE you write a prompt or pick an optimizer. The local `dspy` skill lists the modules but never surfaces the *selection criterion*: reasoning shape is chosen by task structure, not by reflexively defaulting to CoT. Activate every time a new LM-calling node/step is added to a pipeline. Do NOT activate for one-shot prompts, optimizer/teleprompter choice (that is the dspy SOP's job), or non-LM control flow. Search keywords: chain of thought vs ReAct, when to use CoT, reasoning type, ReAct vs CoT vs PoT, which dspy module, predict vs chain of thought.

Security-first SOP for multi-tenant RAG systems. Activate when a calling agent is building, reviewing, or debugging any retrieval pipeline whose vector store is shared across more than one user, organisation, workspace, customer, or permission scope. Encodes the single non-negotiable rule — **filter at the vector store query, never after retrieval / never after rerank** — together with the per-vendor query-time filter APIs (Pinecone namespaces + `$eq`/`$in`, Weaviate `multiTenancyConfig` + tenant handle, Qdrant `is_tenant` payload index + `Filter.must`, Chroma `where`, pgvector RLS), and the cross-framework adapters (LlamaIndex `MetadataFilters`, LangChain `filter=` dict). Frame the work as preventing CVE-2024-41892 / EchoLeak / Slack-AI-class cross-tenant leakage, not as "adding a filter for relevance".

Enhancement-overlay (C5) for RAG over long documents — the chunk-paradox resolution. Activate when a single fixed chunk size cannot satisfy both retrieval precision (small chunks) and generation context (large chunks): small chunks lose surrounding context, large chunks dilute embedding relevance into "topic averages". Encodes the core flip — decouple the embed-unit from the return-unit: embed small for retrieval precision, return large for synthesis context — and the SOP to pick a base chunk size, choose a horizontal (sentence-window) vs vertical (auto-merging / parent-child) expansion strategy, and measure the lift. Cross-links [[llamaindex]] for the full RAG SOP; this overlay supplies the missing "chunk-paradox-resolution" recipe that the framework docs (HierarchicalNodeParser, SentenceWindow) only describe in fragments. Medium-frequency for any RAG over long prose, manuals, filings, or codebases. Search keywords: chunk size, chunking strategy, parent document retriever, sentence window, small-to-big retri

Enhancement-overlay skill — the DECISION + WIRING layer for LM observability that the single-backend skills [[langsmith]], [[phoenix]], [[mlflow]] do NOT cover. Each of those installs one backend; none of them help you DECIDE which backend fits your stack/scale/budget, nor give you a one-line autolog that turns it on fast. Use when starting any LM project, before the first deploy, or the moment someone asks "why did it do that?" and there are no traces to answer with. The skill picks a backend by stack (LangSmith for LangChain/LangGraph; Phoenix for OSS/local OpenTelemetry; MLflow for ML-shops already on MLflow; Langfuse for self-host), wires one-line autolog, verifies traces land, and adds eval hooks — instrumenting BEFORE you need it. Cross-links the first-debug-move skill [[agentsop-prompt-history-inspect]]. Do NOT activate to re-teach a backend you already chose (defer to its own skill), or for non-LM ML experiment tracking with no LLM calls (that is plain MLflow).

Pick an LM output format per (task x consumer x model) rather than by reflex: different formats carry different cognitive load (e.g. code-in-JSON makes the same model write worse code than plain-text+diff, while asking for prose when you need a typed object fails the other way). Use when designing or debugging an LM's output schema, choosing between plain text / diff / JSON / tool-call / grammar-constrained output, or when a model's quality drops after wrapping its output in a structured format.

Lifecycle SOP for **per-model prompt artifacts** — the compiled prompts, instructions, few-shot demos, edit-format pins, and embedding-bound indices that change behavior when the underlying LM, dataset, or framework version changes. Activate when adopting compiled prompts (DSPy, GEPA, BootstrapFewShot output), when supporting multiple LMs in production, when a provider deprecates a model snapshot, or when a framework deprecates a config surface (LlamaIndex `ServiceContext` → `Settings`, Aider edit-format defaults). Do NOT activate for one-off raw prompt edits or for truly model-agnostic system prompts that have been swap-tested. Search keywords: prompt portability, model swap, recompile prompt, model deprecation, prompt per model, prompt breaks on new model, version compiled prompts.

The compile-readiness gate for prompt auto-optimization. Decide whether you have earned the right to run an optimizer (DSPy MIPROv2 / GEPA / BootstrapFewShot) before spending compute. Two preconditions only — a real metric, and enough examples for the optimizer you picked. Garbage metric in, garbage prompt out. Pick the optimizer by data scale; GEPA inverts the scale assumption (~10 examples + textual feedback).

Tool skill — the *first move* in any LM-debugging session: dump the actual rendered prompt the framework sent to the model, before changing anything else. Activate when an LM call produced an unexpected output (wrong answer, schema violation, refusal, truncation, cost spike, latency spike, infinite loop, "model got dumber after upgrade"). The skill enforces a 30-second inspect step BEFORE any prompt edit, model swap, retry, or temperature tweak. Cross-framework cheat sheet: DSPy `inspect_history`, LangGraph `get_state_history`, CrewAI `step_callback`, LangChain `set_debug`/`set_verbose`, Aider `/diff`+`--verbose`, raw OpenAI/Anthropic via `OPENAI_LOG=debug`/`ANTHROPIC_LOG=debug` or HTTPX event hooks. Do NOT activate for first-time prompt authoring, exploratory prompt design, or non-LM bugs.

Enhancement-overlay SOP for query-type routing — sending a query to the right index / tool / engine *before* retrieving, not after. Activate when a calling agent owns a retrieval or answering surface that fronts more than one handler (a summary index, a vector index, a text-to-SQL engine, a tool) and the inbound queries differ in kind: "summarize this doc" vs "find the clause about X" vs "how many orders shipped in Q3". Encodes the one non- negotiable insight — **one retriever cannot serve all query types; route first, retrieve second** — plus the three router families (LLM/selector, embedding/semantic, keyword/rule), confidence-threshold + fallback discipline, and the cross-framework mapping (LlamaIndex `RouterQueryEngine` / `SelectorPromptTemplate`, Dify Question Classifier node, LangGraph conditional edges). This is an ENHANCE overlay over the per-framework skills — cross-link `[[llamaindex]]`, `[[agentsop-dify]]`, `[[agentsop-langgraph]]` for the deep API. Search keywords: route query, semantic router, qu

Build a held-out eval set, run it on every prompt/model change, and block regressions in CI. An LM change is a code change — gate it with a test suite (eval set + metric + threshold). Cross-framework SOP not surfaced by any single base skill.

Symbol-level code context for LLM coder-agents: tree-sitter extracts symbols, PageRank ranks them over the cross-file reference graph, and the top class/function signatures are fed to the LLM as a token-budgeted read-only map (not RAG, no vector index, human-auditable). Use when an agent must locate the right files in a large/multi-file repo, when builds/refreshes/scopes a repo-map, or when "model edits the wrong file" needs fixing.

A 5-minute gate the coder runs at project kickoff (and again whenever the repo shape changes). Classifies the workspace into Greenfield / Brownfield-large / Mid-size-familiar / Library-SDK, then maps the state to an agent strategy (autonomy, context primitive, tool choice). Use BEFORE picking Cursor vs Claude Code vs Aider, BEFORE turning on repo-map, BEFORE writing the first prompt. Skip only if the same repo was gated within the last day and nothing changed. Search keywords: greenfield vs brownfield, new project vs existing codebase, project setup strategy, legacy codebase agent, where to start a coding agent.

Enhancement-overlay SOP for the reranker stage of a RAG pipeline — the "retrieve wide, rerank narrow" discipline. Activate when a calling agent owns a retrieval pipeline whose answers have plateaued: top-k contains the right document but it is buried below noise, or the context window is under pressure from too many marginal chunks. Encodes the one non- negotiable insight — a cheap bi-encoder retrieves *wide* for recall, then a more expensive cross-encoder (which reads query + document *together*) reranks *narrow* for precision; keep top-N=20-50, rerank to top-k=3-5. Covers when to add a reranker (and when not to), N-vs-k tuning, model choice (Cohere/Voyage API vs bge-reranker local vs SentenceTransformer cross-encoder), latency/cost budgeting, and the cross-framework mapping (LlamaIndex node postprocessors, LangChain ContextualCompressionRetriever, Cohere/Voyage rerank APIs, local cross-encoders). This is an ENHANCE overlay over the per- framework skills — cross-link `[[llamaindex]]` and `[[agentsop-hybrid-r

Project-kickoff rubric for the self-host vs managed-cloud decision — when is running your own inference engine / LLM platform worth the ops cost vs paying per-token for a managed API? Decide on two axes — VOLUME (a cost-crossover slider) and COMPLIANCE (a hard gate). Use at kickoff when choosing where to run inference, or when cost / data-residency pressure forces a re-evaluation.