メニュー
Evaluation provider abstraction for dataset experiments (Langfuse today). Use when running an experiment, adding a new eval backend, or wiring evals into CI.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Router for any non-trivial echo-sdk task. Maps user intent to the right module skill and diagram. Invoke FIRST for any change to src/echo/ before reaching for a specific module skill.
Recipe for adding a new provider (LLM, transcriber, eval, prompt). Use whenever adding Cohere, Mistral, Deepgram, Phoenix, Promptlayer, etc. — anything that plugs into an existing get_*() factory.
End-to-end recipe for defining a Skill bundle and wiring it into a GenericAgent with the right activation mode and tool registry. Use when extending agent capability via swappable skills rather than baking tools in.
Building agents with BaseAgent / GenericAgent, including skill registration, activation modes (llm vs manual), and system-prompt composition. Use when editing src/echo/agents/ or building a new agent subclass.
Audio transcription provider abstraction (Gemini, EkaCare). Use when transcribing audio, adding a new transcriber, or shaping AudioInput.
Postgres support — asyncpg client, schema-aware binder, PgQueryTool surface. Use when querying postgres from a tool, modifying the binder, or adding a new DB engine.
| name | echo-sdk-evals |
| description | Evaluation provider abstraction for dataset experiments (Langfuse today). Use when running an experiment, adding a new eval backend, or wiring evals into CI. |
BaseEvalProvider in evals/base.py — abstract; declares run_experiment(name, dataset_name, run_func).LangfuseProvider in evals/langfuse_provider.py — concrete implementation.get_eval_provider() in evals/factory.py — singleton factory.run_func signature: async def run_func(item) -> result — receives a dataset item, returns the model output (and optionally metadata). Provider records the result against the experiment.langfuse is an extra; guard imports.run_experiment.[[echo-sdk-prompts]]) — label="production" drifts under your feet.→ [[echo-sdk-adding-a-provider]]. Implement BaseEvalProvider, register in factory.py.
run_func invocations — items may run concurrently; keep run_func pure.run_func — wrap in asyncio.to_thread or skip.examples/prompt_eval_usage.py[[echo-sdk-prompts]], [[python-optional-deps]]