Menü
Evaluation provider abstraction for dataset experiments (Langfuse today). Use when running an experiment, adding a new eval backend, or wiring evals into CI.
Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.
Basierend auf der SOC-Berufsklassifikation
Router for any non-trivial echo-sdk task. Maps user intent to the right module skill and diagram. Invoke FIRST for any change to src/echo/ before reaching for a specific module skill.
Recipe for adding a new provider (LLM, transcriber, eval, prompt). Use whenever adding Cohere, Mistral, Deepgram, Phoenix, Promptlayer, etc. — anything that plugs into an existing get_*() factory.
End-to-end recipe for defining a Skill bundle and wiring it into a GenericAgent with the right activation mode and tool registry. Use when extending agent capability via swappable skills rather than baking tools in.
Building agents with BaseAgent / GenericAgent, including skill registration, activation modes (llm vs manual), and system-prompt composition. Use when editing src/echo/agents/ or building a new agent subclass.
Audio transcription provider abstraction (Gemini, EkaCare). Use when transcribing audio, adding a new transcriber, or shaping AudioInput.
Postgres support — asyncpg client, schema-aware binder, PgQueryTool surface. Use when querying postgres from a tool, modifying the binder, or adding a new DB engine.
| name | echo-sdk-evals |
| description | Evaluation provider abstraction for dataset experiments (Langfuse today). Use when running an experiment, adding a new eval backend, or wiring evals into CI. |
BaseEvalProvider in evals/base.py — abstract; declares run_experiment(name, dataset_name, run_func).LangfuseProvider in evals/langfuse_provider.py — concrete implementation.get_eval_provider() in evals/factory.py — singleton factory.run_func signature: async def run_func(item) -> result — receives a dataset item, returns the model output (and optionally metadata). Provider records the result against the experiment.langfuse is an extra; guard imports.run_experiment.[[echo-sdk-prompts]]) — label="production" drifts under your feet.→ [[echo-sdk-adding-a-provider]]. Implement BaseEvalProvider, register in factory.py.
run_func invocations — items may run concurrently; keep run_func pure.run_func — wrap in asyncio.to_thread or skip.examples/prompt_eval_usage.py[[echo-sdk-prompts]], [[python-optional-deps]]