Open-source AI observability platform for tracing, evaluating, and improving LLM applications with OpenTelemetry integration
Debug LLM applications using the Phoenix CLI (`px`, from @arizeai/phoenix-cli). Fetch traces, spans, and sessions, annotate them, analyze errors, measure how often a failure or pattern occurs across traces, inspect datasets, review experiments, query…
Build, run, and validate evaluators for AI/LLM applications with Phoenix, in Python or TypeScript (including Vercel AI SDK apps) — code-based checks and LLM-as-a-judge, run over exported traces or dataframes or as Phoenix experiments, gate them in CI, and…
Create a new Harbor task for evaluating agents. Use when the user wants to scaffold, build, or design a new task, benchmark problem, or eval. Guides through instruction writing, environment setup, verifier design (pytest vs Reward Kit vs custom), and solution…
Creates and manages isolated cloud sandboxes (secure code execution environments with dedicated runtimes) on the Daytona platform. Use when a task needs an isolated runtime, sandbox, secure compute, or Daytona SDK/API/CLI operations. Covers Python,…
Manages stacked PRs and splits multi-part work into reviewable branches with gh-stack. Use for stack creation, viewing, edits, push, submit, sync, rebase, merge, or checkout; when asked to split or isolate work for review; whenever a user mentions a stack,…
Set up, write, and publish documentation sites with Mintlify. Use when the user wants to create a docs site for a repo or product, publish docs to a shareable URL, migrate docs from another platform, create docs pages, configure navigation, add components, or…
Write Harbor task verifiers using Reward Kit. Use when creating or editing a task's tests/ directory, adding grading criteria, setting up LLM/agent judges, or designing verifiers that produce a reward score.