Arize-ai/phoenix

arize-phoenix

Open-source AI observability platform for tracing, evaluating, and improving LLM applications with OpenTelemetry integration

phoenix-evals

Build and run evaluators for AI/LLM applications using Phoenix.

phoenix-graphql

Write efficient GraphQL queries against the Phoenix API. Load this skill in two cases: (1) before composing any non-trivial GraphQL query yourself for data analysis (via the `phoenix-gql` bash command) — it contains schema entrypoints and patterns that…

phoenix-tracing

OpenInference semantic conventions and instrumentation for Phoenix AI observability. Use when implementing LLM tracing, creating custom spans, or deploying to production.

phoenix-harbor

Configure and interpret the Phoenix plugin for Harbor agent evaluations. Use when adding `arize-phoenix` to Harbor jobs, choosing ATIF tracing, mapping Harbor tasks and rewards to Phoenix experiments, comparing agents or models, resuming jobs, or…

phoenix-pr-description

Writes and maintains the body of a pull request opened by a coding agent on Arize-ai/phoenix, including a fixed "How this PR was made" section that reports steering, abandoned approaches, and verification gaps so a reviewer can pick a review depth. Use when…

phoenix-pr-screenshot

Screenshot a running Phoenix feature and attach images to a GitHub PR. Builds the frontend, starts Phoenix with env vars, captures browser screenshots, and attaches them to the PR body with `gh pr edit --attach`.

phoenix-cli

Debug LLM applications using the Phoenix CLI. Fetch traces, spans, and sessions, annotate them, analyze errors, inspect datasets, review experiments, query annotation configs, and use the GraphQL API. Use whenever the user works with a Phoenix instance from…

phoenix-design

Design system conventions for the Phoenix frontend — layout, dialogs, error display, BEM CSS class naming, and CSS design tokens. Use when building UI, naming CSS classes, creating or consuming tokens, handling errors, or designing dialog interactions in…

phoenix-frontend

Frontend development guidelines for the Phoenix AI observability platform. Use when writing, reviewing, or modifying React components, TypeScript code, styles, or UI features in the js/app/ directory. Triggers on any frontend task — new components, UI…

phoenix-storybook

Conventions for creating, modifying, and reviewing production-faithful Storybook stories in the Phoenix frontend (js/app/stories, js/app/.storybook). Covers sidebar taxonomy and titles, the tag vocabulary, entry shape, option grids, component audits,…

phoenix-sqlean

Maintaining packages/phoenix-sqlean, the vendored fork of nalgeon/sqlean.py published as arize-phoenix-sqlean. Use when bumping the bundled SQLite, sqlean, or xxHash versions, changing its wheel matrix, or touching its publish path. Trigger on any change…

create-task

Create a new Harbor task for evaluating agents. Use when the user wants to scaffold, build, or design a new task, benchmark problem, or eval. Guides through instruction writing, environment setup, verifier design (pytest vs Reward Kit vs custom), and solution…

harbor-exec

Use when working with Harbor's `harbor exec` CLI workflow: compiling files, directories, or globs into Harbor tasks; running map jobs; configuring artifacts and existence-only verification; using map-reduce; writing or reviewing ExecConfig YAML/JSON/TOML; or…

rewardkit

Write Harbor task verifiers using Reward Kit. Use when creating or editing a task's tests/ directory, adding grading criteria, setting up LLM/agent judges, or designing verifiers that produce a reward score.

daytona

Creates and manages isolated cloud sandboxes (secure code execution environments with dedicated runtimes) on the Daytona platform. Use when a task needs an isolated runtime, sandbox, secure compute, or Daytona SDK/API/CLI operations. Covers Python,…

mintlify

Build and maintain documentation sites with Mintlify. Use when creating docs pages, configuring navigation, adding components, or setting up API references.

issue-triage

Run open-source triage on Arize-ai/phoenix issues — classify type, check sufficiency, apply component and complexity labels, flag work that already shipped, investigate complex bugs, tidy formatting, then gate against a contributor policy such as "good…

phoenix-error-analysis

Find out what is going wrong in LLM or agent traffic by reading sampled Phoenix traces, spans, or sessions, writing free-form notes (open coding), then grouping the notes into a few narrow annotations, one per failure dimension with a small label set and…

phoenix-server

Backend development guide for the Phoenix AI observability platform (Strawberry GraphQL, SQLAlchemy async, FastAPI). Use this skill when writing or modifying Python server code in the phoenix repo — adding mutations, types, migrations, or tests. Trigger on…

evaluators

Author or refine a Phoenix evaluator — code or LLM-as-a-judge — that scores a run's output. Trigger when the user wants to create a new evaluator, improve an existing one's logic or rubric, choose labels, or decide what to measure on a dataset or experiment.…

experiments

Run, read, and compare dataset-backed experiments to find evidence that a prompt or pipeline is improving. Trigger when the user wants to iterate over a dataset with experiments, compare experiment runs, read experiment quality/latency/cost, or decide whether…

phoenix-release-notes

Create Phoenix release documentation grounded in actual code changes. Use this skill whenever the user asks to write release notes, document a release, update release documentation, or mentions undocumented releases. Also trigger when the user wants to update…

phoenix-playwright-tests

Write Playwright E2E tests for the Phoenix AI observability platform. Use when creating, updating, or debugging Playwright tests, or when the user asks about testing UI features, writing E2E tests, or automating browser interactions for Phoenix.

phoenix-rest-api

REST API development for Phoenix. Use when adding, modifying, or reviewing endpoints in src/phoenix/server/api/routers/v1/.

phoenix-skills-audit

Audit recent changes to Phoenix's user-facing surfaces (Python clients, TypeScript clients, CLI, REST/GraphQL APIs) and patch the three external-facing agent skills — `phoenix-tracing`, `phoenix-cli`, and `phoenix-evals` — so they stay in sync with what…

phoenix-client-development

Development guide for the @arizeai/phoenix-client TypeScript SDK — run and resume experiments, manage OpenTelemetry tracer providers with stack-based attach/detach, and write vitest unit and integration tests. Use when adding features to phoenix-client,…

gh-stack

Manages stacked PRs and splits multi-part work into reviewable branches with gh-stack. Use for stack creation, viewing, edits, push, submit, sync, rebase, merge, or checkout; when asked to split or isolate work for review; whenever a user mentions a stack,…

playground

Author, edit, or iterate on prompts in the Phoenix prompt playground, including running experiments over a dataset. Load before any playground `ui.*` operation call, including single-shot prompt rewrites.

phoenix-docs-gap-audit

Audit documentation gaps across the Phoenix repo by analyzing recent commits to main (default: last 7 days). Use this skill whenever the user asks to find undocumented features, identify docs gaps, audit what shipped without docs, check which recent changes…

phoenix-integration-snippets

Generates onboarding code snippets for Phoenix tracing integrations and wires them into the project onboarding UI. Produces install dependencies and implementation sections for SDKs like OpenAI, LangChain, Vercel AI SDK, and others. Supports Python and…

phoenix-pxi-playwright

Write, extend, and debug PXI Playwright E2E tests for Phoenix. Use when adding PXI agent frontend specs, authoring LLM-as-judge rubrics, asserting PXI tool use, persisting PXI test runs as Phoenix experiments, or debugging PXI E2E failures.

phoenix-typescript

TypeScript conventions and patterns for any TypeScript code in the Phoenix monorepo — including js/packages/, js/app/, and any other TS directories. Use this skill whenever writing, reviewing, or modifying TypeScript code — new functions, types, exports,…

typescript-tooling-migration

Migrate or upgrade TypeScript tooling in the Phoenix monorepo. Use when upgrading TypeScript versions, switching tools (ESLint to oxlint, Prettier to oxfmt), upgrading bundlers (Vite, esbuild), or making major dependency upgrades. Triggers on requests to…

phoenix-github

Manage GitHub issues, labels, project boards, sprint operations, and roadmap health for the Arize-ai/phoenix repository. Use when filing roadmap issues, triaging bugs, applying labels, running sprint close-out and rollover, auditing board hygiene, checking…

phoenix-cli-development

Design and implementation guide for the Phoenix CLI (`px`). Covers the noun-verb command structure, dual-audience design (humans and coding agents), Commander.js patterns, configuration resolution, output formats, exit codes, and conventions for adding or…

phoenix-evals-new-metric

Create a new built-in classification evaluator for Phoenix evals. Use this skill whenever the user asks to create a new eval, build a new metric, add a new builtin evaluator, create an LLM-as-a-judge metric, or add a new classification evaluator to Phoenix.

phoenix-typescript-package-docs

Maintain the bundled TypeScript package docs that ship inside Phoenix npm packages. Use this skill whenever adding or updating docs for `@arizeai/phoenix-client`, `@arizeai/phoenix-evals`, or `@arizeai/phoenix-otel`, when changing the Mintlify package-doc…

datasets

Understand what a Phoenix dataset is and reason well about its examples, outputs, splits, and how it feeds evaluators and experiments. Load this whenever a dataset is in view or the user asks what a dataset is, how splits work, what an output "means", or how…

pxi-eval-dataset

Generate synthetic evaluation datasets for the PXI eval harness (evals/pxi/). Use whenever the user asks to create, author, draft, expand, or audit an eval dataset for a PXI tool, skill, or behavior — including phrases like "write evals for <tool>", "test PXI…