Arize-ai

TypeScript conventions and patterns for any TypeScript code in the Phoenix monorepo — including js/packages/, js/app/, and any other TS directories. Use this skill whenever writing, reviewing, or modifying TypeScript code — new functions, types, exports,…

Run and extend Phoenix's executable integration documentation checks. Use when changing an integration doc, its runnable example, or the CI harness.

Triage a window of Phoenix Cloud production errors, read from Datadog across every tenant, into GitHub issues on Arize-ai/phoenix. Use when asked to triage Phoenix Cloud errors or turn a Datadog log search into issues.

Triage a window of production errors from the Phoenix server the team runs on Railway, read through Grafana, into GitHub issues on Arize-ai/phoenix. Use when asked to triage Railway errors or turn a LogQL error query into issues.

Open-source AI observability platform for tracing, evaluating, and improving LLM applications with OpenTelemetry integration

Build, run, and validate evaluators for AI/LLM applications with Phoenix, in Python or TypeScript (including Vercel AI SDK apps) — code-based checks and LLM-as-a-judge, run over exported traces or dataframes or as Phoenix experiments, gate them in CI, and…

OpenInference semantic conventions and instrumentation for Phoenix AI observability. Use when implementing LLM tracing, creating custom spans, or deploying to production.

Write Playwright E2E tests for the Phoenix AI observability platform. Use when creating, updating, or debugging Playwright tests, or when the user asks about testing UI features, writing E2E tests, or automating browser interactions for Phoenix.

Manages Arize users, organizations, spaces, projects, roles, role bindings, resource restrictions, and API keys via the ax CLI. Use for enterprise admin workflows: inviting and offboarding users, onboarding new teams, creating custom roles for SAML/SSO…

Creates, reads, updates, and deletes Arize AI integrations that store LLM provider credentials used by evaluators and other Arize features. Supports any LLM provider (e.g. OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Vertex AI, Gemini, NVIDIA NIM, Fireworks…

Creates and manages annotation configs (categorical, continuous, freeform label schemas) and annotation queues (human review workflows) on Arize. Applies human annotations to project spans via the Python SDK. Use when the user mentions annotation config,…

Creates, manages, and queries Arize datasets and examples. Covers dataset CRUD, appending examples, exporting data, and file-based dataset creation using the ax CLI. Use when the user needs test data, evaluation examples, or mentions create dataset, list…

Handles LLM-as-judge and code evaluator workflows on Arize including creating/updating evaluators, running evaluations on spans or experiments, managing tasks, trigger-run operations, column mapping, and continuous monitoring. Use when the user mentions…

Creates, runs, and analyzes Arize experiments for evaluating and comparing model performance. Covers experiment CRUD, exporting runs, comparing results, and evaluation workflows using the ax CLI. Use when the user mentions create experiment, run experiment,…

Adds Arize AX tracing to an LLM application for the first time. Detects the stack, routes to the single matching integration doc, wires auto-instrumentation after user confirmation, and verifies traces land. Use when the user wants to instrument their app,…

Migrates LLM observability from Braintrust into Arize AX — historical trace import via OTLP (BT output→output.value, span_parents, one AX trace per turn + session.id), datasets, and prompts (latest or versions). Use when migrating from Braintrust to Arize or…

Attach Agent-as-a-Judge (AaaJ / harness) evaluators to any Arize AX Wonder Toys project and trigger a one-time backfill. Reads scoring instructions from evals/aaaj/*.md and creates evaluators + tasks via GraphQL (REST and the ax CLI cannot create harness…

Run a public-flow Playwright smoke test against a freshly-built framework tier's Next.js frontend. Covers home page rendering and product browsing. The Playwright project (package.json, config, tests) lives inside this skill directory and is checked into the…

Add a new agent framework to the Rosetta Stone repo — researches the framework, builds all three observability tiers (no-observability, phoenix, ax), tests each, runs a Playwright smoke against the UI, updates README + TODO, and raises a PR. Trigger when the…

Build a single tier (no-observability, phoenix, or ax) for a new framework. Clones the closest existing tier, swaps in framework-specific agent.py / tools.py / requirements.txt, and (for observability tiers) adds tracing.py + main.py wiring + eval-harness…

Find and update out-of-date OpenAI and Anthropic model references (in docs, MDX, notebooks, and code) to the latest size-equivalent models, and apply the code changes each new generation requires (e.g. max_tokens → max_completion_tokens for GPT-5). Use when…

Capture AX trace UI, Phoenix trace UI, and Wonder Toys app UI screenshots for a framework's PR, then upload as GitHub release assets and embed in the PR body. Called automatically by rosetta-add-framework-docs as part of new-framework PRs; can also be invoked…

Finalise a newly-added framework — updates the README's supported-frameworks table, directory tree, and per-framework "what differs" section, marks off the framework in the orchestrator skill's embedded TODO, commits per tier, and raises a PR. Part of the…

Refresh the list of agent frameworks supported by Arize tracing and diff against what's already in the repo. Pulls live data from https://arize.com/docs/llms.txt and produces a clean to-do list. Part of the rosetta-add-framework flow; can also be invoked…

Use before opening a pull request to this repo. Reviews the working diff for correctness bugs and repo conventions, runs the test/lint gates, and can post findings directly on the PR. Triggers on "review my changes", "review this PR", "pre-PR review".

Triages coding-harness-tracing issues by type, sufficiency, harness, requirements, complexity, and contributor readiness. Use when triaging issues, sweeping the backlog, or auditing existing good-first-issue labels.

Set up and configure Arize tracing for Cursor IDE sessions. Use when users want to set up tracing, configure Arize AX or Phoenix for Cursor, enable/disable tracing, or troubleshoot tracing issues. Triggers on "set up cursor tracing", "configure Arize for…

Set up and configure Arize tracing for OpenAI Codex CLI sessions. Use when users want to set up Codex tracing, configure Arize AX or Phoenix for Codex, enable/disable tracing, or troubleshoot Codex tracing issues. Triggers on "set up codex tracing",…

Debug LLM applications using the Phoenix CLI. Fetch traces, analyze errors, structure trace review with open coding and axial coding, inspect datasets, review experiments, query annotation configs, and use the GraphQL API. Use whenever the user is analyzing…

Set up and configure Arize tracing for Google Antigravity CLI/IDE sessions. Use when users want to set up tracing, configure Arize AX or Phoenix for Antigravity, enable/disable tracing, or troubleshoot tracing issues. Triggers on "set up antigravity tracing",…

Set up and configure Arize tracing for opencode terminal coding sessions. Use when users want to set up tracing, configure Arize AX or Phoenix for opencode, enable/disable tracing, or troubleshoot tracing issues. Triggers on "set up opencode tracing",…

Set up and configure Arize tracing for Claude Code sessions or Agent SDK applications. Use when users want to set up tracing, configure Arize AX or Phoenix, create a new Arize project, get an API key, enable/disable tracing, or troubleshoot tracing issues.…

TypeScript conventions for any TypeScript code in js/ (packages, examples, tests). Use whenever writing, reviewing, or modifying TypeScript in this repo.

Run open-source triage on Arize-ai/openinference issues — classify type, check sufficiency, apply language, instrumentation and component labels, check the mandatory instrumentor requirements (suppress tracing, context attributes, TraceConfig masking,…

Prove an OpenInference instrumentation change end to end by running a package example against a local Phoenix (http://localhost:6006) in a dedicated project and inspecting the resulting spans with the px CLI or the Phoenix MCP server. Use when an…

Run, interpret, and iterate on the OpenInference GenAI conformance MVP at python/openinference-instrumentation/scripts/conformance/. Use when the user mentions GenAI conformance, OTel GenAI semantic conventions, Weaver registry live-check, the dual-write…

Investigate and propose fixes for Python canary cron failures in the openinference repo. Use when the user mentions Python canary failures, Python cron failures, or when the auto-fix CI job reports Python instrumentation canary issues.

Keep hand-written docs/ documentation in JS packages accurate and up to date with their source code. Use this skill whenever: (1) source files in a JS package that has a docs/ folder are modified — especially exports, function signatures, types, or public API…

Review Java OpenInference instrumentation code for correctness and completeness. Use this skill when reviewing a Java instrumentor package — whether it's a new instrumentor, a PR that modifies one, or when the user asks to audit/review/check an existing…

Review Python OpenInference instrumentation code for correctness and completeness. Use this skill when reviewing a Python instrumentor package — whether it's a new instrumentor, a PR that modifies one, or when the user asks to audit/review/check an existing…

Set up and configure Arize tracing for Claude Code sessions or Agent SDK applications. Use when users want to set up tracing, configure Arize AX or Phoenix, create a new Arize project, get an API key, enable/disable tracing, or troubleshoot tracing issues.…

Manage datasets in Arize AI using the ax CLI. Use when users want to list datasets, get dataset details, create new datasets, delete datasets, export dataset data, or work with dataset examples. Triggers on "list datasets", "create dataset", "ax datasets",…

Manage projects in Arize AI using the ax CLI. Use when users want to list projects, get project details, create new projects, delete projects, or organize work within Arize spaces. Triggers on "list projects", "create project", "ax projects", "delete…

Use when testing an already-running Claude collector by sending a live marker trace to its configured OTLP endpoint without starting a second collector.

Use when validating Claude Code telemetry setup for this collector, especially to confirm Claude emits OTLP data to a local file before testing the configured upstream destination.

INVOKE THIS SKILL when translating a context-graph-mining report into an experiment variant for the procurement-agent. Reads the report's proposed diffs and writes a variant config bundle to `experiments/variants/<id>/` (manifest.yaml + system_prompt.txt +…

INVOKE THIS SKILL when mining the procurement-agent Arize project for patterns in agent decisions vs human overrides, building a context graph, and proposing updates to procurement-agent rules. Use after a batch of reviews has been captured (process.run +…

Build and run evaluators for AI/LLM applications using Phoenix.

OpenInference semantic conventions and instrumentation for Phoenix AI observability. Use when implementing LLM tracing, creating custom spans, or deploying to production.

Guides first-time customers through an Arize AX on-prem or self-hosted install: selecting or downloading a distribution tarball, preparing values.yaml, running arize.sh, and validating Kubernetes. Use for "on-prem install", "self-hosted install",…

Diagnoses self-hosted Arize AX clusters using read-only kubectl/Prometheus/Alertmanager access and local distribution docs. Use when troubleshooting self-hosted alerts, finding related documentation for firing alerts, or investigating self-hosted cluster…

Find and update out-of-date OpenAI and Anthropic model references (in docs, MDX, notebooks, and code) to the latest size-equivalent models, and apply the code changes each new generation requires (e.g. max_tokens → max_completion_tokens for GPT-5). Use when…