braintrustdata

Add support for a new LLM provider format to Lingua. Follow the test-first workflow with payload snapshots.

Run transformation coverage tests. Use compact mode when iterating on fixes, full mode when planning or documenting bugs.

Lingua's testing tools for transformation coverage and end-to-end SDK validation.

Add support for a new LLM provider format to Lingua. Follow the test-first workflow with payload snapshots.

Run transformation coverage tests. Use compact mode when iterating on fixes, full mode when planning or documenting bugs.

Lingua's testing tools for transformation coverage and end-to-end SDK validation.

Create or update Braintrust Python SDK integrations built on the integrations API under `py/src/braintrust/integrations/`. Use when adding a new integration package, extending an existing provider integration, changing patchers, tracing, manual `wrap_*()` helpers, integration exports, `auto_instrument()` wiring, `py/noxfile.py` sessions, integration tests, or cassettes. Do not use when migrating an existing legacy wrapper from `py/src/braintrust/wrappers/` into the integrations API; use `sdk-wrapper-migrations` for that.

Review and refresh Braintrust Python SDK dependency update PRs, especially automated `chore(deps): weekly dependency update` PRs. Use when Codex needs to inspect `py/pyproject.toml` and `py/uv.lock`, reproduce the workflow's label decision, decide whether provider `latest` cassettes need re-recording, run the exact nox sessions, and validate playback before merge.

Run, compare, and extend Braintrust Python SDK pyperf benchmarks. Use when touching hot-path code in `py/src/braintrust/` such as serialization, deep-copy, span creation, or logging; when adding or updating files under `py/benchmarks/`; or when you need baseline/branch performance measurements with `cd py && make bench` and `make bench-compare`.

Triage and reproduce Braintrust Python SDK CI failures. Use when asked why CI failed, to fix broken CI on a PR, to inspect a failing GitHub Actions job, or to map a failing matrix job back to the exact local nox session, provider version, or workflow step that must be reproduced.

Work with Braintrust Python SDK VCR and cassette-backed tests. Use when adding or updating cassette-backed provider tests, deciding whether to re-record cassettes, debugging VCR failures, converting mock-heavy coverage to VCR, or handling cassette hygiene for provider integrations and Claude Agent SDK transport recordings.

Suggest a Braintrust SDK repo-style commit message from the current diff and conversation. Use when asked to write, suggest, or generate a commit message for the current changes.

Add or update Braintrust SDK instrumentation. Use when working on instrumentation of any kind - like wrappers, auto-instrumentation configs, tracing channels, provider plugins, vendored SDK typings, or instrumentation-specific tests.

Write, run, and debug end-to-end tests for the Braintrust SDK. Use when asked to add an e2e test, create a scenario, write an e2e scenario, add e2e coverage, debug e2e test, fix e2e snapshot, or any task involving the e2e/ directory.

Deterministic skill used by the Braintrust Flue instrumentation e2e test.

Workflow for fixing bugs in Ruby SDK integrations. Covers reproducing the bug, using appraisals, adding test cases, and TDD-based fixes.

This skill is for writing integrations to the Ruby SDK. Claude acts as the engineer implementing LLM provider or agentic framework integrations. Use when adding support for OpenAI-like providers, Anthropic-like providers, or agent frameworks. Covers TDD workflow, comprehensive testing (streaming/non-streaming/tokens/multimodal), defensive coding, MCP validation, and StandardRB compliance.

This plugin auto-configures a "braintrust" MCP server. If you can't see it or reach it, activate this skill

This skill should be used to review and audit the bt CLI for adherence to CLI best practices from clig.dev AND internal codebase patterns. It checks source code for help text, flags, error handling, output formatting, subcommand structure, pattern consistency, and more. Triggers on "review my code", "audit the CLI", "check CLI best practices", or /bt-review.

This skill is for writing integrations to the Go SDK. Claude acts as the engineer implementing LLM provider integrations. Use when adding support for OpenAI-like providers, Anthropic-like providers, or other LLM frameworks. Covers TDD workflow, comprehensive testing (streaming/non-streaming/tokens), VCR cassettes, orchestrion auto-instrumentation, and golangci-lint compliance.

Use the official Braintrust MCP server to search Braintrust docs, inspect projects, query logs and experiments, summarize eval results, and generate permalinks.

Use when bootstrapping a facet ground-truth dataset, running initial facet evals, or later optimizing a facet prompt in this repo.

Implement and review Braintrust SDK instrumentation for LLM providers and agent frameworks. Use when adding or changing traced span structure, captured inputs and outputs, tool calls, metadata, metrics, streaming behavior, attachments, token caching, or other behavior governed by the Braintrust instrumentation specification.

Improve any Braintrust-instrumented agent by turning production traces into an offline eval loop: mine bad traces, build a failure taxonomy, capture cases in a remote Braintrust dataset, write scorers and an eval file, iterate on prompts/tools, and only then push online scorers. Use when the user wants to write evals for their agent, fix recurring production failures, or set up an auto-improvement loop for an LLM app.