Run Comet's Opik — open-source LLM observability, evaluation, and optimization — from one routing-first skill: install the Python/TypeScript SDK, stand up a server (Comet.com cloud, Docker Compose via `./opik.sh`, or Kubernetes/Helm), wire tracing through `@opik.track` or one of 50+ framework integrations (OpenAI, Anthropic, LangChain, LangGraph, LlamaIndex, CrewAI, DSPy, Ollama, Bedrock, Vercel AI SDK, …), score outputs with LLM-as-a-judge metrics (Hallucination, Moderation, Answer Relevance, Context Precision), and run Datasets/Experiments evaluations including PyTest CI gates. Use when the user wants LLM tracing, prompt evaluation, production LLM monitoring, agent optimization, or guardrails with Opik. Triggers on: opik, comet opik, opik configure, opik.sh, llm observability, llm tracing, llm as a judge, hallucination metric, prompt evaluation, opik dashboard, opik guardrails, agent optimizer.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
Run Comet's Opik — open-source LLM observability, evaluation, and optimization — from one routing-first skill: install the Python/TypeScript SDK, stand up a server (Comet.com cloud, Docker Compose via `./opik.sh`, or Kubernetes/Helm), wire tracing through `@opik.track` or one of 50+ framework integrations (OpenAI, Anthropic, LangChain, LangGraph, LlamaIndex, CrewAI, DSPy, Ollama, Bedrock, Vercel AI SDK, …), score outputs with LLM-as-a-judge metrics (Hallucination, Moderation, Answer Relevance, Context Precision), and run Datasets/Experiments evaluations including PyTest CI gates. Use when the user wants LLM tracing, prompt evaluation, production LLM monitoring, agent optimization, or guardrails with Opik. Triggers on: opik, comet opik, opik configure, opik.sh, llm observability, llm tracing, llm as a judge, hallucination metric, prompt evaluation, opik dashboard, opik guardrails, agent optimizer.
allowed-tools
Bash Read Write Edit Glob Grep WebFetch
compatibility
Cross-platform observability/eval wrapper usable from Claude Code, Codex, Gemini CLI, and OpenCode. Wraps the upstream Opik server (cloud, Docker, Kubernetes) and the Python/TypeScript SDKs. Routes LangSmith-stack work to `langsmith`, generic service dashboards/alerts to `monitoring-observability`, and offline KPI interpretation to `data-analysis`.
Opik (built by Comet) streamlines the
entire LLM application lifecycle: deep tracing of LLM calls and agent
activity, LLM-as-a-judge evaluation, experiment management, production
monitoring at scale (40M+ traces/day), plus the Opik Agent Optimizer and
Opik Guardrails. This skill is the routing-first wrapper — it picks the
right deployment mode, wires the SDK, and drives the trace → evaluate →
monitor → optimize loop.
When to use this skill
The user asks to install or configure Opik (pip install opik,
opik configure, ./opik.sh)
The user wants tracing for LLM calls or agents — via @opik.track or a
framework integration (OpenAI, Anthropic, LangChain, LangGraph, LlamaIndex,
CrewAI, DSPy, Haystack, Ollama, Bedrock, Vercel AI SDK, Pydantic AI, …)
The user wants LLM-as-a-judge metrics: hallucination detection, moderation,
answer relevance, context precision/recall for RAG
The user wants Datasets + Experiments evaluation, or PyTest-integrated
LLM evaluation in CI/CD
The user wants production LLM monitoring dashboards, online evaluation
rules, prompt playground experiments, agent/prompt optimization, or
guardrails
When not to use this skill
The stack is LangSmith, not Opik → use langsmith
The user needs generic service dashboards/alerts (non-LLM telemetry,
uptime, infra metrics) → use monitoring-observability
The user wants offline dataset/KPI interpretation rather than LLM
tracing/eval plumbing → use data-analysis
The user is doing root-cause log forensics on app/container logs →
use log-analysis
Annotate traces/spans with feedback scores via the SDK or the UI.
Step 4 — Evaluate with LLM-as-a-judge metrics
from opik.evaluation.metrics import Hallucination
metric = Hallucination()
score = metric.score(
input="What is the capital of France?",
output="Paris",
context=["France is a country in Europe."],
)
print(score)
Built-in judges include Hallucination, Moderation, Answer Relevance, Context
Precision/Recall; heuristic metrics and custom metrics are also supported.
Step 5 — Datasets, Experiments, and CI gates
Manage Datasets and run Experiments to compare prompt/model
variants during development
Wire evaluations into CI/CD with the PyTest integration so regressions
block merges
Iterate on prompts/models in the Prompt Playground
Step 6 — Production monitoring and optimization
Opik is built for scale: 40M+ traces/day ingestion
Track feedback scores, trace counts, and token usage in the Opik Dashboard
Add Online Evaluation Rules (LLM-as-a-judge on production traffic) to
catch issues live
Use Opik Agent Optimizer (dedicated SDK) to improve prompts/agents and
Opik Guardrails for safe-AI policies
This skill folder is plugin-installable through the standard oh-my-skills
flow so the wrapper, references, and installer script land on disk for any
supported agent runtime:
# Project install (writes into .agents/skills/opik/)
npx skills add https://github.com/akillness/oh-my-skills --skill opik
# Global install for every detected agent
npx skills add -g https://github.com/akillness/oh-my-skills --skill opik
# Target specific agents
npx skills add -g https://github.com/akillness/oh-my-skills --skill opik -a claude-code -a codex -y
The skill also ships scripts/install.sh as a
one-shot installer covering SDK install (uv → pip fallback) and optional
local self-hosting (OPIK_INSTALL_MODE=local).
Output format
When the user asks opik for help, return a compact brief:
# opik Routing Brief## Scope- Server mode: cloud | docker-local | kubernetes | undecided
- SDK: python | typescript | ruby-otel
- Lifecycle stage: tracing | evaluation | ci-gate | production-monitoring | optimization | guardrails
## Recommended next move- install-sdk | opik-configure | start-local-server | wire-integration | add-judge-metric | create-dataset-experiment | enable-online-rules
## Why- 2-3 bullets grounded in the user's packet
## Route-outs-`langsmith` when the observability stack is LangSmith
-`monitoring-observability` for non-LLM dashboards/alerts
-`data-analysis` for offline KPI/metric interpretation
Best practices
Start with cloud or ./opik.sh, not Kubernetes — Helm is for
production scale; local Docker Compose answers "does tracing work" in
minutes.
Prefer a native integration over hand-rolled @opik.track when the
framework is in the support table — integrations capture provider
metadata (tokens, model, latency) automatically.
Check the changelog before upgrading a self-hosted server — e.g.
v1.7.0 shipped breaking changes.
Judge metrics need context — Hallucination and RAG metrics score
against the context you pass; empty context produces misleading scores.
Gate CI on small, stable datasets — PyTest-integrated experiments
should be fast and deterministic; keep large sweeps in scheduled runs.
Turn production checks into Online Evaluation Rules instead of
re-running offline experiments against live traffic.
