| name | honeyhive-instrument |
| description | Instrument an AI application with HoneyHive tracing. Use when the user wants to add observability, capture traces, set up the SDK, configure exporters, or wire OpenTelemetry-compatible instrumentation for an agent, RAG pipeline, or LLM application. Triggers on phrases like "add tracing", "instrument my app", "wire up HoneyHive", "set up the SDK", "capture spans", "OTEL exporter", or any mention of `HoneyHiveTracer.init`, `@honeyhive/api-client`, `client.sessions.startSession`, `client.events.createEvent`, or `OpenInference`/`Traceloop` instrumentors targeting HoneyHive.
|
| license | MIT |
| metadata | {"version":"0.2.1","homepage":"https://docs.honeyhive.ai","feedback_url":"https://github.com/honeyhiveai/skills/issues"} |
HoneyHive Instrument
Wire HoneyHive tracing into a user's application with a small, surgical edit set: initialize the tracer, attach an instrumentor for their LLM/agent framework, and verify spans flush and the trace is visible in HoneyHive.
You are an instrumentation surgeon, not a rewriter. The user has working code. Add tracing in the smallest number of edits possible. Never refactor unrelated code, never change framework versions, never strip features.
Three phases only
This skill has exactly three phases. Each phase has a checkpoint — a concrete artifact that proves the phase ran. If you skip a phase, the checkpoint will be missing and the eval will catch it.
| Phase | Goal | Checkpoint |
|---|
| Validate | Confirm network + runtime before touching code | Curl returns HTTP 200 + runtime findings recorded |
| Setup | Add tracing wiring via the language-specific reference | Exactly one setup reference loaded + edits applied |
| Verify | Smoke run + grade against success.md | Session visible in HoneyHive + success.md loaded and graded |
Phase 1 — Validate (deterministic, no user questions until stuck)
Validation is deterministic discovery. Run checks in order; stop on failure. Only ask the user when a check is ambiguous.
Step 1.1 — Network validation
Load and follow references/network-validation.md.
Run the reachability check:
curl -sS -o /dev/null -w "HTTP %{http_code}\n" \
-H "Authorization: Bearer $HH_API_KEY" \
"${HH_DATA_PLANE_URL:-$HH_API_URL}/healthcheck"
Checkpoint: Curl returns HTTP 200. If not, stop with the exact error from the interpretation table below. Do not retry. Do not proceed.
Step 1.2 — Runtime validation
Load and follow references/runtime-validation.md.
This is a read-only scan. Do not edit any files. Detect:
- Subtree — locate the LLM app in a monorepo (ask if ambiguous)
- Language + runtime — from manifests (pyproject.toml, package.json, go.mod, etc.)
- Framework + version — which LLM SDK is pinned
- Instrumentor family — OpenInference vs Traceloop (Python only)
- Existing HH wiring — grep for
HoneyHiveTracer.init / new Client(
- Existing OTel provider — grep for
TracerProvider setups from other vendors
Checkpoint: All six findings recorded. The setup path (Python / TypeScript / native OTel) is determined.
Step 1.3 — Confirm the path with the user
Present findings and the proposed setup path. Confirm before adding dependencies.
Phase 2 — Setup (load exactly one language reference; follow it end-to-end)
Pick the path that matches runtime-validation findings. Load exactly one setup reference:
Each setup file consumes Phase 1's findings and has its own internal steps (dependency plan, init placement, session boundaries, distributed propagation, verification). Follow it end-to-end.
Load additional references only when findings demand it:
For frameworks with first-class docs coverage, also pull the framework's integration docs page for the exact install + minimal-integration block — copy verbatim, adapt to the user's entry-point:
Checkpoint: Setup reference loaded. Dependency changes proposed (not yet committed). Init placement, session boundaries, and propagation strategy decided.
If the user's framework isn't on the index above, follow the doc-gap protocol in Phase 3 step 3.
Phase 3 — Verify
Step 3.1 — Smoke run
Run the user's main entry-point with a one-line dummy prompt (if env vars are set). Confirm:
- Process exits 0
- No OTEL warnings (
No tracer provider has been initialized, Span context lost)
Step 3.2 — Mechanical trace check
Query for events using the events search endpoint:
curl -s -X POST "${HH_DATA_PLANE_URL:-$HH_API_URL}/v1/events/search" \
-H "Authorization: Bearer $HH_API_KEY" \
-H "Content-Type: application/json" \
-d '{"project": "<project-name>", "filters": [{"field": "event_type", "operator": "is", "value": "session", "type": "string"}], "limit": 5}'
Find the most recent session, then search for its child events:
curl -s -X POST "${HH_DATA_PLANE_URL:-$HH_API_URL}/v1/events/search" \
-H "Authorization: Bearer $HH_API_KEY" \
-H "Content-Type: application/json" \
-d '{"project": "<project-name>", "filters": [{"field": "session_id", "operator": "is", "value": "<session-id>", "type": "string"}], "limit": 50}'
Verify: at least one event exists, and at least one has event_type: "model" (LLM span). If the smoke run exits cleanly but zero events or no model events are visible, do not guess at code changes first. Consult https://docs.honeyhive.ai/v2/introduction/troubleshooting, then inspect env vars, dependency installation, exporter output, and shutdown behavior as applicable. Only return to Phase 2 after those checks point to instrumentation placement or instrumentor choice.
Step 3.3 — Grade against success criteria
Load success.md and grade the trace:
- Trace structure — are event names meaningful? are model events shallow (depth ≤ 2)? are tool calls linked to parent agent?
- Input/output cleanliness — does the session show the user's query? do LLM events show full message history + tools + response?
- Semantic failures — are errored events surfaced clearly, not buried in metadata?
- Summarisability — is there a final event with enough context for end-to-end evaluators?
If the trace is mechanically correct but qualitatively messy, circle back to Phase 2 and adjust session-boundary placement or instrumentor choice. A mechanically-correct integration that produces a messy trace is not done.
Auto-instrumentation may not capture everything important. If the trace is missing key steps (e.g., retrieval, tool calls, orchestration logic), add manual spans and enrichments using the APIs from the setup reference loaded in Phase 2. Auto-instrumentation is the starting point, not the ceiling.
Step 3.4 — Verify against MUSTs
Review the code changes against the Requirements (MUSTs) section below. Confirm no violations (hardcoded keys, dual tracer, global provider hijack, etc.).
Step 3.5 — Doc-gap surfacing
If during discovery you found a framework, instrumentor, or OTel exporter combination not covered above, do not improvise. Tell the user: "I've flagged this as a doc gap; here's the working install I produced; please open an issue at https://github.com/honeyhiveai/skills/issues if you'd like first-class coverage."
Checkpoint: Session visible in HoneyHive with model events. success.md loaded and graded. Trace passes the rubric.
Stop
Single-pass. Do not continue iterating after success criteria are met. Instrumentation is the only goal — once spans are flowing and the trace grades well, hand off to the user. Anything beyond (evaluating quality, improving prompts, tuning retrieval) is out of scope.
References
Load only when the phase calls for them. Progressive disclosure keeps context tight.
Peer files:
- success.md — qualitative rubric for Phase 3 trace grading
Phase 1 — Validation:
Phase 2 — Setup (load exactly one):
Operational (load when findings demand it):
Requirements (MUSTs)
These are hard rules. Violating any one is a skill failure that evaluators will catch.
- MUST NOT proceed past Phase 1 without successful network validation. Wrong URL or rejected key means every later step rebuilds sand.
- MUST NOT globally register HoneyHive as the source-of-truth TracerProvider. The SDK doesn't do this; instrumentation code must not either. Evaluator:
no-global-provider-hijack.
- MUST NOT construct two tracers (Python) or two Client instances (TS). Reuse the existing instance. Evaluator:
no-dual-tracer-instances.
- MUST NOT hardcode API keys. Always sourced from env vars. Evaluator:
no-hardcoded-api-keys.
- MUST NOT let the deployment URL silently fall back to the wrong host. Confirm explicitly — the env var name varies by SDK (
HH_DATA_PLANE_URL for TS/CLI, HH_API_URL for Python). Multi-tenant default is https://api.dp1.us.honeyhive.ai; dedicated and self-host customers have their own.
- MUST NOT switch instrumentor families (OpenInference ↔ Traceloop) without explicit user consent.
- MUST NOT thread session_id through function signatures (TS path). Use a module-level helper or async-context store.
- MUST NOT bump SDK or framework versions to make instrumentation work.
- MUST NOT modify test files or lock files unless the user explicitly asks.
- MUST NOT run a production deploy step. Local smoke run is the verification ceiling.
- MUST load success.md at verification time (Phase 3), not at the start.
- MUST record runtime-validation findings before loading a setup reference. The setup file consumes findings, it does not re-detect.
