confident-tracing

# Confident Trace Use this skill to instrument an **AI application**—an LLM app, agent, RAG pipeline, or chatbot—with the **confident-trace SDK** so its execution is visible span by span in Confident AI. Pick a supported integration when one exists, fall back to custom spans otherwise, give each span a meaningful type, and add useful trace context. This skill stops at producing well-formed traces. Attaching evaluation metrics and running evals is the `deepeval` skill's job. ## Scope: AI Applications Only Instrument only the AI parts of the system: agent loops and planning, LLM calls, retrieval/vector search, and tool calls. Do not trace non-AI software such as web servers, CRUD backends, database layers, or infrastructure. If the target has no LLM, agent, retrieval, or tool-calling component, this skill does not apply. ## When to Use vs the `deepeval` and `confident-otel` Skills - **This skill (`confident-tracing`)**—instrument a Python or TypeScript app with the confident-trace SDK and its integrations. - **`deepeval` skill**—build pytest evaluation suites: datasets, metrics, traced evals, `deepeval test run`, and iteration. - **`confident-otel` skill**—export raw, vendor-neutral OpenTelemetry traces to Confident AI without the confident-trace package. ## Prerequisites - Python 3.10+ or Node.js 22+ with `confident-trace` installed. - A project-scoped `CONFIDENT_API_KEY` for export to Confident AI. ## Workflow 1. Confirm the target is an AI application. If it has no LLM, agent, retrieval, or tool-calling component, stop. 2. Detect the language, framework, model provider, agent SDK, gateway, bundler, and existing OpenTelemetry setup. 3. Read `references/integrations.md` and the exact repository integration documentation for what was detected. Prefer automatic instrumentation. 4. If no integration fits, or the code owns a meaningful outer boundary, instrument it with a custom span. Read `references/tracing.md`. 5. Give each span a meaningful type (`llm`, `retriever`, `tool`, `agent`, or `custom`) and capture useful inputs and outputs. 6. Add trace-level tags, metadata, user ID, customer ID, thread ID, turn ID, and environment where they help diagnose failure patterns. Structured `thread`, `user`, and `customer` objects carry extra properties such as names, tags, and metadata. 7. Never trace secrets, credentials, or unapproved sensitive data. Disable package-owned content capture when required. 8. Finish active work and streams before flushing or shutting down, then verify the trace hierarchy in Confident AI. ## Core Principles 1. Instrument AI components only. 2. Prefer a supported integration over custom instrumentation. 3. Read the exact integration documentation before writing tracing code. 4. Initialize once, before provider or framework calls. TypeScript automatic instrumentation requires both the Node preload and `init()`. 5. Do not wrap calls already covered by an integration. 6. Use `turn()` for conversation turns that must start separate traces; a thread ID associates turns but does not merge their OpenTelemetry traces. 7. Producing traces is the scope. Evals belong to `deepeval`; raw OpenTelemetry export belongs to `confident-otel`. ## References | Topic | File | | ------------------------------------------------------------------------- | ---------------------------- | | SDK instrumentation: initialization, custom spans, context, and lifecycle | `references/tracing.md` | | Integration selection and supported provider/framework index | `references/integrations.md` |