OpenTelemetry tracing discipline: correct spans, propagation, and semantic conventions produce useful traces. Invoke whenever task involves any interaction with distributed tracing — span creation, context propagation, instrumentation, sampling configuration, or OpenTelemetry SDK setup.
OTel Tracing
Create spans for logical operations, propagate context across every boundary, use semantic conventions for attribute
names. Every tracing decision — span granularity, attribute selection, sampling strategy — trades off cost against
visibility.
Timestamped annotations — structured log entries attached to a span.
Use events when the timestamp matters; use attributes for metadata with no meaningful timestamp
Add events for key domain occurrences ("page became interactive", "retry attempted")
Use events for verbose data instead of additional spans
Recording Exceptions
Always pair RecordException with setStatus(ERROR) — RecordException alone does not change span status
Never swallow application exceptions in instrumentation — catch instrumentation errors, log them, but always rethrow
application exceptions
Links
Associate a span with spans from other traces without parent-child hierarchy.
Batch processing: link consumer span to each producer span
Async follow-up: job span links back to triggering span
Add links at span creation when possible — samplers can consider them
See ${CLAUDE_SKILL_DIR}/references/span-data.md for events format, links format, and SDK limits table.
Context Propagation
Context propagation correlates spans across service boundaries. Without it, each service produces isolated spans — no
trace.
Inject / Extract Pattern
Sender creates a span, makes it current, injects context into outgoing carrier (HTTP headers, message
metadata)
Receiverextracts context from incoming carrier and uses it as parent for new spans
Call inject AFTER creating the outgoing CLIENT or PRODUCER span
Call extract BEFORE creating the server/consumer span
Use W3C TraceContext as the default propagation format
In-Process Propagation
Make spans active/current — enables automatic log correlation, nested auto-instrumentation picking up correct
parent, context propagation to child ops
Pass context explicitly in async code — automatic propagation may break across thread boundaries, goroutines, or
async callbacks
Never propagate request context to background work — background tasks should start new traces and link back to the
triggering span
Baggage
Propagates arbitrary key-value pairs across service boundaries alongside trace context.
Use for: tenant ID, request priority, feature flags, sampling hints
NEVER put PII, credentials, or API keys in baggage — visible to all downstream services
See ${CLAUDE_SKILL_DIR}/references/context-propagation.md for W3C header format, propagator selection table, baggage
details, and security considerations.
Instrumentation
Approaches
Approach
When
Trade-off
Automatic
Supported libraries (HTTP, DB, messaging)
Zero code, no business attrs
Manual
Business logic, unsupported libraries
Full control, more code
Hybrid (recommended)
Production
Auto for infra + manual for business logic
Getting a Tracer
Name the tracer after your library, package, or module — not the application
Include a version string matching your library version
Libraries: accept TracerProvider via dependency injection or use the global one
Applications: configure the SDK and set the global TracerProvider
What to Instrument
Good candidates: public API methods with I/O or significant computation, request/message handlers, outbound calls
(HTTP, DB, RPC), background jobs.
Poor candidates: every function call (noise), thin wrapper libraries (already instrumented underneath), pure
computation with no I/O and sub-ms duration.
Decision: Is it a network call or significant I/O? -> Create span (CLIENT/SERVER). Is it a meaningful business
operation? -> Create span (INTERNAL). Would a span event on parent suffice? -> Add event. Otherwise -> don't instrument.
Library Instrumentation Rules
Depend on OpenTelemetry API only — never SDK. The API is a no-op without SDK — zero overhead for users who don't
use OTel.
Follow semantic conventions for your domain
Set the schema_url to record which semantic convention version you use
Prefer events over spans for verbose internal details
Support optional TracerProvider injection for testability
See ${CLAUDE_SKILL_DIR}/references/instrumentation.md for server-side, client-side, and async producer/consumer
patterns, plus testing guidance.
Sampling
Sampling controls which traces are recorded and exported — the primary cost management mechanism.
Head Sampling
Decision at trace creation time, before any spans complete.
AlwaysOn — Record and sample everything
AlwaysOff — Drop everything
TraceIdRatioBased(ratio) — Sample based on trace ID hash
ParentBased(root) — Delegate based on parent sampling decision
ParentBased is the most common production configuration — respects parent decisions, applies custom root sampling.
Default SDK sampler: ParentBased(root=AlwaysOn).
Tail Sampling
Decision after all spans in a trace complete. Requires collector infrastructure.
All services export at 100%
OTel Collector with tail sampling processor buffers spans by trace ID
All spans with the same trace ID MUST reach the same collector — use trace-ID-aware load balancing
Sampling Principles
Start with no sampling — add only when cost or volume requires it
Use ParentBased — children follow parent decisions for complete traces
Provide attributes at span creation — samplers cannot see late-added attributes
Filter health checks — high volume, low value; sample aggressively or filter entirely
See ${CLAUDE_SKILL_DIR}/references/sampling.md for combined head+tail strategies, decision guide, and per-scenario
recommendations.
Semantic Conventions
Use semantic conventions for span names and attributes. Consistent naming enables cross-service analysis without
learning custom attribute names.
HTTP server spans: kind SERVER, name {METHOD} {http.route}, never full URI path
HTTP client spans: kind CLIENT, name {METHOD}
Database spans: kind CLIENT, name {operation} {target}, always sanitize queries
Messaging producer: kind PRODUCER, name {destination} publish
Messaging consumer: kind CONSUMER, name {destination} process
See ${CLAUDE_SKILL_DIR}/references/semantic-conventions.md for required/recommended attributes per domain, status
mapping rules, and general conventions.
SDK Components
TracerProvider
Initialize once, early in application startup
Always call shutdown() on exit — flushes remaining spans
Configure Resource with service.name — identifies your service in backends
Use BatchSpanProcessor in production — SimpleSpanProcessor is for dev/testing only
See ${CLAUDE_SKILL_DIR}/references/sdk-components.md for Resource attributes, BatchSpanProcessor tuning, exporter
types, environment variable configuration, and Collector deployment patterns.
Application
When writing tracing code:
Apply all conventions silently — don't narrate each rule.
Use semantic conventions for attribute names. Check the reference for your domain (HTTP, DB, messaging) before
inventing names.
If an existing codebase contradicts a convention, follow the codebase and flag the divergence once.
When reviewing tracing code:
Cite the specific violation and show the fix inline.
