Implement distributed tracing using logs, including trace context propagation, span logging, correlation IDs, and OpenTelemetry integration for observability
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
Implement distributed tracing using logs, including trace context propagation, span logging, correlation IDs, and OpenTelemetry integration for observability
Implement distributed tracing using logs by propagating trace context, creating span logs, using correlation IDs, and integrating with OpenTelemetry standards to enable end-to-end request tracing across distributed systems.
When to use me
Use this skill when:
Building or maintaining distributed systems (microservices, serverless functions)
Need to trace requests across multiple service boundaries
Debugging issues that span multiple components or services
Implementing observability for complex workflows
Correlating logs from different services for a single user request
Setting up OpenTelemetry or other tracing standards
Analyzing latency and performance across service boundaries
Implementing request context propagation
Building audit trails for business transactions
What I do
1. Trace Context Propagation
Generate trace and span IDs for request initiation
Propagate context through HTTP headers across services
Maintain context through async operations (queues, background jobs, callbacks)
Handle context in batch processing and streaming systems
Implement context extraction and injection middleware
Manage sampling decisions for trace collection
2. Span Logging
Create span start/end logs with timing information
Log span attributes and events during execution
Capture parent-child relationships between spans
Record span status and errors for failed operations
Include business context in span logs
Implement span baggage for custom key-value propagation
3. Correlation & Context Management
Generate correlation IDs for business transactions
Link logs to traces through trace_id fields
Maintain user/session context across service boundaries
Propagate business identifiers (order_id, transaction_id, etc.)
Handle context in distributed transactions
Implement context storage and retrieval for long-running operations
4. OpenTelemetry Integration
Implement OpenTelemetry SDKs for various languages
{"timestamp":"2026-02-26T18:00:00Z","level":"ERROR","trace_id":"0af7651916cd43dd8448eb211c80319c","span_id":"b7ad6b7169203331","span_name":"process_payment","span_kind":"SERVER","event":"span_end","duration_ms":5123,"status":"ERROR","error_code":"PAYMENT_GATEWAY_TIMEOUT","error_message":"Payment gateway timeout after 5000ms","stack_trace":"...","attributes":{"order_id":"ord-789","retry_count":3,"gateway":"stripe"}}
from opentelemetry import trace
from opentelemetry.trace import Status, StatusCode
tracer = trace.get_tracer(__name__)
defprocess_payment(order_id, amount):
with tracer.start_as_current_span("process_payment") as span:
span.set_attribute("order_id", order_id)
span.set_attribute("amount", amount)
try:
# Business logic
result = charge_credit_card(order_id, amount)
span.set_status(Status(StatusCode.OK))
span.set_attribute("payment_id", result.payment_id)
return result
except Exception as e:
span.record_exception(e)
span.set_status(Status(StatusCode.ERROR, str(e)))
raise
Automatic Instrumentation
Configuration for automatic instrumentation of common frameworks:
opentelemetry:instrumentations:-name:"opentelemetry-instrumentation-flask"enabled:true-name:"opentelemetry-instrumentation-sqlalchemy"enabled:true-name:"opentelemetry-instrumentation-requests"enabled:truesampling:type:"parentbased_traceidratio"ratio:0.1# Sample 10% of traces in productionexporters:-type:"otlp"endpoint:"http://otel-collector:4317"-type:"logging"# Also log spans for local debuggingresource:attributes:service.name:"payment-service"service.version:"1.2.3"deployment.environment:"production"
Examples
# Generate trace context for new request
npm run tracing:generate-context -- --service payment-service --output context.json
# Propagate trace context through HTTP call
npm run tracing:propagate -- --trace-id abc123 --span-id def456 --target http://api.example.com
# Analyze trace from logs
npm run tracing:analyze -- --trace-id abc123 --sources "app.log,api.log,db.log" --output trace.json
# Set up OpenTelemetry instrumentation
npm run tracing:setup-otel -- --language nodejs --exporter jaeger --sampling-ratio 0.1
# Extract trace timeline from logs
npm run tracing:timeline -- --trace-id abc123 --output timeline.html
Output format
Trace Context Configuration:
tracing:standard:"W3C TraceContext"headers:traceparent:"traceparent"tracestate:"tracestate"correlation_id:"X-Correlation-Id"request_id:"X-Request-Id"propagation:http:truemessaging:truedatabase:truerpc:truesampling:strategy:"probability"rate:0.1# 10% sampling in productiondecision_deferred:falsespan_logging:enabled:trueformat:"json"include_fields:-trace_id-span_id-parent_span_id-span_name-span_kind-event-duration_ms-statusevents:-span_start-span_end-span_event-span_errorcorrelation:business_ids:-order_id-user_id-transaction_id-session_id