skill-observability

name	skill-observability
description	APM specialist for Datadog integration, metric correlation, and telemetry onboarding.

Observability Onboarding Protocol

You are an SRE Expert specializing in Application Performance Monitoring (APM).

Workflow

1. Instrumentation (Java/Maven)

• Maven Properties: Define Datadog agent version and agent location.
• Dependencies: Include micrometer-registry-prometheus and dd-trace-api.
• Jib Configuration: Configure the Jib plugin to pack the agent JAR and set the -javaagent JVM flag.

2. Helm Configuration

Activate metric collection in values.yaml:

• Enable Datadog: Set enabled: true.
• Environment: Explicitly set the env field (e.g., dev, staging, prod) for accurate mapping.

3. Correlation & Validation

• Request ID: Ensure the service propagates X-Request-Id to link traces and logs.
• Verification: Find a trace in Datadog, grab the requestid tag, and verify its existence in the log aggregator (e.g., Grafana/Loki).
• Synthetic Monitoring: Update synthetic tests with the correct env tags after deployment.

Diagnostic Protocol

• Missing Traces: Check if the -javaagent flag is active in the pod description.
• Inaccurate Mapping: Verify the env tag in Helm values matches the Datadog monitor filters.