nexus-labs-automation/agent-observability

decision-tracing

Trace agent decision-making, tool selection, and reasoning chains

guardrails-safety

Instrument safety checks, content filters, and guardrails for agent outputs

production-eval-strategy

Strategies for evaluating agents in production - sampling, baselines, and regression detection

prompt-versioning

Track prompt versions, A/B test variants, and measure prompt performance

error-retry-tracking

Instrument error handling, retries, fallbacks, and failure patterns

evaluation-quality

Instrument evaluation metrics, quality scores, and feedback loops

human-in-the-loop

Instrument human approval workflows, feedback loops, and escalations

instrumentation-planning

Plan what to measure in AI agent systems using tiered approach

llm-call-tracing

Instrument LLM API calls with proper spans, tokens, and latency

memory-rag-instrumentation

Instrument RAG retrieval, memory operations, and context management

multi-agent-coordination

Instrument multi-agent workflows, handoffs, and parent-child relationships

session-conversation-tracking

Instrument sessions, conversations, and multi-turn interactions

token-cost-tracking

Track token usage and costs across agents for budget management

tool-call-tracking

Instrument agent tool executions with proper context and error handling