| name | agent-debugging |
| description | Debug and troubleshoot ElevenLabs conversational AI agents and Twilio calls. Use when diagnosing agent issues, analyzing failed calls, troubleshooting audio problems, investigating conversation breakdowns, reviewing error logs, or optimizing underperforming agents. Includes transcript analysis, error diagnosis, and performance troubleshooting. |
| allowed-tools | Read, Write, Edit, Bash, Glob, Grep |
Agent Debugging Skill
Systematic debugging and troubleshooting framework for ElevenLabs Conversational AI + Twilio integration on the Next Level Real Estate platform. This skill provides diagnostic procedures, root cause analysis techniques, and resolution strategies.
When to Use This Skill
Invoke this skill when you need to:
- ✅ Debug agent configuration issues
- ✅ Analyze failed or poor-quality calls
- ✅ Troubleshoot audio/latency problems
- ✅ Investigate conversation breakdowns
- ✅ Review and interpret error logs
- ✅ Diagnose context injection failures
- ✅ Identify performance bottlenecks
- ✅ Generate diagnostic reports
Debugging Methodology
The 5-Step Debug Process
1. OBSERVE → Gather symptoms and error data
2. REPRODUCE → Confirm issue is consistent
3. ISOLATE → Narrow down to specific component
4. DIAGNOSE → Identify root cause
5. FIX → Implement and verify solution
Common Issues & Solutions
Issue Category 1: Agent Configuration
Problem: Agent Not Starting/Responding
Symptoms:
- Agent status shows "inactive" or "error"
- Conversation fails to start
- No greeting heard on call
- Error: "Agent not found" or "Invalid agent ID"
Diagnostic Steps:
Use mcp__elevenlabs__elevenlabs_get_agent with:
{
"agentId": "your_agent_id"
}
Use mcp__elevenlabs__elevenlabs_get_voice with:
{
"voiceId": "voice_from_agent_config"
}
Read .claude/agents/elevenlabs-agent-manager.md
Common Root Causes:
| Root Cause | Symptoms | Solution |
|---|
| Invalid voice ID | Agent fails to start | Use list_voices to find valid ID |
| Voice not available for TTS | "Voice not supported" error | Select voice with availableForTts: true |
| Empty system prompt | Generic/confused responses | Add detailed system prompt |
| Wrong model ID | "Model not found" error | Use: eleven_flash_v2_5, eleven_turbo_v2_5, or eleven_multilingual_v2 |
| Agent not deployed | "Agent not found" | Re-create agent or verify agent ID |
Fix Template:
const agentConfig = {
name: "Fixed Lead Qualifier",
voiceId: "21m00Tcm4TlvDq8ikWAM",
modelId: "eleven_flash_v2_5",
greeting: "Hi, this is Sarah from Next Level Real Estate.",
systemPrompt: `[Complete, detailed prompt here]`,
tcpaCompliance: true,
recordingConsent: true
}
const agent = await createAgent(agentConfig)
console.log(`Agent created: ${agent.agentId}`)
Problem: Agent Gives Incorrect/Off-Topic Responses
Symptoms:
- Agent doesn't follow system prompt instructions
- Responses are generic or unrelated
- Agent doesn't use injected context
- Agent goes off-topic frequently
Diagnostic Steps:
Use mcp__elevenlabs__elevenlabs_get_agent to retrieve full config
Use mcp__elevenlabs__elevenlabs_get_conversation with conversationId
Root Cause Analysis:
## System Prompt Issues
### Too Vague
❌ "You are helpful"
✅ "You are Sarah, a wholesale real estate buyer. Qualify leads by asking about motivation, timeline, and property condition."
### Missing Constraints
❌ No "WHAT NOT TO DO" section
✅ Include explicit constraints: "DON'T discuss pricing until property details gathered"
### No Examples
❌ Abstract instructions only
✅ Include example conversations showing desired behavior
### Context Not Referenced
❌ Prompt doesn't mention lead data
✅ "You will receive lead data including name, property address, and motivation. Use these naturally."
Fix:
await updateAgent(agentId, {
systemPrompt: `
You are Sarah, a professional real estate investor representative for Next Level Real Estate.
OBJECTIVE:
Qualify motivated sellers in under 3 minutes by gathering property details, assessing motivation, and scheduling viewings.
CONTEXT PROVIDED:
- Lead name and contact info
- Property address
- Estimated value
- Source of inquiry
USE THIS CONTEXT NATURALLY:
"Hi [name], I'm following up on your inquiry about [address]..."
KEY QUESTIONS TO ASK:
1. "What's prompting you to consider selling?"
2. "How soon are you looking to move?"
3. "Can you describe the property's current condition?"
CONVERSATION GUIDELINES:
- Be empathetic and professional
- Listen more than you talk
- Keep responses under 2 sentences
- Build rapport before asking detailed questions
WHAT NOT TO DO:
- Don't pressure or use aggressive tactics
- Don't discuss specific offers without property viewing
- Don't ask about price until property details gathered
- Don't use jargon unless lead uses it first
SUCCESS:
Lead qualified with viewing scheduled within 48 hours.
`
})
Issue Category 2: Audio & Call Quality
Problem: Poor Audio Quality
Symptoms:
- Choppy, robotic voice
- Echo or feedback
- Audio dropouts or silence
- Laggy responses (>500ms)
Diagnostic Steps:
Use mcp__elevenlabs__elevenlabs_get_conversation with conversationId
Grep logs/calling-service.log for:
- WebSocket connection errors
- Audio buffer underruns
- Network timeout errors
Root Cause Analysis:
| Symptom | Likely Cause | Quick Test |
|---|
| Robotic voice | Wrong model or voice settings | Try different voice ID |
| High latency | Model too slow or overloaded | Switch to Flash 2.5 |
| Dropouts | Network issues | Test network bandwidth |
| Echo | Audio feedback loop | Check Twilio echo cancellation |
| Choppy | Buffer issues | Adjust jitter buffer size |
Diagnostic Script:
async function diagnoseAudioQuality(conversationId: string) {
const conversation = await getConversation(conversationId)
const responseLatencies = []
for (let i = 1; i < conversation.transcript.length; i++) {
const userTurn = conversation.transcript[i - 1]
const agentTurn = conversation.transcript[i]
if (userTurn.role === 'user' && agentTurn.role === 'agent') {
const latency = new Date(agentTurn.timestamp).getTime() -
new Date(userTurn.timestamp).getTime()
responseLatencies.push(latency)
}
}
const avgLatency = average(responseLatencies)
const maxLatency = Math.max(...responseLatencies)
const transcriptText = conversation.transcript
.map(t => t.text.toLowerCase())
.join(' ')
const clarificationRequests = (transcriptText.match(/what\?|pardon|repeat|didn't hear/g) || []).length
const incompleteResponses = conversation.transcript.filter(t =>
t.text.endsWith('...') || t.text.length < 5
).length
const diagnosis = {
avgLatency,
maxLatency,
clarificationRequests,
incompleteResponses,
quality: avgLatency < 200 ? 'excellent' :
avgLatency < 400 ? 'good' :
avgLatency < 600 ? 'acceptable' : 'poor',
issues: []
}
if (avgLatency > 400) {
diagnosis.issues.push('High average latency - consider switching to Flash 2.5 model')
}
if (maxLatency > 1000) {
diagnosis.issues.push('Extreme latency spikes detected - check network stability')
}
if (clarificationRequests > 2) {
diagnosis.issues.push('Multiple clarification requests - audio clarity issue')
}
if (incompleteResponses > 1) {
diagnosis.issues.push('Incomplete responses - possible audio dropouts')
}
return diagnosis
}
Fixes:
await updateAgent(agentId, {
modelId: "eleven_flash_v2_5",
responseLatency: 75
})
await updateAgent(agentId, {
voiceSettings: {
stability: 0.5,
similarityBoost: 0.75,
useSpeakerBoost: true
}
})
{
codec: 'opus',
jitterBufferSize: 'small',
echoCancellation: true
}
Problem: Awkward Pauses or Interruptions
Symptoms:
- Agent waits too long to respond
- Agent interrupts user mid-sentence
- Conversation feels unnatural
- Multiple false starts
Diagnostic Steps:
Use mcp__elevenlabs__elevenlabs_get_conversation
Use mcp__elevenlabs__elevenlabs_get_agent
Fix:
await updateAgent(agentId, {
interruptionSensitivity: "high",
responseLatency: 75
})
Issue Category 3: Context & Integration
Problem: Agent Not Using Lead Context
Symptoms:
- Agent doesn't mention lead's name
- Doesn't reference property address
- Ignores motivation or timeline info
- Generic, impersonal responses
Diagnostic Steps:
Grep logs/calling-service.log for "context injection"
Use mcp__elevenlabs__elevenlabs_get_conversation
Root Cause Analysis:
await startConversation({ agentId: "abc123" })
await startConversation({
agentId: "abc123",
context: "John Smith, 123 Main St"
})
await startConversation({
agentId: "abc123",
context: {
lead: {
personal: {
name: "John"
}
}
}
})
await startConversation({
agentId: "abc123",
context: {
leadData: {
name: "John Smith",
phone: "+1234567890",
motivation: "probate",
timeline: "urgent"
},
propertyInfo: {
address: "123 Main St, Austin TX",
estimatedValue: 250000,
condition: "needs_repairs"
}
}
})
Fix:
const context = {
leadData: {
name: leadName,
phone: leadPhone,
motivation: leadMotivation,
timeline: leadTimeline
},
propertyInfo: {
address: propertyAddress,
estimatedValue: propertyValue,
condition: propertyCondition
},
strategyRules: {
minEquity: 0.20,
targetARV: targetARV
}
}
await updateAgent(agentId, {
systemPrompt: `
...existing prompt...
CONTEXT PROVIDED TO YOU:
You will receive information about the lead and property:
- leadData.name: The person's name
- leadData.motivation: Why they're selling (use this to show empathy)
- leadData.timeline: How soon they need to sell
- propertyInfo.address: The property location
- propertyInfo.condition: Property state
USE THIS INFORMATION NATURALLY:
- Start with: "Hi [leadData.name], I'm calling about [propertyInfo.address]"
- Reference motivation: "I understand you're dealing with [leadData.motivation]"
- Adapt urgency based on timeline
EXAMPLE:
"Hi John, this is Sarah from Next Level Real Estate. I saw you inquired about your property at 123 Main St. I understand you're dealing with a probate situation and need to move quickly. I'd love to help..."
`
})
const testCall = await startConversation({
agentId: agentId,
context: context
})
console.log('Context sent:', JSON.stringify(context, null, 2))
const conversation = await getConversation(testCall.conversationId)
const transcript = conversation.transcript.map(t => t.text).join(' ')
console.log('Name used:', transcript.includes(context.leadData.name))
console.log('Address mentioned:', transcript.includes(context.propertyInfo.address))
Problem: Twilio Call Connection Failures
Symptoms:
- Call status: "failed" or "busy"
- No audio on either end
- Call drops immediately
- Webhook not called
Diagnostic Steps:
curl -X POST https://your-domain.com/twiml/test-123
Grep logs/calling-service.log for:
- "Twilio error"
- "Webhook failed"
- "Connection refused"
Common Error Codes:
| Error Code | Meaning | Solution |
|---|
| 11200 | HTTP retrieval failure | Check webhook URL accessibility |
| 11205 | HTTP connection failure | Verify server is running |
| 11206 | Connection refused | Check firewall rules |
| 12100 | Document parse failure | Validate TwiML XML syntax |
| 13224 | Invalid phone number | Verify "to" number format |
| 13225 | Invalid callback URL | Check webhook URL format |
| 21211 | Invalid 'To' phone number | Verify number is E.164 format |
Fix:
function validatePhoneNumber(phone: string): boolean {
const e164Regex = /^\+[1-9]\d{1,14}$/
return e164Regex.test(phone)
}
router.post('/twiml/:conversationId', (req, res) => {
try {
const { conversationId } = req.params
const twiml = new VoiceResponse()
twiml.say({
voice: 'alice',
language: 'en-US'
}, 'This call may be recorded.')
const connect = twiml.connect()
connect.stream({
url: `wss://api.elevenlabs.io/v1/convai/${conversationId}/stream`
})
res.type('text/xml')
res.send(twiml.toString())
} catch (error) {
console.error('TwiML generation error:', error)
const errorTwiml = new VoiceResponse()
errorTwiml.say('An error occurred. Please try again later.')
errorTwiml.hangup()
res.type('text/xml')
res.send(errorTwiml.toString())
}
})
async function initiateCallWithRetry(config, maxRetries = 3) {
for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
const call = await twilioClient.initiateCall(config)
console.log(`Call initiated successfully on attempt ${attempt}`)
return call
} catch (error) {
console.error(`Call attempt ${attempt} failed:`, error.message)
if (attempt === maxRetries) {
throw error
}
await sleep(1000 * Math.pow(2, attempt - 1))
}
}
}
async function testWebhook() {
const response = await fetch('https://your-domain.com/twiml/test', {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: 'From=%2B1234567890&To=%2B0987654321'
})
console.log('Webhook status:', response.status)
console.log('Webhook body:', await response.text())
if (response.status !== 200) {
throw new Error(`Webhook test failed with status ${response.status}`)
}
}
Issue Category 4: Performance & Reliability
Problem: High Call Failure Rate
Symptoms:
-
10% of calls fail
- Inconsistent connect rates
- Random disconnections
- System errors under load
Diagnostic Steps:
Use mcp__elevenlabs__elevenlabs_list_conversations with:
{
"status": "failed"
}
Grep logs/*.log for:
- "Out of memory"
- "Connection timeout"
- "Rate limit exceeded"
- "Service unavailable"
Root Cause Analysis:
async function analyzeFailures(timeRange: { start: Date, end: Date }) {
const conversations = await listConversations(timeRange)
const failures = conversations.filter(c => c.status === 'failed')
const errorTypes = {}
failures.forEach(f => {
const error = f.error || 'unknown'
errorTypes[error] = (errorTypes[error] || 0) + 1
})
const hourlyFailures = new Array(24).fill(0)
failures.forEach(f => {
const hour = new Date(f.startedAt).getHours()
hourlyFailures[hour]++
})
const agentFailures = {}
failures.forEach(f => {
agentFailures[f.agentId] = (agentFailures[f.agentId] || 0) + 1
})
return {
totalFailures: failures.length,
failureRate: (failures.length / conversations.length) * 100,
errorTypes,
hourlyFailures,
agentFailures,
recommendations: generateRecommendations(errorTypes, hourlyFailures, agentFailures)
}
}
function generateRecommendations(errors, hourly, agents) {
const recommendations = []
if (errors['Rate limit exceeded'] > 10) {
recommendations.push('Implement rate limiting and request throttling')
}
if (errors['Connection timeout'] > 5) {
recommendations.push('Increase timeout settings and check network stability')
}
if (errors['Agent not found'] > 0) {
recommendations.push('Validate agent IDs before starting conversations')
}
const peakHour = hourly.indexOf(Math.max(...hourly))
if (hourly[peakHour] > 10) {
recommendations.push(`High failures at hour ${peakHour}:00 - consider load balancing`)
}
const problematicAgent = Object.entries(agents)
.sort(([,a], [,b]) => b - a)[0]
if (problematicAgent && problematicAgent[1] > 5) {
recommendations.push(`Agent ${problematicAgent[0]} has high failure rate - review configuration`)
}
return recommendations
}
Fix:
import rateLimit from 'express-rate-limit'
const callLimiter = rateLimit({
windowMs: 60 * 1000,
max: 100,
message: 'Too many calls, please try again later'
})
app.use('/api/calls', callLimiter)
class CircuitBreaker {
private failureCount = 0
private lastFailureTime = 0
private state = 'closed'
async execute(fn: Function) {
if (this.state === 'open') {
if (Date.now() - this.lastFailureTime > 60000) {
this.state = 'half-open'
} else {
throw new Error('Circuit breaker is open')
}
}
try {
const result = await fn()
if (this.state === 'half-open') {
this.state = 'closed'
this.failureCount = 0
}
return result
} catch (error) {
this.failureCount++
this.lastFailureTime = Date.now()
if (this.failureCount >= 5) {
this.state = 'open'
console.error('Circuit breaker opened due to failures')
}
throw error
}
}
}
async function healthCheck() {
const checks = {
elevenlabs: false,
twilio: false,
database: false
}
try {
await listVoices()
checks.elevenlabs = true
} catch (error) {
console.error('ElevenLabs health check failed:', error)
}
try {
await twilioClient.getCallStatus('test')
checks.twilio = true
} catch (error) {
console.error('Twilio health check failed:', error)
}
try {
await database.ping()
checks.database = true
} catch (error) {
console.error('Database health check failed:', error)
}
return {
healthy: Object.values(checks).every(v => v),
checks
}
}
setInterval(healthCheck, 5 * 60 * 1000)
async function retryWithBackoff(fn: Function, maxRetries = 3) {
for (let attempt = 1; attempt <= maxRetries; attempt++) {
try {
return await fn()
} catch (error) {
if (attempt === maxRetries) throw error
const delay = Math.min(1000 * Math.pow(2, attempt), 10000)
console.log(`Retry attempt ${attempt} failed, waiting ${delay}ms`)
await sleep(delay)
}
}
}
Logging & Monitoring
Comprehensive Logging Setup
import winston from 'winston'
export const logger = winston.createLogger({
level: process.env.LOG_LEVEL || 'info',
format: winston.format.combine(
winston.format.timestamp(),
winston.format.errors({ stack: true }),
winston.format.json()
),
transports: [
new winston.transports.File({
filename: 'logs/error.log',
level: 'error'
}),
new winston.transports.File({
filename: 'logs/combined.log'
}),
new winston.transports.Console({
format: winston.format.combine(
winston.format.colorize(),
winston.format.simple()
)
})
]
})
export function createCallLogger(conversationId: string) {
return logger.child({ conversationId })
}
const callLogger = createCallLogger(conversationId)
callLogger.info('Call initiated', {
to: phoneNumber,
agentId: agentId
})
callLogger.debug('Context injected', {
leadData: leadData
})
callLogger.error('Call failed', {
error: error.message,
stack: error.stack
})
Monitoring Dashboard Queries
Grep logs/error.log "$(date -d '1 hour ago' '+%Y-%m-%d %H')"
Grep logs/combined.log "duration" | awk '$NF > 300'
Grep logs/combined.log "Call failed" |
grep -oP 'agentId":"[^"]+' |
sort | uniq -c
Grep logs/error.log "error" |
grep -oP '"message":"[^"]+' |
sort | uniq -c
Diagnostic Report Template
# Agent Debugging Report
## Issue Summary
**Date:** [Date of issue]
**Agent ID:** [Agent identifier]
**Conversation ID(s):** [Affected conversations]
**Severity:** [Critical / High / Medium / Low]
## Symptoms Observed
- [Symptom 1]
- [Symptom 2]
- [Symptom 3]
## Diagnostic Steps Taken
1. [Step 1 with results]
2. [Step 2 with results]
3. [Step 3 with results]
## Root Cause Analysis
**Primary Cause:** [Main issue identified]
**Contributing Factors:**
- [Factor 1]
- [Factor 2]
**Evidence:**
- [Log excerpt or transcript snippet]
- [Metric data]
## Solution Implemented
**Changes Made:**
```typescript
// Configuration changes
Verification:
- [How you verified the fix]
- [Test results]
Prevention Measures
- [Step to prevent recurrence]
- [Monitoring to add]
- [Documentation to update]
Follow-up Actions
## Troubleshooting Checklist
```markdown
## Pre-Flight Checklist
Before debugging, verify basics:
### Configuration
- [ ] Agent exists and status is "active"
- [ ] Voice ID is valid and available for TTS
- [ ] Model ID is correct (flash/turbo/multilingual)
- [ ] System prompt is non-empty and well-formed
- [ ] TCPA compliance flags are enabled
### Infrastructure
- [ ] Webhook endpoint is accessible (200 OK)
- [ ] Twilio account has sufficient balance
- [ ] Phone number is active and voice-enabled
- [ ] ElevenLabs API key is valid
- [ ] Environment variables are set correctly
### Network
- [ ] Internet connectivity is stable
- [ ] No firewall blocking WebSocket connections
- [ ] DNS resolution working
- [ ] Sufficient bandwidth available
### System
- [ ] Server/application is running
- [ ] No resource exhaustion (CPU, memory, disk)
- [ ] Logs are being written
- [ ] Database is accessible
Resources
Log Locations
Useful Commands
tail -f logs/calling-service.log
Grep logs/combined.log "conversation_abc123"
Grep logs/error.log "error" | cut -d'"' -f4 | sort | uniq -c
Grep logs/combined.log "latency" | awk '$NF > 500'
External Resources
Remember: Systematic debugging saves time. Follow the 5-step process: Observe, Reproduce, Isolate, Diagnose, Fix. Document everything for future reference.