twilio-debugging-observability

## Overview Twilio provides several layers of debugging and observability: the Console Debugger for interactive troubleshooting, the Monitor REST API for programmatic alert queries, Event Streams for real-time error streaming, and status callbacks for per-resource delivery tracking. This skill covers the systematic approach to diagnosing issues and setting up production monitoring. --- ## Prerequisites - Twilio account with `TWILIO_ACCOUNT_SID` and `TWILIO_AUTH_TOKEN` -- see `twilio-iam-auth-setup` - SDK: `pip install twilio requests` / `npm install twilio` - For Event Streams: a publicly accessible HTTPS endpoint or AWS Kinesis stream --- ## Quickstart Check for recent errors on your account using the Monitor Alerts API. **Python** ```python import os from twilio.rest import Client client = Client(os.environ["TWILIO_ACCOUNT_SID"], os.environ["TWILIO_AUTH_TOKEN"]) alerts = client.monitor.alerts.list(log_level="error", limit=10) for alert in alerts: print(f"{alert.date_created}: [{alert.error_code}] {alert.alert_text}") ``` **Node.js** ```node const twilio = require("twilio"); const client = twilio(process.env.TWILIO_ACCOUNT_SID, process.env.TWILIO_AUTH_TOKEN); const alerts = await client.monitor.alerts.list({ logLevel: "error", limit: 10 }); alerts.forEach(a => { console.log(`${a.dateCreated}: [${a.errorCode}] ${a.alertText}`); }); ``` --- ## Key Patterns ### 1. Systematic Debugging Workflow When something fails, work through these layers in order: ``` 1. Check status callbacks FIRST (Did your endpoint receive delivery/call status? What error code?) | 2. Check the resource directly via REST API (GET /Messages/{sid} or /Calls/{sid} — current state + error_code) | 3. Check number reputation / sender registration (Is the number spam-flagged? Is A2P 10DLC registered? Toll-free verified?) | 4. Check the Console Debugger for webhook/TwiML errors (Console > Monitor > Errors — shows HTTP request/response details) | 5. Check your webhook endpoint (Is it reachable? Responding within 15s? Returning valid TwiML/200?) | 6. Query Monitor Alerts API or Event Streams (For patterns across many messages/calls, or historical analysis) ``` **Why status callbacks first:** Status callbacks tell you the exact error code for the specific message or call that failed. The Console Debugger aggregates errors across your account and may not surface the one you're looking for. Start specific, then broaden. **Number reputation checklist:** - SMS 30007 (carrier filtering) → Check A2P 10DLC registration status, content for spam triggers - SMS 30034 → Sender not registered for A2P 10DLC — register brand + campaign - Calls going to voicemail / "Spam Likely" → Check STIR/SHAKEN attestation, Voice Integrity status (see `twilio-numbers-senders`) - Toll-free SMS blocked → Check toll-free verification status **Rule of thumb:** If status callbacks show `delivered` but the user says they didn't receive it, the issue is on the carrier/device side (not Twilio). If the Console Debugger shows no errors at all, the problem is in your application (webhook, TwiML, business logic). ### 2. Console Debugger The [Console Debugger](https://console.twilio.com/us1/monitor/logs/debugger) shows errors and warnings for your account in real time. Each entry includes: - The exact error or warning that occurred - Potential causes and suggested solutions - The full HTTP request and response for the associated webhook **Configure a Debugger webhook** for real-time alerting: Console > Monitor > Logs > Debugger > (gear icon) > set Callback URL Debugger webhook POST parameters: | Parameter | Description | |---|---| | `Sid` | Debugger event identifier | | `AccountSid` | Account that generated the event | | `Level` | `Error` or `Warning` | | `Timestamp` | ISO 8601 time | | `Payload` | JSON: `resource_sid`, `error_code`, `more_info`, `webhook` (full request/response) | **Python (Flask) -- debugger webhook handler** ```python import json, os from flask import Flask, request from twilio.request_validator import RequestValidator app = Flask(__name__) validator = RequestValidator(os.environ["TWILIO_AUTH_TOKEN"]) @app.route("/debugger", methods=["POST"]) def debugger_event(): sig = request.headers.get("X-Twilio-Signature", "") if not validator.validate(request.url, request.form, sig): return "Forbidden", 403 level = request.form.get("Level") payload = json.loads(request.form.get("Payload", "{}")) error_code = payload.get("error_code") resource_sid = payload.get("resource_sid") msg = payload.get("more_info", {}).get("msg", "") print(f"[{level}] Error {error_code} on {resource_sid}: {msg}") return "", 204 ``` **Node.js (Express) -- debugger webhook handler** ```node const express = require("express"); const twilio = require("twilio"); const app = express(); app.use(express.urlencoded({ extended: false })); app.post("/debugger", (req, res) => { const valid = twilio.validateRequest( process.env.TWILIO_AUTH_TOKEN, req.headers["x-twilio-signature"], `https://${req.headers.host}${req.originalUrl}`, req.body ); if (!valid) return res.status(403).send("Forbidden"); const payload = JSON.parse(req.body.Payload || "{}"); const { error_code, resource_sid } = payload; const msg = payload.more_info?.msg || ""; console.log(`[${req.body.Level}] Error ${error_code} on ${resource_sid}: ${msg}`); res.sendStatus(204); }); ``` ### 3. Monitor Alerts API The Monitor REST API (`monitor.twilio.com/v1/Alerts`) provides programmatic access to error and warning logs. Individual alert instances include the full HTTP request and response data. **Python -- query alerts with date filtering** ```python from datetime import datetime, timedelta # Alerts from the last 24 hours start = datetime.utcnow() - timedelta(days=1) alerts = client.monitor.alerts.list( start_date=start, log_level="error", limit=50 ) for alert in alerts: print(f"{alert.date_created} [{alert.error_code}]") # Fetch full details including HTTP request/response detail = client.monitor.alerts(alert.sid).fetch() print(f" Request URL: {detail.request_url}") print(f" Response body: {detail.response_body}") ``` **Node.js -- query alerts with date filtering** ```node const startDate = new Date(Date.now() - 24 * 60 * 60 * 1000); const alerts = await client.monitor.alerts.list({ startDate, logLevel: "error", limit: 50, }); for (const alert of alerts) { console.log(`${alert.dateCreated} [${alert.errorCode}]`); const detail = await client.monitor.alerts(alert.sid).fetch(); console.log(` Request URL: ${detail.requestUrl}`); console.log(` Response body: ${detail.responseBody}`); } ``` **Retention:** Enterprise accounts: 13 months. Free accounts: 30 days. ### 4. Monitor Events API The Events resource (`monitor.twilio.com/v1/Events`) tracks all changes to Twilio resources -- phone number provisioning, account settings, recording access, API key creation, and more. **Python -- audit recent account changes** ```python events = client.monitor.events.list(limit=20) for event in events: print(f"{event.event_date}: {event.event_type}") print(f" Resource: {event.resource_type} ({event.resource_sid})") print(f" Actor: {event.actor_type} ({event.actor_sid}) from {event.source_ip_address}") ``` Each event captures: event type, resource, actor (who triggered it), source (API / Console / Twilio admin), and IP address. **Use cases:** - Audit who changed a phone number's webhook URL - Track API key creation and deletion - Detect unexpected configuration changes - Feed events into a SIEM for security monitoring ### 5. Event Streams for Error Log Streaming For production monitoring, stream errors to your infrastructure in real time using Event Streams. The Twilio SDK does not wrap Event Streams -- use `requests` / `fetch` directly. **Python -- set up error log streaming to a webhook** ```python import os, requests account_sid = os.environ["TWILIO_ACCOUNT_SID"] auth_token = os.environ["TWILIO_AUTH_TOKEN"] # Step 1: Create a webhook sink sink = requests.post( "https://events.twilio.com/v1/Sinks", auth=(account_sid, auth_token), data={ "Description": "Error monitoring sink", "SinkType": "webhook", "SinkConfiguration": '{"destination": "https://yourapp.com/twilio-errors", "method": "POST"}' } ).json() # Step 2: Subscribe to error log events subscription = requests.post( "https://events.twilio.com/v1/Subscriptions", auth=(account_sid, auth_token), data={ "Description": "Error log subscription", "SinkSid": sink["sid"], "Types": '[{"type": "com.twilio.error-logs.error.logged"}]' } ).json() print(f"Sink: {sink['sid']}, Subscription: {subscription['sid']}") ``` **Sink types:** `webhook`, `kinesis`, `segment` **Useful event types for observability:** | Event type | Description | |---|---| | `com.twilio.error-logs.error.logged` | All errors and warnings on account | | `com.twilio.messaging.message.delivered` | Message delivered successfully | | `com.twilio.messaging.message.undelivered` | Message delivery failed | | `com.twilio.voice.insights.call-summary` | Post-call quality and status summary | ### 6. Status Callback Monitoring Status callbacks are the most granular observability mechanism -- they fire for individual resource state changes. **Message delivery tracking:** ```python # Attach when sending message = client.messages.create( to="+15558675310", from_="+15017122661", body="Hello!", status_callback="https://yourapp.com/msg-status" ) ``` **Call lifecycle tracking:** ```python call = client.calls.create( to="+15558675310", from_="+15017122661", url="https://yourapp.com/voice", status_callback="https://yourapp.com/call-status", status_callback_event=["initiated", "ringing", "answered", "completed"] ) ``` **Recording completion tracking:** ```python # In TwiML response = VoiceResponse() response.record( recording_status_callback="https://yourapp.com/recording-status", recording_status_callback_event="completed absent failed" ) ``` Recording status values: `in-progress`, `completed`, `absent`, `failed`. The `RecordingUrl` is available when status is `completed`. ### 7. Debugging Webhooks When Twilio can't reach your webhook or receives an error, the problem is often in your infrastructure. **Common causes and fixes:** | Symptom | Likely cause | Fix | |---|---|---| | Error 11200 in Debugger | Webhook URL returned non-200 / unreachable | Verify endpoint is live: `curl -I https://yourapp.com/sms` | | Error 11205 | HTTP connection failure (port closed, refused, firewall) | Verify server is running and port is open: `curl -I https://yourapp.com/sms` | | Error 12100 | TwiML document could not be parsed | Check for debug output, BOM characters, or malformed XML | | Parameters missing after redirect | HTTP 301/302 strips POST body | Fix URL to avoid redirect (add/remove `www.`, use HTTPS directly) | | Webhook works locally but not deployed | Tunnel expired or firewall | Use `curl` from an external host to test | | Intermittent failures | ngrok session expired / recycled | Deploy to a stable host for anything beyond quick tests | **Test webhooks manually:** ```bash # Simulate an inbound SMS webhook curl -X POST https://yourapp.com/sms \ -d "From=+15551234567" \ -d "To=+15559876543" \ -d "Body=Test message" \ -d "MessageSid=SM00000000000000000000000000000000" ``` **Browser testing:** Visit your webhook URL in Firefox -- it highlights XML errors in the response. ### 8. Common Error Codes | Code | Name | Cause | Fix | |---|---|---|---| | 11200 | HTTP retrieval failure | Twilio cannot reach your webhook URL | Check URL, DNS, firewall, SSL cert | | 11205 | HTTP connection failure | Webhook endpoint refused connection | Verify server is running and port is open | | 11751 | Media download failure | MMS media URL unreachable | Check media URL accessibility | | 12100 | Document parse failure | TwiML is not valid XML | Validate XML; remove debug output | | 12200 | Schema compliance failure | TwiML verbs/attributes are invalid | Check TwiML reference for correct syntax | | 20003 | Authentication error | Invalid Account SID or Auth Token | Verify credentials in environment | | 21211 | Invalid `To` number | Number not in E.164 format | Use `+` country code + number | | 21608 | Unverified number (trial) | Trial accounts can only send to verified numbers | Verify number or upgrade account |