Notion Incident Runbook
Overview
Rapid incident response for Notion API failures. This runbook drives a sub-5-minute
triage that classifies the failure as Notion-side vs. integration-side, then routes to
per-error-type mitigation, cached fallback, and a structured postmortem.
Deep material — full triage scripts, remediation code, and templates — lives in
references/; this file is the high-level flow you follow live.
Prerequisites
- Access to application monitoring dashboards and log aggregator
NOTION_TOKEN environment variable set for diagnostic API callscurl and jq installed for quick CLI triage- Python alternative:
notion-client (pip install notion-client)
- Communication channels configured (Slack webhook, PagerDuty, etc.)
Instructions
Step 1: Quick Triage (Under 5 Minutes)
At first alert, decide whether the fault is Notion's or yours. Check the platform
status page, then test your own auth. This one call is enough to start:
curl -sf -o /dev/null -w "%{http_code}" \
https://api.notion.com/v1/users/me \
-H "Authorization: Bearer ${NOTION_TOKEN}" \
-H "Notion-Version: 2022-06-28"
Map the result: 200 → integration-side, 401 → token expired/revoked, 429 →
rate limited, 000 → network/DNS. If status.notion.so shows an active incident, it
is Notion-side regardless of your auth code.
Read references/triage.md for the full notion-triage.sh
diagnostic (status page + auth + DB-query latency + auto-classification) and a
TypeScript triageNotionHealth() equivalent for in-app health checks.
Step 2: Decision Tree and Mitigation
Route the classification to a remediation path:
- Notion-side outage — enable cached/fallback mode, notify users, monitor the
status page. Do NOT restart or rotate tokens.
- 401 token expired/revoked — regenerate at
notion.so/my-integrations, update the
secret manager, restart the app.
- 429 rate limited — you are exceeding the 3 req/s average; find runaway loops or
webhook storms, drop concurrency to 1, add exponential backoff.
- 404 on known resources — pages unshared or trashed; re-share via the Connections menu.
- 400 validation errors — the database schema changed in the UI; re-fetch with
databases.retrieve and update property mappings.
Read references/mitigation.md for the full decision tree,
token-rotation commands (AWS/GCP Secret Manager + kubectl restart), the
queryWithFallback() cached-fallback pattern, and detectSchemaChanges().
Step 3: Communication and Postmortem
Post an internal Slack update on every state change (INVESTIGATING → MITIGATING →
RESOLVED), an external status-page notice if users are impacted, and file a structured
postmortem once resolved. Copy-paste templates for all three are in
references/communication-and-postmortem.md.
Output
- Automated triage script classifying incidents in under 5 minutes
- Decision tree mapping HTTP status codes to root causes
- Per-error-type mitigation procedures with real code
- Cached fallback mode for Notion outages
- Schema change detection for 400 validation errors
- Communication templates for internal and external stakeholders
- Postmortem template with timeline and action items
Error Handling
| Scenario | Triage Signal | Immediate Action |
|---|
| Notion platform outage | status.notion.so incident | Enable fallback mode, notify users |
| Token expired/revoked | All requests return 401 | Rotate token in secret manager, restart |
| Rate limited | 429 errors spiking | Reduce concurrency to 1, check for loops |
| Schema changed | 400 on specific operations | Run databases.retrieve, update mappings |
| Network/DNS issue | Timeouts, no HTTP response | Check firewall, DNS resolution, proxy config |
| Pages unshared | 404 on previously working pages | Re-share via Connections menu in Notion |
Examples
One-Line Health Check
curl -sf https://api.notion.com/v1/users/me \
-H "Authorization: Bearer ${NOTION_TOKEN}" \
-H "Notion-Version: 2022-06-28" \
| jq '{name: .name, type: .type}' \
|| echo "UNHEALTHY: Notion API unreachable or auth failed"
Python Quick Triage
from notion_client import Client, APIResponseError
import os
def quick_triage():
try:
client = Client(auth=os.environ["NOTION_TOKEN"], timeout_ms=10_000)
me = client.users.me()
print(f"OK: Connected as {me['name']}")
except APIResponseError as e:
print(f"ERROR: {e.code} (HTTP {e.status}): {e.message}")
except Exception as e:
print(f"NETWORK ERROR: {e}")
quick_triage()
For the full notion-triage.sh diagnostic and the TypeScript in-app variant, see
references/triage.md.
Resources
