third-party-outage-response

name	third-party-outage-response
enabled	true
description	Use when performing third party outage response — response playbook for when a third-party vendor or external dependency experiences an outage. Covers impact assessment, customer communication, workaround activation, vendor status monitoring, internal coordination, and post-outage follow-up actions to minimize blast radius from upstream failures.
required_connections	[{"prefix":"slack","label":"Slack (for coordination)"}]
config_fields	[{"key":"vendor_name","label":"Vendor / Dependency Name","required":true,"placeholder":"e.g., Stripe, AWS S3, Twilio"},{"key":"vendor_status_url","label":"Vendor Status Page URL","required":false,"placeholder":"e.g., https://status.stripe.com"},{"key":"affected_services","label":"Our Affected Services","required":true,"placeholder":"e.g., payment processing, notification service"}]
features	["INCIDENT"]

Third-Party Outage Response Playbook

Vendor: {{ vendor_name }} | Status: {{ vendor_status_url }} Our Affected Services: {{ affected_services }}

Immediate Actions (0-10 min)

1. Confirm the Outage

• Check vendor status page: {{ vendor_status_url }}
• Check vendor's Twitter/X for announcements
• Verify the issue from our own monitoring (not just vendor self-report)
• Confirm it is a vendor issue, not our integration code
• Check community reports (e.g., Downdetector, Hacker News)

2. Assess Our Impact

• Which of our services depend on {{ vendor_name }}?
• What is the user-facing impact?
• Are there any workarounds or fallbacks available?
• How many users/customers are affected?
• Is there revenue impact?

3. Declare Incident (if warranted)

• Open incident channel
• Assign IC and roles
• Set severity based on OUR customer impact, not vendor severity

Impact Assessment Matrix

Our Service	Dependency Type	Fallback Available	User Impact	Severity
service	critical/degraded/optional	yes/no	description	SEV

Workaround Activation

Available Fallbacks

For each affected service, document available workarounds:

Service	Workaround	Activation Steps	Limitations
service	cache/queue/alternate provider/manual	steps	what is lost

Fallback Decision Criteria

• Activate automatic fallback if available and tested
• For manual fallbacks, IC must approve before activation
• Document when each fallback was activated and its limitations

Vendor Monitoring

Status Tracking

Monitor these sources every 15 minutes:

• Vendor status page: {{ vendor_status_url }}
• Vendor support ticket (if filed)
• Vendor Slack/community channel
• Our own health checks against the vendor

Vendor Communication

• File support ticket with vendor (reference case number: ___)
• Request ETA for resolution
• Ask for root cause information
• Document all vendor communications in incident channel

Customer Communication

Key Message Points

• Acknowledge the issue transparently
• State that a third-party provider is experiencing issues (name the vendor only if contractually permitted)
• Describe the impact on our service
• Share workarounds if available
• Provide next update time

Status Page Update Template

We are experiencing issues with {{ affected_services }} due to a
disruption at one of our infrastructure providers. Our team is
actively monitoring the situation and working on mitigations.

Impact: [describe customer impact]
Workaround: [if available]
Next update: [time]

Recovery Phase

When Vendor Recovers

• Confirm vendor reports resolution on their status page
• Verify our services are recovering (check metrics)
• Deactivate workarounds/fallbacks in controlled manner
• Verify data consistency (any queued/cached operations need replay?)
• Run integration tests against vendor
• Update status page to "Monitoring"
• Wait 30 minutes of stable operation before declaring resolved

Data Reconciliation

• Check for failed transactions that need retry
• Verify queued operations are processing
• Reconcile any data that used fallback paths
• Confirm webhook/callback backlog is draining

Post-Outage Follow-Up

Within 48 Hours

• Conduct internal post-incident review
• Request vendor's post-incident report
• Evaluate our dependency on {{ vendor_name }}

Longer Term

• Assess need for multi-vendor strategy
• Improve fallback mechanisms based on lessons learned
• Update SLAs/contracts with vendor if needed
• Add vendor outage scenario to chaos engineering drills
• Review circuit breaker and timeout configurations

Counter-Rationalizations

Shortcut	Counter	Why
"We can skip some steps for this case"	Adapt the workflow steps, don't skip them	Skipped steps are where incidents and oversights originate
"The user seems to already know what to do"	Complete all workflow phases with the user	The workflow catches blind spots that experience alone misses
"This is a minor case, full process is overkill"	Scale the process down, don't turn it off	Minor cases become major when unstructured; the process scales, not disappears
"I'll fill in the details later"	Complete each section before moving on	Deferred details are forgotten; real-time capture is more accurate
"The template output isn't necessary"	Always produce the structured output format	Structured output enables comparison, audit trails, and handoff to other teams