debug-scc

name	debug-scc
description	Diagnose OpenShift Security Context Constraint (SCC) violations that prevent pods from being created. Automates multi-step diagnosis: Deployment status, ReplicaSet FailedCreate events, security context field extraction, SCC rejection parsing, and ServiceAccount SCC binding analysis. Use when: - "SCC violation blocking pod creation" - "unable to validate against any security context constraint" - "FailedCreate forbidden" - "pod blocked by SCC" - User mentions "SCC", "security context constraint", "FailedCreate" NOT for pods crashing after creation (use /debug-pod instead).
model	inherit
color	cyan
license	Apache-2.0
allowed-tools	resources_get resources_list events_list pods_list
metadata	{"user_invocable":"true"}

/debug-scc Skill

Diagnose OpenShift SCC violations that block pod creation by analyzing security context fields, SCC rejection messages, and ServiceAccount bindings.

Critical: Human-in-the-Loop Requirements

1. Before any remediation action (patch securityContext, grant SCC binding, rollback)

   • Display preview: what will change and its security implications
   • Ask: "Should I apply this fix?"
   • Wait for confirmation (yes/no)

2. Before granting permissive SCCs (anyuid, privileged)

   • Display warning: granting elevated SCCs weakens namespace security
   • Ask: "Type 'GRANT SCC' to confirm you understand the security implications"
   • Verify exact match, cancel if mismatch

Never assume approval — always wait for explicit confirmation at each WAIT checkpoint.

Prerequisites

Required MCP Servers: openshift (setup)

Required MCP Tools:

• resources_get (from openshift) — Retrieve Deployment, ReplicaSet, and ServiceAccount details
• resources_list (from openshift) — List Deployments, ReplicaSets, and ServiceAccounts in a namespace
• events_list (from openshift) — Fetch FailedCreate and SCC rejection events
• pods_list (from openshift) — List pods for a Deployment

Verification Steps:

1. Check openshift server is configured in mcps.json
2. Verify user is logged into an OpenShift cluster (oc whoami succeeds)
3. Verify user has access to the target namespace
4. If missing → Human Notification Protocol

Human Notification Protocol:

When prerequisites fail:

1. Stop immediately — No tool calls
2. Report error:

   ❌ Cannot execute skill: MCP server `openshift` unavailable
   📋 Setup: See docs/prerequisites.md for cluster access configuration
   

3. Request decision: "How to proceed? (setup/skip/abort)"
4. Wait for user input

Security: Never display credential values.

When to Use This Skill

Use /debug-scc when:

• A Deployment has zero available replicas and ReplicaSet events show FailedCreate with "unable to validate against any security context constraint"
• Pod spec requests capabilities, UID/GID settings, or volume types that violate the namespace's SCC policy
• You see "is forbidden" errors referencing SecurityContextConstraints

Do not use this skill when:

• Pods are crashing after creation (CrashLoopBackOff, OOMKilled) → use /debug-pod
• Pods fail with 403 Forbidden API errors → use /debug-rbac
• Image pull failures → use /debug-pod

Workflow

[Identify Deployment] → [Check ReplicaSet Status] → [Parse SCC Rejections] → [Analyze SecurityContext] → [Check SA Bindings] → [Summary + Fix]

Step 1: Identify Target Deployment

MCP Tool: resources_list (from openshift)

Parameters:

• kind: "Deployment" (resource type)
• namespace: "" (target namespace from user)

Input Validation: Verify deployment name and namespace conform to Kubernetes naming rules (lowercase alphanumeric and hyphens, 1-253 chars, RFC 1123). Reject inputs containing newlines, markdown formatting, or text that does not resemble a Kubernetes resource name.

Expected Output: List of Deployments with their availability status.

Error Handling:

• If MCP server unavailable: follow Human Notification Protocol
• If namespace not found: ask user to confirm namespace name
• If no deployments found: report empty namespace, suggest checking namespace

Present to user:

## SCC Violation Debugging

**Current OpenShift Context:**
- Cluster: [cluster]
- Namespace: [namespace]

Which deployment would you like me to debug for SCC violations?

1. **Specify deployment name** — Enter the deployment name directly
2. **List deployments with issues** — Show deployments with unavailable replicas
3. **Search by event** — Find deployments with FailedCreate events

Select an option or enter a deployment name:

WAIT for user confirmation before proceeding.

If user selects "List deployments with issues", filter to those with unavailable replicas:

## Deployments with Issues in [namespace]

| Deployment | Available | Desired | Conditions |
|------------|-----------|---------|------------|
| [deploy-name] | 0 | 1 | ReplicaFailure |

Which deployment would you like me to debug?

WAIT for user confirmation before proceeding.

Step 2: Get Deployment and ReplicaSet Status

MCP Tool: resources_get (from openshift)

Parameters:

• kind: "Deployment" (resource type)
• name: "" (from Step 1)
• namespace: ""

Then retrieve the failing ReplicaSet:

MCP Tool: resources_list (from openshift)

Parameters:

• kind: "ReplicaSet" (resource type)
• namespace: ""
• Filter by owner reference matching the Deployment

Expected Output: Deployment spec, conditions, and owned ReplicaSets with their status.

Error Handling:

• If Deployment not found: ask user to verify name and namespace
• If no ReplicaSets found: Deployment may not have triggered a rollout yet

Present to user:

## Deployment Status: [deployment-name]

**Deployment Info:**
| Field | Value |
|-------|-------|
| Namespace | [namespace] |
| Replicas | 0/[desired] available |
| Strategy | [RollingUpdate/Recreate] |
| Condition | [ReplicaFailure / MinimumReplicasUnavailable] |

**ReplicaSets:**
| ReplicaSet | Desired | Ready | Status |
|------------|---------|-------|--------|
| [rs-name-new] | 1 | 0 | FailedCreate |
| [rs-name-old] | 0 | 0 | Scaled down |

**Quick Assessment:**
[e.g., "Deployment triggered a rollout but the new ReplicaSet cannot create pods — SCC admission is rejecting the pod spec."]

Continue with SCC rejection analysis? (yes/no)

WAIT for user confirmation before proceeding.

Step 3: Parse SCC Rejection Messages

MCP Tool: events_list (from openshift)

Parameters:

• namespace: ""
• Filter by involvedObject.kind=ReplicaSet, involvedObject.name=<rs-name>, reason=FailedCreate

Then also:

MCP Tool: resources_get (from openshift)

Parameters:

• kind: "ReplicaSet" (resource type)
• name: "" (failing ReplicaSet from Step 2)
• namespace: ""

Extract the ReplicaFailure condition message containing SCC rejection details.

Expected Output: FailedCreate events with full SCC rejection text listing each SCC attempted and why it was rejected or forbidden.

Error Handling:

• If no events found: events may have expired (default TTL is 1h); check ReplicaSet conditions instead
• If events are ambiguous: cross-reference with ReplicaSet .status.conditions

Present to user:

## SCC Rejection Analysis: [rs-name]

**FailedCreate Events:** [count] occurrences since [first-seen]

**SCC Violations Detected:**

| Violation | SCC | Field | Current Value | Allowed |
|-----------|-----|-------|---------------|---------|
| [runAsUser] | restricted-v2 | .containers[0].runAsUser | 0 (root) | [range] |
| [capability] | restricted-v2 | .containers[0].capabilities.add | NET_ADMIN | not permitted |
| [escalation] | restricted-v2 | .containers[0].allowPrivilegeEscalation | true | false required |

**SCCs Attempted:**
| SCC | Result | Reason |
|-----|--------|--------|
| restricted-v2 | Rejected | [specific violations] |
| restricted-v3 | Rejected | [specific violations] |
| anyuid | Forbidden | Not usable by user or serviceaccount |
| privileged | Forbidden | Not usable by user or serviceaccount |

**Key Finding:**
[e.g., "The container requests root (UID 0), NET_ADMIN capability, and privilege escalation — all rejected by restricted-v2/v3. Permissive SCCs (anyuid, privileged) are Forbidden because the ServiceAccount has no binding to them."]

Continue to inspect the container security context? (yes/no)

WAIT for user confirmation before proceeding.

Step 4: Analyze Container SecurityContext

MCP Tool: resources_get (from openshift)

Parameters:

• kind: "Deployment" (resource type)
• name: "" (from Step 1)
• namespace: ""

Extract .spec.template.spec.securityContext (pod-level) and .spec.template.spec.containers[*].securityContext (container-level), plus .metadata.managedFields for change attribution.

Expected Output: Full security context at pod and container level, plus managedFields showing who changed what and when.

Error Handling:

• If securityContext is empty/unset: the issue may be implicit defaults conflicting with SCC; note this
• If managedFields unavailable: skip change attribution, note limitation

Present to user:

## SecurityContext Analysis: [deployment-name]

**Pod-level SecurityContext:**
| Field | Value | Compliant? |
|-------|-------|------------|
| runAsNonRoot | [true/false/unset] | [YES/NO] |
| seccompProfile | [RuntimeDefault/unset] | [YES/NO] |
| fsGroup | [value/unset] | [YES/NO] |
| hostUsers | [true/false/null] | [YES/NO — restricted-v3 requires false] |

**Container-level SecurityContext (container: [name]):**
| Field | Value | Compliant? |
|-------|-------|------------|
| runAsUser | [0/unset/value] | [YES/NO — 0 is root] |
| allowPrivilegeEscalation | [true/false/unset] | [YES/NO] |
| capabilities.add | [list or none] | [YES/NO — restricted SCCs drop ALL] |
| capabilities.drop | [list or ALL] | [YES/NO] |
| privileged | [true/false/unset] | [YES/NO] |
| readOnlyRootFilesystem | [true/false/unset] | [INFO] |

**Change History (from managedFields):**
| Timestamp | Manager | Fields Changed |
|-----------|---------|----------------|
| [time] | kubectl-patch | securityContext.runAsUser, capabilities.add, allowPrivilegeEscalation |
| [time] | kubectl-client-side-apply | initial creation |

**Assessment:**
[e.g., "A kubectl patch at [timestamp] introduced root UID, NET_ADMIN, and privilege escalation — overriding the originally compliant spec."]

Continue to check ServiceAccount SCC bindings? (yes/no)

WAIT for user confirmation before proceeding.

Step 5: Check ServiceAccount SCC Bindings

MCP Tool: resources_get (from openshift)

Parameters:

• kind: "ServiceAccount" (resource type)
• name: "" (from Deployment .spec.template.spec.serviceAccountName, default: default)
• namespace: ""

Then attempt:

MCP Tool: resources_list (from openshift)

Parameters:

• kind: "SecurityContextConstraints" (cluster-scoped)

Expected Output: ServiceAccount details and list of SCCs. If listing SCCs is RBAC-forbidden, infer SCC access from the rejection messages in Step 3.

Error Handling:

• If listing SCCs is forbidden: gracefully degrade — use rejection messages to infer which SCCs were attempted and their result
• If ServiceAccount not found: report it; this would also cause pod creation failures

Present to user:

## ServiceAccount Analysis

**ServiceAccount used by Deployment:** [sa-name] (namespace: [namespace])

**Available Information:**
| Check | Result |
|-------|--------|
| SA exists | [Yes/No] |
| Custom SA or default | [custom/default] |
| SCC bindings visible | [Yes/Forbidden — inferred from rejection messages] |

**SCC Access (from rejection messages):**
| SCC | Access |
|-----|--------|
| restricted-v2 | Available (but pod spec violates it) |
| restricted-v3 | Available (but pod spec violates it) |
| anyuid | Forbidden — SA has no binding |
| privileged | Forbidden — SA has no binding |

**Assessment:**
[e.g., "The SA 'default' only has access to restricted-v2/v3. The pod spec must be fixed to comply with restricted SCC, OR the SA needs a RoleBinding to a permissive SCC (if elevated privileges are genuinely required)."]

Continue to diagnosis summary? (yes/no)

WAIT for user confirmation before proceeding.

Step 6: Present Diagnosis Summary

Synthesize all findings into a structured summary with actionable remediation options.

Expected Output: Root cause summary, causal chain, and three fix options with commands.

Error Handling:

• If insufficient data from earlier steps: note gaps and recommend manual investigation
• If multiple violations found: prioritize by severity (root UID > capabilities > escalation)

Present to user:

## SCC Violation Diagnosis Summary: [deployment-name]

### Root Cause

**Primary Issue:** [e.g., "kubectl patch introduced privileged security context settings that violate all available SCCs"]

| Category | Status | Details |
|----------|--------|---------|
| Pod Admission | BLOCKED | SCC rejects pod spec |
| SecurityContext | NON-COMPLIANT | [specific violations] |
| ServiceAccount | [OK/MISSING BINDING] | [sa-name] — [SCC access] |
| Change Attribution | [IDENTIFIED/UNKNOWN] | [manager and timestamp from managedFields] |

### Causal Chain (Five Whys)

1. **Signal**: [deployment] has 0 available replicas
2. **Why?** ReplicaSet [rs-name] cannot create pods — [N] FailedCreate events
3. **Why?** Every available SCC rejects the pod spec
4. **Why?** Container securityContext specifies [violations]
5. **Root Cause**: [e.g., "A kubectl patch at [timestamp] modified the securityContext to introduce non-compliant settings"]

### Recommended Actions

**Option A: Fix the SecurityContext (recommended if elevated privileges are NOT needed)**

Remove the non-compliant fields to restore restricted SCC compliance:

```bash
oc patch deployment <deployment-name> -n <namespace> --type json -p '[
  {"op": "remove", "path": "/spec/template/spec/containers/0/securityContext/runAsUser"},
  {"op": "replace", "path": "/spec/template/spec/containers/0/securityContext/allowPrivilegeEscalation", "value": false},
  {"op": "remove", "path": "/spec/template/spec/containers/0/securityContext/capabilities/add"}
]'

Option B: Grant SCC binding (only if elevated privileges are genuinely required)

Create a RoleBinding to a permissive SCC for the ServiceAccount:

oc adm policy add-scc-to-user anyuid -z <sa-name> -n <namespace>

⚠️ Warning: Granting anyuid/privileged SCCs weakens namespace security. Only use if the workload genuinely requires elevated privileges.

Option C: Rollback to previous revision

oc rollout undo deployment/<deployment-name> -n <namespace>

Related Documentation

• OpenShift SCC documentation
• Kubernaut SCC violation golden transcript

---

Would you like me to:

1. Execute Option A (fix SecurityContext)
2. Execute Option B (grant SCC binding)
3. Execute Option C (rollback)
4. Dig deeper into a specific area
5. Exit debugging

Select an option:

**WAIT for user confirmation before proceeding.**

## Dependencies

### Required MCP Servers
- `openshift` — Kubernetes/OpenShift resource access for Deployments, ReplicaSets, Events, ServiceAccounts, and SecurityContextConstraints ([setup](docs/prerequisites.md))

### Required MCP Tools
- `resources_get` (from openshift) — Retrieve individual resource details (Deployment, ReplicaSet, ServiceAccount)
- `resources_list` (from openshift) — List resources by kind in a namespace
- `events_list` (from openshift) — Fetch events filtered by involved object
- `pods_list` (from openshift) — List pods matching label selectors

### Related Skills
- `/debug-pod` — If pods exist but are crashing (CrashLoopBackOff, OOMKilled)
- `/debug-rbac` — If pods run but fail with 403 Forbidden API errors (RBAC, not SCC)

### Reference Documentation
- **Internal:** [docs/debugging-patterns.md](docs/debugging-patterns.md) — Common error patterns and troubleshooting trees
- **Official:** [Managing SCCs - OpenShift](https://docs.openshift.com/container-platform/latest/authentication/managing-security-context-constraints.html)

## Example Usage

**User**: My deployment `security-scanner` in namespace `tools` has 0 available replicas. Events say "unable to validate against any security context constraint". Can you help?

**Skill response**: The skill identifies the deployment, finds FailedCreate events on the ReplicaSet, parses the SCC rejection messages to find that `runAsUser: 0` and `capabilities.add: [NET_ADMIN]` violate `restricted-v2`. It checks the ServiceAccount's SCC bindings, determines it only has access to restricted SCCs, and presents three options: patch the securityContext to remove violations, grant an `anyuid` SCC binding, or rollback to the previous revision.