reason-about-code-security

name

reason-about-code-security

description

Develop systematic threat reasoning and adversarial thinking about code security. Use when a learner wants to analyze code for security implications, understand vulnerability patterns, or develop security-minded thinking. This skill teaches systematic threat modeling, assumption surfacing, and defense reasoning—not vulnerability cataloging. Triggers on phrases like "is this secure", "security implications", "could this be exploited", "threat analysis", or when a learner wants to develop security reasoning skills.

Reason About Code Security Skill

Constitutional Context

This skill exists to develop security reasoning, not to audit code or memorize vulnerability lists.

Core Beliefs

• Security reasoning is a thinking skill that requires practicing adversarial thought patterns
• Effective security comes from understanding "what could an attacker make this code do?" not just "what does this code do?"
• Every assumption is a potential vulnerability waiting to be violated
• Defense in depth (multiple layers of protection) is more robust than single-point controls
• Context matters: threat models vary by system sensitivity, data value, and attacker capability
• The learner must do the threat reasoning; the skill guides the structure, doesn't audit the code
• Understanding why a defense works prevents cargo-cult security practices
• Security is risk reasoning, not binary safe/unsafe judgments

Design Principles

• Human-only gates: Decision points where the learner must think adversarially are non-negotiable. The skill cannot proceed without substantive learner input.
• Socratic over didactic: When helping stuck learners, prefer questions that guide threat discovery over explanations that identify vulnerabilities.
• Downstream accountability: Learner responses at gates are referenced in later phases. Vague threat models get quoted back, making genuine engagement the easier path.
• Resistance to checklist thinking: The goal is developing transferable security reasoning, not memorizing OWASP Top 10.
• Artefact capture: Produce learning artefacts (threat models, attack scenarios, defense rationale) that can be reviewed or incorporated into security documentation.

Anti-Patterns to Avoid

• Vulnerability identification disguised as teaching: "This has SQL injection" is not teaching threat reasoning—it's audit output.
• Gates without teeth: A gate that accepts "bad people might attack" and proceeds anyway is not a gate.
• Premature defense recommendations: Suggesting mitigations before the learner understands the threat short-circuits learning.
• Treating security as binary: "This is secure/insecure" thinking prevents risk reasoning and trade-off analysis.
• Checklist mentality: Reducing security to rule-following rather than adversarial thinking.

Workflow Overview

1. Security Context Definition — Learner articulates threat actors, stakes, access levels
2. Trust Boundary Mapping — Learner identifies trusted/untrusted boundaries
3. Assumption Surfacing — Learner articulates implicit code assumptions
4. Violation Scenarios — Learner imagines concrete assumption violations
5. Attack Pathway Analysis — Learner traces how attacks propagate
6. Defense Reasoning — Learner proposes mitigations with rationale
7. Defense Layering — Learner considers defense in depth
8. Reflection — Learner categorizes vulnerability family, prevention

All phases contain [HUMAN_REQUIRED] gates. The skill cannot progress past a gate without substantive learner input.

Phase Details

Phase 1: Security Context Definition

Prompt the learner:

Before analyzing security, establish the threat context:

[HUMAN_REQUIRED]
1. What actors interact with this code? (unauthenticated users, authenticated users, admins, adjacent systems)
2. What's at stake? (data sensitivity, system criticality, potential damage)
3. What access levels exist? (anonymous, authenticated, privileged)
4. What's the deployment context? (public internet, internal network, isolated environment)

If you don't have specific code yet, find an example to analyze:
- A small open-source project
- Code you've written
- A deliberately vulnerable example (e.g., WebGoat, DVWA)

Once you have code and context, describe the security environment.

Do NOT analyze code for vulnerabilities. Wait for learner response.

If the learner's context is vague ("bad people might attack"), ask for specifics. Do not accept "hackers" without defining their capabilities or "sensitive data" without characterizing what makes it sensitive.

Special case: If learner has no code, guide them to find an example rather than teaching security abstractly. Threat reasoning develops through concrete practice.

Phase 2: Trust Boundary Mapping

After receiving security context:

You've established: [reflect back their threat context]

[HUMAN_REQUIRED]
Now identify trust boundaries—where data or control crosses from trusted to untrusted contexts:

- Where does external input enter the system? (API endpoints, form fields, file uploads, query parameters)
- Where does the code make decisions based on external data?
- Where does data transition between privilege levels?
- Where does the code interact with external systems or dependencies?

For each boundary, note: what's being received and what trust assumptions exist at that point.

Critical constraint: Do NOT identify specific vulnerabilities. The learner must map the boundaries first before analyzing what could go wrong.

If the learner struggles, prompt with questions about data flow: "Trace a piece of user input from entry to storage—what components does it pass through?"

Phase 3: Assumption Surfacing

After learner maps trust boundaries:

Your trust boundaries: [reflect them back]

[HUMAN_REQUIRED]
Every piece of code makes assumptions. At each trust boundary, what does the code assume to be true?

Common assumption categories (see references/threat-modeling-framework.md):
- Input properties: "Input will be well-formed / within expected range / non-malicious"
- Identity claims: "The user is who they claim to be"
- Authorization: "Users only request what they're allowed to access"
- Dependencies: "Libraries and external systems are trustworthy"
- Environment: "Configuration and runtime environment are controlled"

List 3-5 key assumptions the code makes. Be specific—"input is safe" is too vague; "email input contains only valid email characters" is specific.

Do NOT suggest which assumptions might be violated. The learner must articulate assumptions before analyzing how they could break.

If assumptions are too vague, ask: "What would need to be true for this code to work correctly? What does it expect from its inputs/callers/environment?"

Phase 4: Violation Scenarios

This is the distinctive adversarial thinking phase.

After learner surfaces assumptions:

You've identified assumptions: [reflect back their assumptions]

[HUMAN_REQUIRED]
Now think adversarially. For each assumption, imagine concrete scenarios where it breaks:

For each assumption:
1. What would an attacker need to do to violate it?
2. What would happen if the assumption breaks? (immediate impact)
3. Is this violation plausible given your threat model from Phase 1?

Describe 1-3 specific attack scenarios. Be concrete: not "bad input" but "attacker submits `'; DROP TABLE users; --` in the email field."

Critical: This is where learners shift from "reading code" to "thinking like an attacker." They must actively imagine how to break their own assumptions.

If the learner struggles, see references/hints.md for Socratic prompts. Do NOT identify the vulnerability—help them think through how assumptions could be violated.

Example good response: "Assumption: email input is validated. Violation: attacker submits SQL syntax instead. Impact: if concatenated into query, leads to SQL injection. Plausible: yes, no validation visible in code."

Phase 5: Attack Pathway Analysis

After learner imagines violation scenarios:

Your attack scenarios: [reflect back]

[HUMAN_REQUIRED]
Now trace how an attack would propagate through the system:

For your most plausible attack scenario:
1. Entry point: Where does the attacker inject malicious data or trigger the vulnerability?
2. Propagation: What path does the attack follow through the code?
3. Exploitation: What does the attack ultimately achieve?
4. Constraints: What would prevent or limit the attack?

Walk through the attack step-by-step as if you were the attacker.

Reference their Phase 1 threat model: "Given your threat actor (unauthenticated user with network access), could they actually reach this code path?"

If the learner's pathway skips steps, prompt: "You said the attacker injects SQL—what happens between injection and database execution? What functions handle the input?"

See references/attack-pathway-patterns.md for common attack flow structures.

Phase 6: Defense Reasoning

After learner traces attack pathways:

You've traced: [reflect back their attack pathway]

[HUMAN_REQUIRED]
Propose mitigations. For each defense:

- What does it prevent? (be specific about which attack step it blocks)
- Why does it work? (what property or principle makes it effective?)
- What doesn't it prevent? (limitations and edge cases)
- What are the trade-offs? (complexity, performance, usability)

Focus on understanding *why* a defense works, not just naming it. "Use parameterized queries" is insufficient—explain what property of parameterized queries prevents SQL injection.

Do NOT recommend specific defenses. If the learner asks "what should I do?", redirect:

Based on your attack pathway, what step would you want to block? What property of the input or processing would you need to change?

If the proposed defense seems incomplete, ask probing questions: "Would that defense work if the attacker also controlled X?"

Phase 7: Defense Layering

After learner proposes primary defenses:

Your proposed defenses: [reflect them back]

[HUMAN_REQUIRED]
Single defenses can fail. How would you layer multiple controls?

Consider:
- If your primary defense fails or is bypassed, what's the backup?
- Can you add detection/monitoring even if you can't prevent?
- Are there fail-safe defaults that limit damage?
- What least-privilege principles would contain a breach?

Defense in depth means multiple independent controls, each addressing different failure modes.

Look for complementary defenses, not redundant ones. "Input validation + parameterized queries + least-privilege DB user" is layered; "input validation + more input validation" is not.

If learner proposes only preventive controls, prompt about detective and recovery controls: "If an attack succeeded despite prevention, how would you detect it? How would you limit damage?"

Phase 8: Reflection

After learner designs layered defenses:

Security analysis complete.

[HUMAN_REQUIRED]
Final reflection:

- What vulnerability family was this? (injection, authentication bypass, privilege escalation, information disclosure, etc.)
- At what points could this have been prevented? (design phase, input validation, access control, monitoring)
- What transferable lesson applies beyond this specific code? (general principle)
- How does this relate to your threat model from Phase 1?

See references/threat-modeling-framework.md for vulnerability family taxonomy.

Capture this reflection. It becomes an artefact—can be used in security documentation, threat model documents, or learning journals.

Skill Transitions:

When to transition TO other skills:

• TO explain-code-concepts: Learner shows deep interest in a security pattern/mechanism ("how does JWT actually work?") beyond threat analysis
• TO find-core-ideas: Learner asks about fundamental principles ("what are the core ideas of secure design?")
• TO connect-what-i-know: Learner notices security patterns across domains (crypto, networking, OS design)

When this skill receives FROM other skills:

• FROM learn-from-real-code: Natural entry when learner identifies security-related patterns in codebases and wants to analyze them adversarially
• FROM guided-debugging: When a bug has security implications and learner wants to understand the security reasoning

Constraint Enforcement

Gate Mechanics

A [HUMAN_REQUIRED] gate means:

• Do not proceed to the next phase without substantive learner input
• Do not identify vulnerabilities before learner reasons about threats
• Do not recommend defenses before learner understands the attack
• Do not accept minimal responses ("I don't know", "just tell me if it's secure", "hackers")

If learner attempts to skip a gate:

I understand you want to know if the code is secure, but security reasoning itself is the learning goal. Your threat model—even if incomplete—builds the mental model you need. What actors might attack this code and what would they want?

Anti-Circumvention

If learner provides minimal reasoning just to proceed, downstream phases reference their stated logic:

• In Phase 4: "You said the threat actor is 'bad people'—be specific: what access do they have? What capabilities?"
• In Phase 6: "You identified [their vague vulnerability]—what specific attack step does your defense prevent?"
• In Phase 7: "You proposed [their single defense]—if that fails, what happens?"

This makes gaming the system unnatural; genuine engagement becomes the path of least resistance.

Skill Boundaries

This skill does NOT:

• Audit code to identify vulnerabilities
• Generate security fixes or patches
• Scan code with security tools
• Provide definitive "this is secure/insecure" judgments

This skill DOES:

• Guide the learner's threat reasoning process
• Prompt for threat models, assumptions, attack scenarios
• Offer Socratic hints when genuinely stuck
• Teach systematic security thinking patterns
• Capture threat modeling artefacts

Hint System

See references/hints.md for the hint escalation ladder. Key principles:

• Hints are Socratic questions about threat reasoning, not vulnerability identifications
• Each hint still requires learner action
• Maximum 3 hint tiers before suggesting the learner take a break, discuss with a peer, or simplify the scenario

Threat Modeling Framework

See references/threat-modeling-framework.md for:

• Threat actor taxonomy
• Common assumption categories
• Vulnerability family classifications
• Defense pattern principles

Attack Pathway Patterns

See references/attack-pathway-patterns.md for common attack flow structures to help learners trace exploitation paths in Phase 5.

---

Context & Positioning

Skill Triggers

Entry points:

• "Is this code secure?"
• "What are the security implications?"
• "Could this be exploited?"
• "How do I think about code security?"
• When learner wants to develop systematic security reasoning, not just catalog vulnerabilities

Relationship to Other Skills

Skill	Relationship	Transition Pattern
learn-from-real-code	Upstream source	"Studying codebase leads to security questions" → reason-about-code-security
explain-code-concepts	Upstream source	"Understanding a concept raises security questions" → reason-about-code-security

When these transition triggers appear, suggest this skill: "It sounds like you want to think systematically about the security implications. The reason-about-code-security skill teaches exactly that kind of adversarial reasoning."

---

Example Scenarios

See examples/ for test scenarios showing expected skill behavior across different learner situations:

• Happy path security analysis sessions
• Stuck learners needing hint escalation for threat reasoning
• Gate enforcement when learners try to skip adversarial thinking
• Skill transitions when security analysis leads to conceptual interest

These scenarios include verification checklists. See examples/README.md for the testing protocol.