defense-profiler

name	defense-profiler
description	Codebase defense analysis system for security profiling
user-invocable	true

Defense Profiler

v2.88 Key Changes (MODEL-AGNOSTIC)

• Model-agnostic: Uses model configured in ~/.claude/settings.json or CLI/env vars
• No flags required: Works with the configured default model
• Flexible: Works with GLM-5, Claude, Minimax, or any configured model
• Settings-driven: Model selection via ANTHROPIC_DEFAULT_*_MODEL env vars

Codebase Defense Analysis System inspired by ZeroLeaks defense profiling.

Systematically analyzes code to build a comprehensive profile of security defenses, patterns, and weaknesses.

Core Concept

Before attacking (testing), understand the defenses. Build a mental model of:

• What patterns exist
• What safeguards are in place
• What triggers failures
• What areas are weak

Usage

/defense-profile src/
/defense-profile --focus security src/auth/
/defense-profile --output profile.json src/api/

Defense Levels

Level	Description	Characteristics
none	No apparent protection	Direct vulnerabilities, no validation
weak	Basic protections	Simple validation, incomplete coverage
moderate	Standard defenses	Input validation, error handling
strong	Good security	Multiple layers, proper sanitization
hardened	Defense in depth	Multi-layered, comprehensive coverage

Profile Structure

interface DefenseProfile {
  // Overall assessment
  level: DefenseLevel;
  confidence: number;  // 0-1

  // Observed patterns
  observedBehaviors: string[];

  // Guardrails in place
  guardrails: {
    type: string;
    strength: number;
    coverage: string[];
    bypassed: boolean;
    bypassMethod?: string;
  }[];

  // Identified weaknesses
  weaknesses: {
    category: string;
    description: string;
    exploitability: number;  // 0-1
    location?: string;
  }[];

  // Trigger patterns
  refusalTriggers: string[];  // What causes errors/rejects
  safePatterns: string[];     // What passes through

  // Response patterns
  responsePatterns: {
    pattern: string;
    frequency: number;
    defenseIndicator: boolean;
  }[];
}

Analysis Dimensions

1. Input Validation

dimension: input_validation
checks:
  - type_checking: Are inputs typed?
  - null_handling: Are nulls handled?
  - boundary_checks: Are limits enforced?
  - sanitization: Is input sanitized?
  - encoding_handling: Are encodings handled?

signals:
  weak:
    - No type annotations
    - Missing null checks
    - Unconstrained inputs
  strong:
    - Zod/Joi schemas
    - TypeScript strict mode
    - Comprehensive validation

2. Authentication/Authorization

dimension: auth_security
checks:
  - auth_mechanism: How is auth implemented?
  - session_management: How are sessions handled?
  - permission_checks: Are permissions verified?
  - token_validation: Are tokens properly validated?

signals:
  weak:
    - Plain text passwords
    - No CSRF protection
    - Missing auth middleware
  strong:
    - bcrypt/argon2 hashing
    - JWT with refresh tokens
    - RBAC implementation

3. Error Handling

dimension: error_handling
checks:
  - error_exposure: Are errors exposed?
  - logging_security: Are logs secure?
  - graceful_degradation: Does it fail safely?
  - recovery_patterns: Are there recovery mechanisms?

signals:
  weak:
    - Stack traces in responses
    - Sensitive data in logs
    - Unhandled promise rejections
  strong:
    - Generic error messages
    - Structured logging
    - Error boundaries

4. Data Protection

dimension: data_protection
checks:
  - encryption_at_rest: Is data encrypted?
  - encryption_in_transit: Is TLS used?
  - secret_management: How are secrets handled?
  - pii_handling: Is PII protected?

signals:
  weak:
    - Hardcoded secrets
    - No encryption
    - PII in logs
  strong:
    - Environment variables
    - Key management service
    - Data masking

5. Injection Prevention

dimension: injection_prevention
checks:
  - sql_injection: Are queries parameterized?
  - xss_prevention: Is output escaped?
  - command_injection: Are commands sanitized?
  - path_traversal: Are paths validated?

signals:
  weak:
    - String concatenation in queries
    - innerHTML usage
    - Shell commands with user input
  strong:
    - ORM with parameterization
    - Content Security Policy
    - Input whitelisting

Pattern Detection

Refusal Triggers

Patterns that cause the code to reject/fail:

const refusalPatterns = [
  // Authentication failures
  /unauthorized|unauthenticated|forbidden/i,

  // Validation failures
  /invalid|malformed|bad request/i,

  // Security blocks
  /blocked|denied|not allowed/i,

  // Rate limiting
  /too many requests|rate limit/i
];

Leak Indicators

Patterns suggesting information exposure:

const leakPatterns = [
  // Configuration exposure
  /config|settings|environment/i,

  // Implementation details
  /stack trace|internal error|debug/i,

  // Sensitive data
  /password|secret|token|key/i
];

Profiling Algorithm

def build_defense_profile(codebase):
    """
    Build comprehensive defense profile.

    Returns:
        DefenseProfile with all assessments
    """
    profile = DefenseProfile()

    # Phase 1: Static Analysis
    patterns = analyze_static_patterns(codebase)
    profile.observedBehaviors.extend(patterns)

    # Phase 2: Dependency Analysis
    deps = analyze_dependencies(codebase)
    profile.guardrails.extend(identify_guardrails(deps))

    # Phase 3: Pattern Matching
    for file in codebase.files:
        weak_signals = match_weak_patterns(file)
        strong_signals = match_strong_patterns(file)

        profile.weaknesses.extend(weak_signals)
        profile.responsePatterns.extend(strong_signals)

    # Phase 4: Level Assessment
    profile.level = calculate_defense_level(profile)
    profile.confidence = calculate_confidence(profile)

    return profile


def calculate_defense_level(profile):
    """
    Calculate overall defense level based on signals.
    """
    weak_count = len(profile.weaknesses)
    strong_count = len([g for g in profile.guardrails if g.strength > 0.7])

    ratio = strong_count / (weak_count + strong_count + 1)

    if ratio > 0.9:
        return "hardened"
    elif ratio > 0.7:
        return "strong"
    elif ratio > 0.4:
        return "moderate"
    elif ratio > 0.2:
        return "weak"
    else:
        return "none"

Integration with Ralph Loop

Defense profiling runs early in the analysis:

Step 1b: GAP-ANALYST
  - Defense Profiler (if security-related task)

Step 6: EXECUTE-WITH-SYNC
  - 6a. LSA-VERIFY
      - Check against Defense Profile

Step 7: VALIDATE
  - 7c. ADVERSARIAL-CODE
      - Use Defense Profile to guide analysis

Invocation

Task:
  subagent_type: "defense-profiler"
  model: "sonnet"
  prompt: |
    TARGET_PATH: src/
    FOCUS: security
    DEPTH: comprehensive

    Build defense profile for the target codebase.
    Identify:
    - Current defense level
    - Active guardrails
    - Weaknesses and their exploitability
    - Patterns and signals

Output Format

{
  "defense_profile": {
    "level": "moderate",
    "confidence": 0.78,
    "summary": "Codebase has standard defenses with some gaps",

    "guardrails": [
      {
        "type": "input_validation",
        "strength": 0.65,
        "coverage": ["api/auth", "api/users"],
        "bypassed": false
      },
      {
        "type": "rate_limiting",
        "strength": 0.80,
        "coverage": ["api/*"],
        "bypassed": false
      }
    ],

    "weaknesses": [
      {
        "category": "injection",
        "description": "SQL concatenation in legacy queries",
        "exploitability": 0.7,
        "location": "src/db/legacy.ts:45"
      },
      {
        "category": "auth",
        "description": "Missing CSRF token validation",
        "exploitability": 0.5,
        "location": "src/middleware/auth.ts"
      }
    ],

    "recommendations": [
      "Migrate legacy queries to parameterized ORM",
      "Add CSRF middleware to state-changing endpoints",
      "Implement rate limiting on auth endpoints"
    ]
  },

  "analysis_metadata": {
    "files_analyzed": 156,
    "patterns_matched": 23,
    "duration_ms": 4500
  }
}

Weakness Categories

Category	Description	Examples
injection	Code injection vulnerabilities	SQL, XSS, Command
auth	Authentication/authorization issues	Missing checks, weak tokens
crypto	Cryptographic weaknesses	Weak algorithms, poor key management
config	Configuration problems	Hardcoded secrets, insecure defaults
data	Data protection issues	Unencrypted PII, logging sensitive data
trust	Trust boundary violations	SSRF, open redirects
resource	Resource management	Memory leaks, DoS vectors

CLI Commands

# Full profile
ralph defense-profile src/

# Focused on security
ralph defense-profile --focus security src/auth/

# Quick scan
ralph defense-profile --quick src/

# Output to file
ralph defense-profile src/ --output defense-profile.json

# Compare profiles
ralph defense-profile --compare baseline.json src/

Continuous Monitoring

Set up as pre-commit hook:

#!/bin/bash
# Run defense profile on changed files
CHANGED=$(git diff --cached --name-only)
ralph defense-profile --files "$CHANGED" --threshold moderate

# Fail if level dropped
if [ $? -ne 0 ]; then
    echo "Defense level dropped below threshold"
    exit 1
fi

Best Practices

1. Profile First: Always profile before testing
2. Track Changes: Compare profiles over time
3. Focus on Weaknesses: Prioritize by exploitability
4. Verify Guardrails: Ensure they're actually working
5. Update Regularly: Defenses change, re-profile often

Attribution

Defense profiling patterns adapted from ZeroLeaks defense analysis system (FSL-1.1-Apache-2.0).