visa-vulnerability-agentic-harness

name	visa-vulnerability-agentic-harness
description	AI-powered SAST pipeline for autonomous vulnerability discovery using LLMs with multi-stage analysis, threat modeling, and SARIF output
triggers	["scan this repository for security vulnerabilities","run VVAH security scan on my code","set up vulnerability scanning with AI agents","analyze attack surface and threats","generate SARIF report for security findings","estimate cost for vulnerability scan","configure VVAH for my project","troubleshoot VVAH scan errors"]

Visa Vulnerability Agentic Harness (VVAH) Skill

Skill by ara.so — AI Agent Skills collection.

VVAH is Visa's open-source harness for autonomous vulnerability discovery using frontier AI models. It implements a 9-stage pipeline: attack surface mapping → threat modeling → multi-lens research → adversarial verification → exploit chaining → SARIF reporting. Designed to reduce Mean Time to Adapt (MTTA) from AI-discovered weakness to validated fix.

Installation

Prerequisites:

• Python ≥ 3.10
• Claude Code session (claude login) OR API key (Anthropic/OpenAI)

Install into virtual environment (recommended):

python3 -m venv .venv
source .venv/bin/activate  # On Windows: .\.venv\Scripts\Activate.ps1
pip install .

Or install globally with pipx:

pipx install .

This installs the vvaharness CLI command.

Configuration

Initial setup:

# Copy environment template
cp .env.example .env

# Edit .env with your credentials
# For Claude Code (default): set CLAUDE_CODE_OAUTH_TOKEN or run `claude login`
# For Anthropic SDK: set ANTHROPIC_SDK_API_KEY
# For OpenAI: set OPENAI_API_KEY and OPENAI_BASE_URL

Backend selection (via profiles):

The default profile uses via: cli (Claude Code) for all stages. To customize:

# Copy a profile template
cp vvaharness/config/profiles/full.yaml ./config.yaml

# Edit config.yaml to select backends per stage

Profile structure:

roles:
  surface_explorer:
    via: cli              # Options: cli, sdk, openai
    model: claude-sonnet-4-6
    computer_use: false
    
  threat_modeler:
    via: sdk
    model: claude-opus-4
    temperature: 1.0
    
  vulnerability_researcher:
    via: openai
    model: gpt-4
    base_url: ${OPENAI_BASE_URL}

Environment variables:

# Claude Code (cli backend)
CLAUDE_CODE_OAUTH_TOKEN=your_token_here
CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1  # For enterprise gateways

# Anthropic SDK (sdk backend)
ANTHROPIC_SDK_API_KEY=sk-ant-xxxxx
ANTHROPIC_SDK_BASE_URL=https://your-gateway.com  # Optional
ANTHROPIC_SDK_CA_CERT=/path/to/ca.pem            # For mTLS
ANTHROPIC_SDK_CLIENT_CERT=/path/to/client.pem    # For mTLS

# OpenAI (openai backend)
OPENAI_API_KEY=sk-xxxxx
OPENAI_BASE_URL=https://api.openai.com/v1        # Or compatible endpoint

# Enterprise gateway
NODE_EXTRA_CA_CERTS=/path/to/ca-bundle.pem

Key Commands

Setup and Validation

# Verify configuration and credentials
vvaharness doctor

# Check what's needed for your setup
vvaharness setup

# Install agent instructions (Claude/Copilot/Gemini)
vvaharness setup --install-agents

Cost Estimation

# Estimate scan cost without spending
vvaharness estimate --repo /path/to/target

# Estimate for remote repository
vvaharness estimate --repo https://github.com/org/repo --application-id 12345

Single Repository Scan

# Basic scan
vvaharness scan --repo /path/to/target --application-id 12345

# Scan with custom config
vvaharness scan \
  --repo /path/to/target \
  --application-id 12345 \
  --config ./custom-config.yaml

# Scan specific module
vvaharness scan \
  --repo /path/to/target \
  --application-id 12345 \
  --module backend

# Dry run (no LLM calls)
vvaharness scan --repo /path/to/target --dry-run

Batch Scanning

# Scan multiple repos from CSV
vvaharness scan \
  --repo-file repos.csv \
  --workspace ./scans \
  --group-by-app \
  --keep-clones

# repos.csv format:
# repo_url,application_id,module_name
# https://github.com/org/app1,101,frontend
# https://github.com/org/app2,102,backend

Output Locations

Scan outputs are written to <target>/security-scan/:

<target>/
└── security-scan/
    ├── <module>_<timestamp>_report.md        # Human-readable findings
    ├── <module>_<timestamp>_report.sarif     # SARIF 2.1.0 format
    ├── <module>_<timestamp>_errors.jsonl     # Non-fatal errors
    └── run_manifest.json                     # Metadata (version, models, git SHA)

Pipeline Stages

VVAH runs a 9-stage pipeline across three phases:

Discovery & Modeling (S1-S3):

• S1: Attack surface mapping (code, CMDB, CVE, controls)
• S2: Threat modeling (STRIDE, OWASP, trust boundaries)
• S3: Vulnerability research strategy (taint, API boundaries)

Deep Dive & Verification (S4-S6):

• S4: Multi-lens research (Language, Crypto, Logic, Access Control, IaC)
• S5: Policy gates
• S6: Adversarial verification (exploit chains, trust boundaries)

Synthesis & Reporting (S7-S9):

• S7: Deduplication
• S8: Exploit chain construction (CWE, attack paths)
• S9: SARIF emission

Configuration Examples

Budget Controls

step4:
  max_budget_usd: 5.0          # Per-stage spending cap
  max_budget_usd_per_finding: 0.50  # Per-finding cap

step6:
  max_budget_usd: 3.0
  verification_passes: 3       # Majority-vote rounds (sdk/openai only)

Language-Specific Research

lenses:
  - name: language_specific
    languages: [python, java, javascript]
    focus_areas:
      - injection vulnerabilities
      - unsafe deserialization
      - path traversal
      
  - name: crypto_specific
    focus_areas:
      - weak algorithms
      - hardcoded keys
      - IV reuse
      
  - name: access_control
    focus_areas:
      - broken authorization
      - privilege escalation
      - IDOR

Model Selection Strategy

# High-reasoning stages use Opus
roles:
  threat_modeler:
    via: sdk
    model: claude-opus-4
    
  adversarial_reviewer:
    via: sdk
    model: claude-opus-4

# Volume stages use Sonnet    
  vulnerability_researcher:
    via: cli
    model: claude-sonnet-4-6
    
  exploit_strategist:
    via: sdk
    model: claude-sonnet-4-6

Common Patterns

Enterprise Gateway Setup

# .env configuration for enterprise
ANTHROPIC_SDK_BASE_URL=https://ai-gateway.corp.com
ANTHROPIC_SDK_CA_CERT=/etc/ssl/certs/corp-ca.pem
ANTHROPIC_SDK_CLIENT_CERT=/etc/ssl/certs/client.pem
ANTHROPIC_SDK_CLIENT_KEY=/etc/ssl/private/client-key.pem
NODE_EXTRA_CA_CERTS=/etc/ssl/certs/ca-bundle.pem
CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1

CI/CD Integration

# .github/workflows/security-scan.yml
name: VVAH Security Scan

on:
  pull_request:
    branches: [main]
  schedule:
    - cron: '0 2 * * 1'  # Weekly Monday 2am

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.10'
          
      - name: Install VVAH
        run: |
          python -m pip install --upgrade pip
          pip install .
          
      - name: Run scan
        env:
          ANTHROPIC_SDK_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
        run: |
          vvaharness scan \
            --repo . \
            --application-id ${{ github.repository_id }} \
            --config .vvah-config.yaml
            
      - name: Upload SARIF
        uses: github/codeql-action/upload-sarif@v2
        with:
          sarif_file: security-scan/*_report.sarif

Programmatic Integration

import subprocess
import json
from pathlib import Path

def run_vvah_scan(repo_path: str, app_id: str) -> dict:
    """Run VVAH scan and return manifest."""
    result = subprocess.run(
        [
            "vvaharness", "scan",
            "--repo", repo_path,
            "--application-id", app_id,
            "--config", "config.yaml"
        ],
        capture_output=True,
        text=True,
        check=True
    )
    
    # Load manifest
    manifest_path = Path(repo_path) / "security-scan" / "run_manifest.json"
    with open(manifest_path) as f:
        return json.load(f)

def estimate_scan_cost(repo_path: str) -> float:
    """Get cost estimate before running."""
    result = subprocess.run(
        ["vvaharness", "estimate", "--repo", repo_path],
        capture_output=True,
        text=True,
        check=True
    )
    # Parse estimate from stdout
    # Format: "Estimated cost: $X.XX"
    for line in result.stdout.splitlines():
        if "Estimated cost:" in line:
            return float(line.split("$")[1])
    return 0.0

Multi-Model Strategy

# config.yaml - Mix backends for cost/quality tradeoff
roles:
  # Fast exploration with CLI (no API cost if using Claude Code)
  surface_explorer:
    via: cli
    model: claude-sonnet-4-6
    
  # Deep reasoning with SDK Opus
  threat_modeler:
    via: sdk
    model: claude-opus-4
    temperature: 1.0
    
  # Bulk research with OpenAI (if cheaper in your region)
  vulnerability_researcher:
    via: openai
    model: gpt-4-turbo
    
  # Final verification with SDK Opus
  adversarial_reviewer:
    via: sdk
    model: claude-opus-4

Troubleshooting

Authentication Issues

# Check credential status
vvaharness doctor

# For Claude Code
claude login
# Or
claude setup-token  # Get token for CLAUDE_CODE_OAUTH_TOKEN

# For SDK
echo $ANTHROPIC_SDK_API_KEY  # Should start with sk-ant-

# For OpenAI
echo $OPENAI_API_KEY  # Should start with sk-

Budget Exceeded

# Review per-stage budgets in config
cat config.yaml | grep max_budget_usd

# Reduce budgets
step4:
  max_budget_usd: 2.0          # Down from 5.0
  max_budget_usd_per_finding: 0.25  # Down from 0.50

Gateway/Proxy Issues

# Test connectivity
curl -H "x-api-key: $ANTHROPIC_SDK_API_KEY" \
  $ANTHROPIC_SDK_BASE_URL/v1/messages \
  -d '{"model":"claude-sonnet-4","max_tokens":10,"messages":[{"role":"user","content":"hi"}]}'

# Check CA cert
openssl verify -CAfile $NODE_EXTRA_CA_CERTS $ANTHROPIC_SDK_CA_CERT

# Disable betas for enterprise gateways
export CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1

No Findings Generated

# Check errors log
cat <repo>/security-scan/*_errors.jsonl

# Increase verbosity (if supported in future version)
vvaharness scan --repo . --application-id 123 --verbose

# Verify lenses are enabled
grep -A5 "lenses:" config.yaml

SARIF Validation

# Validate SARIF output
npm install -g @microsoft/sarif-multitool
sarif-multitool validate security-scan/*_report.sarif

# View in GitHub Code Scanning format
gh api repos/:owner/:repo/code-scanning/sarifs \
  -F sarif=@security-scan/*_report.sarif

Performance Optimization

# Limit to specific file patterns (custom config)
discovery:
  include_patterns:
    - "**/*.py"
    - "**/*.js"
  exclude_patterns:
    - "**/node_modules/**"
    - "**/venv/**"
    - "**/*.test.js"
    
# Reduce verification passes
step6:
  verification_passes: 1  # Single-pass (faster, less FP filtering)

Best Practices

1. Always estimate first: Run vvaharness estimate before full scans
2. Use profiles: Copy and customize profiles/full.yaml rather than editing defaults
3. Stage budgets: Set max_budget_usd per stage to prevent runaway costs
4. Batch scans: Use --repo-file + --group-by-app for multi-repo workflows
5. Human review required: VVAH findings are triage candidates, not confirmed CVEs
6. Keep manifests: run_manifest.json tracks model versions for reproducibility
7. Authorized use only: Scan only code you own or have permission to test

Output Format

Markdown Report Structure:

# Security Scan Report

## Executive Summary
- Total findings: X
- Critical: X | High: X | Medium: X | Low: X

## Findings

### F-001: SQL Injection in User Authentication
**Severity:** Critical
**CWE:** CWE-89
**Location:** src/auth/login.py:45-52

**Description:** User-controlled input flows to SQL query...

**Exploit Path:**
1. Attacker provides `' OR '1'='1` as username
2. Query becomes: `SELECT * FROM users WHERE username='' OR '1'='1'`
3. Authentication bypassed

**Remediation:**
- Use parameterized queries
- Apply input validation

## Dropped Findings
[Findings rejected during adversarial review]

SARIF 2.1.0 Structure:

{
  "version": "2.1.0",
  "runs": [{
    "tool": {
      "driver": {
        "name": "VVAH",
        "version": "1.0.0",
        "rules": [...]
      }
    },
    "results": [{
      "ruleId": "CWE-89",
      "level": "error",
      "message": {"text": "SQL Injection in User Authentication"},
      "locations": [{
        "physicalLocation": {
          "artifactLocation": {"uri": "src/auth/login.py"},
          "region": {"startLine": 45, "endLine": 52}
        }
      }]
    }]
  }]
}