| name | triage-security |
| description | Triage open security alerts on the current GitHub repo — both Dependabot (dependency CVEs) and Code Scanning (CodeQL / SnykCode / other tools that emit to GitHub's code-scanning API). For each alert, decide whether the bug's threat model is actually exploitable given this codebase's archetype, calling code, and reachability. Applicable findings get a tracking issue opened; not-applicable findings get dismissed with the appropriate per-source reason (`inaccurate` / `not_used` for Dependabot; `false positive` / `used in tests` for Code Scanning). Repo-agnostic — works across the Valory fleet. |
| argument-hint | [--limit N] [--rerun-dismissed] [--source dependabot|code-scanning|both] # --limit caps alerts processed; --rerun-dismissed walks already-dismissed Dependabot alerts and reports verdict drift (no mutations, Dependabot-only); --source defaults to `both` |
| disable-model-invocation | true |
Triage security alerts (Dependabot + Code Scanning)
Walk every open security alert on the current repo from both Dependabot (/repos/{r}/dependabot/alerts) and Code Scanning (/repos/{r}/code-scanning/alerts). For each alert, classify by whether it is reachable from production code and then whether the bug's threat model preconditions are satisfied by this codebase's actual usage. Then act in one pass:
| Source | Classification | Confidence + extras | Action |
|---|
| Dependabot | Production-reachable AND exploit-applicable | any | Open tracking issue, leave Dependabot alert open. |
| Dependabot | Production-reachable but exploit NOT applicable — cli-tool | high | Dismiss with inaccurate (+ audit issue per §3.1c). |
| Dependabot | Production-reachable but exploit NOT applicable — framework / scaffold + §2.5.6b passes | high | Dismiss with inaccurate (+ audit issue). |
| Dependabot | Production-reachable but exploit NOT applicable — moderate/low conf OR framework + §2.5.6b fails | — | Open issue + needs-human-review. Do NOT dismiss. |
| Dependabot | Test / dev / CI-only import path | n/a | Dismiss with not_used (+ audit issue). |
| Code Scanning | Rule matches RULE_HUMAN_REVIEW_PATTERNS (cert/TLS bypass, injection in prod, hardcoded secrets non-test, auth/authz, RCE/deserialization, path traversal) | any | Open issue + needs-human-review. Never auto-dismiss this rule class. |
| Code Scanning | Applicable — flagged code is reachable from prod entry point AND §2.5 preconditions hold | any | Open tracking issue, leave code-scanning alert open. |
| Code Scanning | NOT applicable (false positive) — cli-tool + §2.5 ruled preconditions absent | high | Dismiss with false positive (+ audit issue). |
| Code Scanning | NOT applicable — framework/service archetype OR moderate/low confidence | — | Open issue + needs-human-review. Do NOT dismiss. |
| Code Scanning | Flagged file is under tests/ / PKG_TEST_DIRS | n/a | Dismiss with used in tests (+ audit issue). |
| Either source | Unclassifiable | n/a | Skip. stderr line for manual review. Never auto-dismiss without evidence. |
Conservative defaults: when uncertain, open the issue. A false-positive issue is cheap to close; a false-negative dismissal hides a real vulnerability. won't fix (code-scanning) and tolerable_risk (Dependabot) are never set by the skill — those are maintainer-only risk-accept calls.
Three paths to autonomous dismissal:
- Dependabot: cli-tool + high confidence + preconditions absent →
inaccurate
- Dependabot: framework/scaffold + high confidence + §2.5.6b structural-impossibility passes →
inaccurate
- Dependabot: test/dev-only import path (Signal C) →
not_used
- Code-scanning: file under tests/ →
used in tests
- Code-scanning: cli-tool + high confidence + §2.5 preconditions absent + rule NOT in
RULE_HUMAN_REVIEW_PATTERNS → false positive
Every dismissal produces a closed audit-trail issue (§3.1c) labelled security-audit. The 280-char dismissed_comment carries a one-line summary + audit-issue URL pointer.
This skill runs fully autonomously on invocation — it mutates GitHub state (dismisses alerts, opens issues). Do not invoke from conversational context; require explicit /triage-security.
Phase 0 — Ground truth
case "$(uname -s 2>/dev/null)" in
Linux*|Darwin*) ;;
MINGW*|MSYS*|CYGWIN*)
echo "WARN: running under $(uname -s) — Git Bash / MSYS path is untested. Most operations should work; report any failures." >&2
;;
*)
echo "WARN: unrecognized platform $(uname -s). Skill is bash-only and may fail on this OS." >&2
;;
esac
for cmd in gh jq python3 grep find sed tr printf mktemp; do
command -v "$cmd" >/dev/null 2>&1 \
|| { echo "ERROR: required CLI not found on PATH: $cmd"; exit 1; }
done
gh repo view --json owner,name --jq '"\(.owner.login)/\(.name)"' > /tmp/td_repo.txt 2>/dev/null \
|| { echo "ERROR: not in a GitHub repo (gh repo view failed)"; exit 1; }
REPO=$(cat /tmp/td_repo.txt)
echo "operating on $REPO"
DEPENDABOT_AVAILABLE="no"
CODESCAN_AVAILABLE="no"
_check_alerts_api() {
local label="$1" url="$2" err
if err=$(gh api "$url" --jq 'length' 2>&1 >/dev/null); then
return 0
fi
if printf '%s' "$err" | grep -qiE 'HTTP 40[13]|Bad credentials|requires authentication|missing.*scope|needs the .*scope'; then
echo "ERROR: gh token lacks required scope for $label alerts on $REPO." >&2
echo " Run: gh auth refresh -h github.com -s security_events" >&2
echo " Underlying error: $err" >&2
exit 1
fi
echo "WARN: $label alerts API unreachable on $REPO (feature likely not enabled) — $label path will be skipped" >&2
return 1
}
_check_alerts_api "Dependabot" "repos/$REPO/dependabot/alerts?per_page=1" && DEPENDABOT_AVAILABLE="yes"
_check_alerts_api "code-scanning" "repos/$REPO/code-scanning/alerts?per_page=1" && CODESCAN_AVAILABLE="yes"
[[ "$DEPENDABOT_AVAILABLE" == "no" && "$CODESCAN_AVAILABLE" == "no" ]] \
&& { echo "ERROR: neither Dependabot nor code-scanning is enabled on $REPO. Nothing to triage."; exit 1; }
test -f pyproject.toml \
|| { echo "ERROR: no pyproject.toml — skill only supports Python repos right now"; exit 1; }
SKILL_VERSION="2026-05-14"
export SKILL_VERSION
0.2 RULE_HUMAN_REVIEW_PATTERNS — code-scanning rule classes that NEVER auto-dismiss
For code-scanning alerts (Phase 2-cs), some bug classes carry asymmetric risk: a false-positive dismissal of a real vulnerability is catastrophic, while a false-positive issue is cheap to close. For these classes the skill bypasses §2.5 and routes directly to a needs-human-review issue — regardless of how clean the confidence-tier analysis looks.
Pattern-match each alert's rule.id (and rule.tags for CWE-based matching) against this list. Match → force needs-human-review.
RULE_HUMAN_REVIEW_PATTERNS=(
"*disabled-certificate-check*"
"*SSLVerificationBypass*"
"*TooPermissiveTrustManager*"
"*MissingCertVerification*"
"*InsecureProtocol*"
"*SQLInjection*"
"*XSS*"
"*CommandInjection*"
"*ShellInjection*"
"*InsecureXmlParser*"
"*XmlInjection*"
"*LdapInjection*"
"*TaintedFormatString*"
"*HardcodedPassword"
"*HardcodedCredential*"
"*HardcodedNonCryptoSecret"
"*HardcodedKey"
"*EmbeddedCredentials*"
"*MissingAuth*"
"*WeakAuthorization*"
"*BrokenAccessControl*"
"*authentication*"
"*authorization*"
"*UnsafeDeserialization*"
"*UnsafePickle*"
"*PickleLoad*"
"*UnsafeYaml*"
"*CodeInjection*"
"*EvalUsage*"
"*UnsafeReflection*"
"*PathTraversal*"
"*TaintedPath*"
"*ZipSlip*"
)
matches_human_review_pattern() {
local rule_id="$1"
local rid_lower pat_lower
rid_lower=$(printf '%s' "$rule_id" | tr '[:upper:]' '[:lower:]')
for pat in "${RULE_HUMAN_REVIEW_PATTERNS[@]}"; do
pat_lower=$(printf '%s' "$pat" | tr '[:upper:]' '[:lower:]')
[[ "$rid_lower" == $pat_lower ]] && return 0
done
return 1
}
The list is deliberately conservative on injection / RCE / auth — false-positive issues on *PathTraversal* are cheap; false-negative dismissals of a real one are expensive. Refine over time based on the actual false-positive rate observed in dismissed audit issues.
The skill does NOT match rule IDs with a /test suffix (e.g. python/NoHardcodedPasswords/test) against these patterns — those are tool-vendor-tagged test variants and are handled by the file-path-in-tests/ check in Phase 2-cs. Pattern matching applies to the bare (non-/test) rule IDs.
Capture into working memory:
| Field | Source |
|---|
REPO | <owner>/<name> from gh repo view |
PROD_PATHS | Default: autonomy/ packages/ agent/ — adjust to repo. See Phase 2.1 for repo-specific overrides. |
TEST_PATHS | Default: tests/ scripts/ .github/ plus every **/tests/ subdir under packages/. |
LIMIT | Optional --limit N arg; otherwise process all open alerts. |
ARCHETYPE | Repo archetype (see §0.1). Drives the exploit-surface threshold in Phase 2.5. |
0.1 Repo archetype
Different archetypes have radically different exploit surfaces for the same CVE. A header-leak CVE in urllib3 matters for an internet-facing web service handling user auth tokens; it doesn't matter for a CLI link checker that holds no auth state. The skill needs this distinction baked into Phase 2.5's reasoning, not bolted on after the fact.
Classify the current repo into one of:
| Archetype | Signal | Exploit-surface posture |
|---|
cli-tool | Entry point is a console_scripts / [tool.poetry.scripts], runs once and exits, no long-lived HTTP listener, processes developer-controlled inputs. E.g. tomte, aea-helpers, dev/CI plugins. | Narrow. CVEs requiring browser sessions, server-side cookies, persistent attacker control, or untrusted-remote inputs do not apply. Memory-DoS CVEs only matter if uptime is a contract. |
framework | Library/SDK consumed by other projects. No own entry point. E.g. open-autonomy, open-aea, mech-interact. | Wide. Any CVE that consumers may hit applies — must err on the side of bumping, because downstream usage isn't fully known. |
service | Long-running process, network-facing, handles untrusted input. E.g. middleware HTTP API, mech-server, agent HTTP endpoints. | Wide. Default-apply for any CVE in the request/response or auth path. Memory-DoS, header-leak, deserialization CVEs all real. |
scaffold | A starter template — runtime exposure depends on downstream usage, not on this repo's own code. E.g. olas-sdk-starter. | Inherit consumer's archetype. Treat as framework unless a specific consumer is in scope. |
unknown | None of the above. | Default to framework posture. |
Heuristics to detect automatically (run in Phase 0):
PROD_DIRS=()
for d in autonomy packages plugins libs aea operate agent; do
[[ -d "$d" ]] && PROD_DIRS+=("$d")
done
HAS_SCRIPTS=$(python3 -c "
import tomllib, pathlib
d = tomllib.loads(pathlib.Path('pyproject.toml').read_text())
print(bool(
d.get('project', {}).get('scripts')
or d.get('tool', {}).get('poetry', {}).get('scripts')
))" 2>/dev/null)
if [[ ${#PROD_DIRS[@]} -gt 0 ]]; then
HAS_SERVER=$(grep -rlE "^(import|from)[[:space:]]+(fastapi|flask|aiohttp\.web|uvicorn|starlette)" \
"${PROD_DIRS[@]}" --include="*.py" 2>/dev/null | grep -v "/tests/" | head -1)
else
HAS_SERVER=""
fi
HAS_PACKAGES_TREE=$([ -d packages/valory ] && echo "yes" || echo "no")
IS_SCAFFOLD=$(grep -iE "starter|template|scaffold|boilerplate" README.md 2>/dev/null | head -1)
if [[ -n "$IS_SCAFFOLD" ]]; then
ARCHETYPE="scaffold"
elif [[ -n "$HAS_SERVER" ]]; then
ARCHETYPE="service"
elif [[ "$HAS_PACKAGES_TREE" == "yes" ]]; then
ARCHETYPE="framework"
elif [[ "$HAS_SCRIPTS" == "True" ]]; then
ARCHETYPE="cli-tool"
else
ARCHETYPE="framework"
fi
export ARCHETYPE
echo "ARCHETYPE=$ARCHETYPE (HAS_SCRIPTS=$HAS_SCRIPTS HAS_SERVER=${HAS_SERVER:+yes} HAS_PACKAGES_TREE=$HAS_PACKAGES_TREE IS_SCAFFOLD=${IS_SCAFFOLD:+yes})"
These are hints, not proofs. When the classification is ambiguous, default to framework so the skill errs on the side of bumping. A human can override by writing archetype: cli-tool in a per-repo .triage-security.yaml config (future enhancement — for now, hardcode the override at the top of the per-alert loop).
Phase 1 — Fetch alerts (Dependabot + Code Scanning)
TMP=$(mktemp -d)
set -euo pipefail
LIMIT=""
MODE="live"
SOURCE="both"
while [[ $# -gt 0 ]]; do
case "$1" in
--limit) LIMIT="$2"; shift 2 ;;
--limit=*) LIMIT="${1#*=}"; shift ;;
--rerun-dismissed) MODE="rerun-dismissed"; shift ;;
--source) SOURCE="$2"; shift 2 ;;
--source=*) SOURCE="${1#*=}"; shift ;;
*) shift ;;
esac
done
[[ -n "$LIMIT" ]] && [[ ! "$LIMIT" =~ ^[0-9]+$ ]] \
&& { echo "ERROR: --limit must be a non-negative integer, got: $LIMIT"; exit 1; }
case "$SOURCE" in
dependabot|code-scanning|both) ;;
*) echo "ERROR: --source must be dependabot|code-scanning|both, got: $SOURCE"; exit 1 ;;
esac
if [[ "$MODE" == "rerun-dismissed" ]]; then
if [[ "$SOURCE" == "code-scanning" ]]; then
echo "ERROR: --rerun-dismissed is incompatible with --source=code-scanning. Use --source=dependabot or --source=both (code-scanning fetch is skipped in rerun mode)."
exit 1
fi
SOURCE="dependabot"
STATE_FILTER="state=dismissed"
echo "MODE=rerun-dismissed — read-only verdict-drift report on Dependabot dismissals only"
else
STATE_FILTER="state=open"
fi
echo "[]" > "$TMP/alerts.json"
if [[ "$SOURCE" == "dependabot" || "$SOURCE" == "both" ]]; then
if [[ "$DEPENDABOT_AVAILABLE" == "yes" ]]; then
gh api "repos/$REPO/dependabot/alerts?${STATE_FILTER}&per_page=100" --paginate \
> "$TMP/alerts_dependabot.json" \
|| { echo "ERROR: failed to list Dependabot alerts"; exit 1; }
jq -e 'type == "array"' "$TMP/alerts_dependabot.json" > /dev/null \
|| { echo "ERROR: Dependabot response not a JSON array"; head -20 "$TMP/alerts_dependabot.json"; exit 1; }
jq 'map(. + {_source: "dependabot"})' "$TMP/alerts_dependabot.json" \
> "$TMP/alerts_dependabot_tagged.json"
jq -s 'add' "$TMP/alerts.json" "$TMP/alerts_dependabot_tagged.json" \
> "$TMP/alerts_merged.json"
mv "$TMP/alerts_merged.json" "$TMP/alerts.json"
echo "fetched $(jq 'length' "$TMP/alerts_dependabot_tagged.json") Dependabot alerts"
else
echo "Dependabot unavailable on $REPO — skipping that source"
fi
fi
if [[ "$SOURCE" == "code-scanning" || "$SOURCE" == "both" ]]; then
if [[ "$CODESCAN_AVAILABLE" == "yes" ]]; then
gh api "repos/$REPO/code-scanning/alerts?${STATE_FILTER}&per_page=100" --paginate \
> "$TMP/alerts_codescan.json" \
|| { echo "ERROR: failed to list code-scanning alerts"; exit 1; }
jq -e 'type == "array"' "$TMP/alerts_codescan.json" > /dev/null \
|| { echo "ERROR: code-scanning response not a JSON array"; head -20 "$TMP/alerts_codescan.json"; exit 1; }
jq 'map(. + {_source: "code-scanning"})' "$TMP/alerts_codescan.json" \
> "$TMP/alerts_codescan_tagged.json"
jq -s 'add' "$TMP/alerts.json" "$TMP/alerts_codescan_tagged.json" \
> "$TMP/alerts_merged.json"
mv "$TMP/alerts_merged.json" "$TMP/alerts.json"
echo "fetched $(jq 'length' "$TMP/alerts_codescan_tagged.json") code-scanning alerts"
else
echo "Code scanning unavailable on $REPO — skipping that source"
fi
fi
N_ALERTS=$(jq 'length' "$TMP/alerts.json")
if [[ -n "$LIMIT" && "$LIMIT" -lt "$N_ALERTS" ]]; then
echo "found $N_ALERTS alerts on $REPO ($SOURCE); processing first $LIMIT per --limit"
N_PROCESS="$LIMIT"
else
echo "found $N_ALERTS alerts on $REPO ($SOURCE)"
N_PROCESS="$N_ALERTS"
fi
The per-alert loop in Phase 3 / the reference loop below must iterate 0..N_PROCESS-1, not 0..N_ALERTS-1. Phase 4 summary should report both seen (N_ALERTS) and processed (N_PROCESS) when they differ, so the operator knows how many alerts remain in the backlog.
1.1 Fields per alert source
The _source field added in Phase 1 routes each alert into the correct Phase 2 path.
Dependabot alert fields:
| Field | Path |
|---|
| Alert number | .number |
| Alert URL | .html_url |
| Severity | .security_advisory.severity |
| GHSA / CVE | .security_advisory.ghsa_id, .security_advisory.cve_id |
| Summary | .security_advisory.summary |
| Package name | .security_vulnerability.package.name |
| Ecosystem | .security_vulnerability.package.ecosystem (skip if ≠ pip) |
| Vulnerable range | .security_vulnerability.vulnerable_version_range |
| First patched | .security_vulnerability.first_patched_version.identifier |
| Direct/transitive | .dependency.scope (runtime or development) |
| Manifest path | .dependency.manifest_path |
Code-scanning alert fields:
| Field | Path |
|---|
| Alert number | .number |
| Alert URL | .html_url |
| Tool | .tool.name (CodeQL, SnykCode, …) |
| Rule ID | .rule.id |
| Rule severity | .rule.severity (note / warning / error) or .rule.security_severity_level (low/medium/high/critical) |
| Rule description | .rule.description (one-liner) |
| Rule help / extended | .rule.help or .rule.full_description |
| Rule tags | .rule.tags (array — CWE-N entries surface here as external/cwe/cwe-N) |
| Flagged file | .most_recent_instance.location.path |
| Flagged line range | .most_recent_instance.location.start_line, .end_line |
| Flagged code message | .most_recent_instance.message.text |
Skip immediately any alert where:
- (Dependabot)
ecosystem != "pip" — this skill only handles Python deps right now; bucket as non-pip-ecosystem skip (informational, no exit-code impact)
state != "open" in live mode (defensive; pagination races happen)auto_dismissed_at != null (already auto-dismissed by GitHub)- (Code Scanning)
.most_recent_instance.location.path == null — the alert has no resolvable file location; flag for manual review (rare, usually a tool config error)
- (Code Scanning) flagged file path does not exist in the working tree — likely a stale alert against a now-deleted file; bucket as
stale-alert skip
Phase 2 — Classify each alert (per-source dispatch)
For every alert, the per-source dispatch in Phase 2 produces a normalized classification (PROD / DEV / UNCLASSIFIABLE) and a per-source verdict. Then Phase 2.5 runs the exploit-surface analysis (shared between sources). Finally Phase 3 acts based on the combined verdict.
AUTO_DISMISSED_AT=$(jq -r ".[$i].auto_dismissed_at // \"\"" "$TMP/alerts.json")
if [[ -n "$AUTO_DISMISSED_AT" ]]; then
echo "SKIP #$ALERT_NUM: auto-dismissed by GitHub at $AUTO_DISMISSED_AT" >&2
SKIPPED+=("$ALERT_NUM:auto-dismissed:$AUTO_DISMISSED_AT")
continue
fi
SOURCE_OF_THIS_ALERT=$(jq -r ".[$i]._source" "$TMP/alerts.json")
case "$SOURCE_OF_THIS_ALERT" in
dependabot) ;;
code-scanning) ;;
*) echo "ERROR: unknown _source: $SOURCE_OF_THIS_ALERT"; SKIPPED+=("$ALERT_NUM:unknown-source"); continue ;;
esac
The Dependabot logic that follows (§2.1–§2.4) is unchanged from the previous skill version. The code-scanning logic (§2.cs) is new and lives after §2.4.
For every alert, gather evidence in a defined order. First decisive signal wins, with conservative "default to production" as the tie-breaker.
2.1 Resolve production vs test path sets
Production paths and test paths vary by repo. Compute once per skill run and export so the values cross subshell / <<<PY heredoc boundaries cleanly. Then use bash arrays plus "${arr[@]}" expansion at every grep call site — never store the dir set as a space-joined string that gets re-split implicitly across function calls.
PROD_DIRS=()
for d in autonomy packages plugins libs aea operate agent; do
[[ -d "$d" ]] && PROD_DIRS+=("$d")
done
_TEST_DIR_CANDIDATES=(tests scripts .github benchmark examples mints docs)
TEST_DIRS=()
for d in "${_TEST_DIR_CANDIDATES[@]}"; do
[[ -d "$d" ]] && TEST_DIRS+=("$d")
done
PKG_TEST_DIRS=()
while IFS= read -r _d; do
PKG_TEST_DIRS+=("$_d")
done < <(find packages plugins -type d -name tests 2>/dev/null)
printf '%s\n' "${PROD_DIRS[@]}" > "$TMP/prod_dirs.txt"
printf '%s\n' "${TEST_DIRS[@]}" > "$TMP/test_dirs.txt"
printf '%s\n' "${PKG_TEST_DIRS[@]}" > "$TMP/pkg_test_dirs.txt"
export TMP
echo "PROD_DIRS=${PROD_DIRS[*]}"
echo "TEST_DIRS=${TEST_DIRS[*]}"
echo "PKG_TEST_DIRS count=${#PKG_TEST_DIRS[@]}"
Critical: every grep invocation that uses these sets MUST expand the array with "${arr[@]}" (quoted-each-element), not the unquoted string form $ARR. The unquoted form gets re-split by IFS, which silently fails on directory names with spaces and — more commonly — drops the entire list when the variable inherits empty across a subshell boundary. Signal C grepped 12 alerts to "zero hits" against open-aea before this was caught.
grep -rlnE "$REGEX" "${PROD_DIRS[@]}" --include="*.py"
grep -rlnE "$REGEX" $PROD_DIRS --include="*.py"
Important nuance: under packages/<author>/skills/<skill_name>/, the skill's own production code (behaviours.py, rounds.py, models.py, handlers.py) is production; its sibling tests/ subdir is not. Per-skill tests/ dirs are captured by the find packages plugins -type d -name tests line above and grepped as TEST.
2.2 Signal A — GitHub's own dependency.scope
Direct dependencies have a reliable scope field set by Dependabot from manifest parsing:
SCOPE=$(jq -r ".[$i].dependency.scope // \"unknown\"" "$TMP/alerts.json")
runtime → direct production dep. Strong signal toward PROD, but verify with Signal C (an unimported dep is still dev-effectively).development → direct dev/test dep. Strong signal toward DEV, but verify with Signal C (config drift is real).unknown / null → transitive dep. Fall through to Signal B.
2.3 Signal B — pyproject.toml dep-group membership
Direct or transitive, the top-level pin (if any) tells us which group the dep entered the project via:
PKG=$(jq -r ".[$i].security_vulnerability.package.name" "$TMP/alerts.json")
grep -qE "^[[:space:]]*\"$PKG([\[<>=!~,].*)?\"" pyproject.toml \
&& grep -qE "^[[:space:]]*\"$PKG" <(awk '/^dependencies\s*=\s*\[/,/^\]/' pyproject.toml) \
&& echo "found in [project] dependencies (production)"
awk '/^\[dependency-groups\]/,/^\[/' pyproject.toml \
| grep -qE "\"$PKG([\[<>=!~,].*)?\"" \
&& echo "found in [dependency-groups] (test/dev)"
awk '/^\[tool\.poetry\.group\..*\.dependencies\]/,/^\[/' pyproject.toml \
| grep -B5 "^$PKG\s*=" | head -1 \
| grep -q "test\|dev\|tests" && echo "found in poetry dev/test group"
This is shell-form for documentation. In practice, use Python with tomllib (stdlib in 3.11+) for accuracy:
import tomllib, sys, pathlib
data = tomllib.loads(pathlib.Path("pyproject.toml").read_text())
def group_for(pkg: str) -> str | None:
"""Return 'prod', 'dev', or None for this package's declaration group."""
for spec in data.get("project", {}).get("dependencies", []):
if spec.split("[")[0].split("=")[0].split(">")[0].split("<")[0].strip() == pkg:
return "prod"
for group_name, specs in data.get("dependency-groups", {}).items():
for spec in specs:
spec_str = spec if isinstance(spec, str) else spec.get("include-group", "")
if spec_str.split("[")[0].split("=")[0].split(">")[0].split("<")[0].strip() == pkg:
return "dev"
for group_name, group in data.get("tool", {}).get("poetry", {}).get("group", {}).items():
deps = group.get("dependencies", {})
if pkg in deps:
return "dev" if group_name in {"dev", "test", "tests", "lint", "ci"} else "prod"
if pkg in data.get("tool", {}).get("poetry", {}).get("dependencies", {}):
return "prod"
return None
2.4 Signal C — import-graph scan (decisive)
Whatever the pyproject says, the import graph is ground truth. If production code does import <pkg> (or from <pkg> import …), the alert is production regardless of pin group.
Step 1: map the package distribution name to its importable top-level module(s). PyPI distribution names and Python module names diverge frequently (PyYAML → yaml, protobuf → google.protobuf, python-dotenv → dotenv, Pillow → PIL). Resolve via importlib.metadata if the package is installed locally:
import importlib.metadata as im
top_modules: list[str] = []
try:
files = im.files(pkg) or []
pkg_dirs = sorted({
str(f).split("/")[0]
for f in files
if str(f).endswith("/__init__.py") and not str(f).startswith(".")
})
single_files = sorted({
str(f).removesuffix(".py")
for f in files
if str(f).endswith(".py") and "/" not in str(f) and not str(f).startswith(".")
})
top_modules = sorted(set(pkg_dirs) | set(single_files))
except im.PackageNotFoundError:
pass
if not top_modules:
top_modules = [pkg.replace("-", "_")]
If the package isn't installed (deep transitive, never landed in the venv), fall back to the conservative substitution pkg.replace("-", "_") and grep both forms.
Step 2: grep across every top-level module the package ships. PyPI packages frequently expose multiple importable top-levels (setuptools → setuptools + pkg_resources + _distutils_hack; protobuf → google for from google.protobuf...; Pillow → PIL). Build an alternation so a single grep catches any of them:
MODULE_RE=$(printf '%s\n' "${top_modules[@]}" | sed 's/\./\\./g' | paste -sd'|' -)
if [[ -z "$MODULE_RE" || "$MODULE_RE" == "null" ]]; then
echo "WARN: empty/null MODULE_RE for $PKG (Signal C inconclusive) — skipping" >&2
SKIPPED+=("$ALERT_NUM:empty-module-regex:$PKG")
continue
fi
PROD_HIT_FILES=$(grep -rlnE "^(import|from)[[:space:]]+(${MODULE_RE})([[:space:]]|\.|$)" \
"${PROD_DIRS[@]}" --include="*.py" 2>/dev/null \
| grep -v "/build/" | grep -v "/tests/")
PROD_HITS=$(echo "$PROD_HIT_FILES" | grep -c . || echo 0)
TEST_HIT_FILES=$(grep -rlnE "^(import|from)[[:space:]]+(${MODULE_RE})([[:space:]]|\.|$)" \
"${TEST_DIRS[@]}" "${PKG_TEST_DIRS[@]}" --include="*.py" 2>/dev/null \
| grep -v "/build/")
TEST_HITS=$(echo "$TEST_HIT_FILES" | grep -c . || echo 0)
Two extra filters worth keeping:
grep -v "/build/" — many plugins/<plugin>/build/lib/... paths exist after a poetry build run; those are dist-copies of source under the same plugins/<plugin>/<plugin>/ tree. They double-count if not filtered.grep -v "/tests/" in the PROD pipeline — guards against the case where PROD_DIRS includes packages/ (or plugins/) and the recursive grep walks into per-package tests/ subdirs. Those should always count as TEST, never PROD, regardless of which top-level dir they live under.
The ^(import|from)\s+<mod>([\s.]|$) anchor is deliberate: it matches import foo, from foo import …, from foo.bar import …, but not # import foo (comment) or from foobar import … (different package). The trailing alternation ([\s.]|$) is what prevents foobar from matching when grepping for foo.
Step 3: classify. The verdict consults all three signals — A (dependency.scope), B (group_for(pkg) from §2.3), and C (import counts above) — in priority order:
PROD_HITS > 0 → PROD, regardless of A/B (import-graph evidence wins).PROD_HITS == 0 AND TEST_HITS > 0 → DEV (only reachable from tests/CI).- No imports anywhere; classification falls through to A then B:
scope == "development" → DEV.scope == "runtime" → PROD (declared prod, even if currently unimported — better to fix the dep than the imports).scope == "unknown" AND group_for(pkg) == "prod" → PROD (the project explicitly lists this in [project] dependencies / Poetry main → it's production-relevant, dismissing it as unimported would silently bypass an asserted prod dep).scope == "unknown" AND group_for(pkg) == "dev" → DEV (declared in a dev/test group; no imports means it's safe to dismiss as unused).scope == "unknown" AND group_for(pkg) is None (true transitive) → UNCLASSIFIABLE (no signal at all; skip and flag for human review, or fall through to §2.5 transitive reverse-walk if available).
Without the Signal B clauses, the verdict drops a real layer of information on the floor — exactly the case where stacking signals is supposed to pay off.
2.cs Code-scanning classification (alternate to §2.2–§2.4)
For alerts with _source == "code-scanning", the Dependabot signals (A: dependency.scope, B: pyproject group, C: import-graph scan) don't apply — the alert points at a specific line in our own code, not a vulnerable dependency. The code-scanning classification uses three different signals applied in priority order. First decisive signal wins.
2.cs.1 Signal F — file location
The flagged file's location alone usually decides DEV vs PROD for code-scanning. Use the same PROD_DIRS / TEST_DIRS / PKG_TEST_DIRS arrays from §2.1.
ALERT_FILE=$(jq -r ".[$i].most_recent_instance.location.path" "$TMP/alerts.json")
ALERT_LINE=$(jq -r ".[$i].most_recent_instance.location.start_line // 0" "$TMP/alerts.json")
[[ -f "$ALERT_FILE" ]] || { SKIPPED+=("$ALERT_NUM:stale-alert:$ALERT_FILE"); continue; }
ALERT_IN_TESTS="no"
ALERT_IN_PROD="no"
for d in "${TEST_DIRS[@]}" "${PKG_TEST_DIRS[@]}"; do
if [[ "$ALERT_FILE" == "$d/"* ]] || [[ "$ALERT_FILE" == *"/$d/"* ]]; then
ALERT_IN_TESTS="yes"
break
fi
done
if [[ "$ALERT_IN_TESTS" == "no" ]]; then
for d in "${PROD_DIRS[@]}"; do
if [[ "$ALERT_FILE" == "$d/"* ]]; then
ALERT_IN_PROD="yes"
break
fi
done
fi
Verdict from Signal F:
ALERT_IN_TESTS == "yes" → DEV → Phase 3.1d dismiss as used in tests. Skip §2.5 entirely (the analysis doesn't matter — the bug isn't in prod).ALERT_IN_PROD == "yes" → PROD → continue to Signal G + §2.5.- Neither → UNCLASSIFIABLE — file is outside both prod and test trees (scripts/, docs/, top-level config files, etc.). Skip with a stderr note for manual review.
2.cs.2 Signal G — rule-pattern force-review
For PROD-classified alerts, check whether the rule ID matches RULE_HUMAN_REVIEW_PATTERNS (§0.2). If yes, skip §2.5 and force route to Phase 3.2 with needs-human-review. This is the "high-stakes bug class" carve-out from the design discussion.
RULE_ID=$(jq -r ".[$i].rule.id" "$TMP/alerts.json")
RULE_SEVERITY=$(jq -r ".[$i].rule.severity // .[$i].rule.security_severity_level // \"unknown\"" "$TMP/alerts.json")
if matches_human_review_pattern "$RULE_ID"; then
FORCE_REVIEW="true"
REVIEW_REASON="rule.id matches RULE_HUMAN_REVIEW_PATTERNS — high-stakes bug class, never auto-dismiss"
fi
2.cs.3 Signal H — function-privacy heuristic (input to §2.5)
For PROD-classified alerts that didn't get force-reviewed, walk the AST of the flagged file and find the enclosing function for the flagged line. If the enclosing function's name starts with _ (Python's private-by-convention), that's evidence the flagged code is less likely reachable from a public entry point — feeds into §2.5.6 confidence as a positive signal toward "not-applicable" verdicts.
import ast, pathlib, os, sys
file_path = os.environ["ALERT_FILE"]
target_line = int(os.environ["ALERT_LINE"])
try:
tree = ast.parse(pathlib.Path(file_path).read_text())
except (SyntaxError, UnicodeDecodeError, FileNotFoundError):
print("unknown")
sys.exit()
enclosing = None
for node in ast.walk(tree):
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
start = node.lineno
end = getattr(node, "end_lineno", start)
if start <= target_line <= end:
if enclosing is None or start > enclosing.lineno:
enclosing = node
if enclosing is None:
print("module-level")
elif enclosing.name.startswith("_"):
print("private")
else:
print("public")
PY
The output (public / private / module-level / unknown) feeds into §2.5.6 as one of the confidence-tier inputs. private shifts confidence toward high for not-applicable verdicts (since the code is one layer further from external callers); module-level shifts confidence toward moderate (executes at import time, less constrained); public is neutral.
This is not a reachability proof — a _helper can be called from a public API in the same file. But it's a useful priors signal absent a full call-graph trace.
2.cs.4 Signal I — for code-scanning, §2.5.1 source is the rule itself
For Dependabot, §2.5.1 fetches the GHSA description. For code-scanning, the equivalent advisory text comes directly from the alert payload — no external fetch needed:
| Phase 2.5 input | Dependabot source | Code-scanning source |
|---|
$ADVISORY_DESCRIPTION | gh api /advisories/$GHSA_ID .description | .rule.help // .rule.full_description // .rule.description |
$CWE_IDS | .security_advisory.cwe_ids | .rule.tags filtered for external/cwe/cwe-N patterns, mapped to CWE-N form |
$VULNERABLE_SYMBOL (§2.5.4) | extracted from advisory description | the actual flagged line(s) — pulled from $ALERT_FILE via sed -n "${ALERT_LINE}p" |
$ADVISORY_SUMMARY | .security_advisory.summary | .rule.description // .most_recent_instance.message.text |
§2.5.2 (per-advisory checklist derivation) works the same way — read the rule description, derive Qs against the flagged code. §2.5.4 morphs from "is the vulnerable symbol used anywhere in prod" to "what does the flagged code actually do that triggered this rule" — the answer is already in $ALERT_FILE at $ALERT_LINE, no grep needed.
2.cs.5 Skip §2.5.6b for code-scanning
The structural-impossibility check (§2.5.6b) is Dependabot-specific — it asks "does the framework's public API expose attacker-controlled input to the vulnerable subsystem of a third-party dep?" For code-scanning, the bug is in our code; the structural-impossibility framing doesn't apply. Set STRUCTURAL_IMPOSSIBLE="n/a" for code-scanning alerts so the §2.5.7 action matrix routes through the cli-tool / framework / service rows on confidence alone.
2.5 Exploit-surface analysis — does the CVE's threat model map to this codebase?
This is the most important step the skill performs, and the one most likely to be skipped by a naive "imported in prod → open issue" classifier. A CVE describes a bug under specific preconditions; whether those preconditions are reachable in this codebase is a separate question.
Real failure mode this phase prevents: imagine urllib3 ships a CVE where the streaming-redirect path with proxy auth leaks Authorization headers to the redirect target. The codebase imports urllib3 for a CLI link checker that issues stateless HEAD requests to documented URLs and carries no auth headers. Signal C → PROD hit → issue opened → maintainer comments "this CVE doesn't apply to a link checker" → wasted cycles.
This phase exists to catch that case before the issue is opened.
2.5.1 Fetch the GHSA's actual description
The Dependabot alert payload's .security_advisory.summary is a one-liner. The fuller threat-model description lives in the GHSA database. Pull it:
if ! gh api "/advisories/$GHSA_ID" \
--jq '{summary: .summary, description: .description, severity: .severity, cwe_ids: [.cwe_ids[]?.cwe_id]}' \
> "$TMP/advisory_${GHSA_ID}.json" 2>"$TMP/advisory_${GHSA_ID}.err"; then
echo "ERROR: failed to fetch advisory $GHSA_ID: $(cat "$TMP/advisory_${GHSA_ID}.err" 2>/dev/null)" >&2
rm -f "$TMP/advisory_${GHSA_ID}.json"
SKIPPED+=("$ALERT_NUM:advisory-fetch-failed:$GHSA_ID")
continue
fi
if ! jq -e '.summary != null and ((.description // "") | length) > 0' \
"$TMP/advisory_${GHSA_ID}.json" >/dev/null 2>&1; then
echo "ERROR: advisory $GHSA_ID returned empty/malformed payload — manual review" >&2
rm -f "$TMP/advisory_${GHSA_ID}.json"
SKIPPED+=("$ALERT_NUM:advisory-empty:$GHSA_ID")
continue
fi
Read the description and cwe_ids fields. The advisory description is the primary source of truth for §2.5.2 — it carries the specific threat model for this advisory, which is usually narrower than the CWE class average. The cwe_ids are a hint about the bug class, used as a sanity check (see §2.5.2's reference patterns) and to drive optional MITRE supplementation in §2.5.1b.
2.5.1b MITRE CWE supplementation (optional, sparse-advisory fallback)
Most GHSA descriptions are rich enough to derive a checklist directly (see §2.5.2). When an advisory is sparse — one-line summary, no code samples, no named vulnerable API — supplement with the MITRE CWE definition for additional context:
for CWE_ID in $(jq -r '.cwe_ids[]' "$TMP/advisory_${GHSA_ID}.json"); do
CWE_NUM="${CWE_ID#CWE-}"
curl -sf "https://cwe.mitre.org/data/definitions/${CWE_NUM}.json" \
-o "$TMP/cwe_${CWE_NUM}.json" \
|| echo "WARN: MITRE fetch failed for $CWE_ID — proceeding with advisory text only" >&2
done
When the MITRE fetch succeeds, the fields most useful for §2.5.2 are:
Description / Extended_Description — abstract bug-class descriptionCommon_Consequences — typed list of impacts (Confidentiality, Integrity, Availability, Authentication) which suggests which Qs to askDetection_Methods — hints about what code shapes to grep forPotential_Mitigations — inversion of these often surfaces preconditions
This step is optional and skippable — most advisories don't need it. Add it to the workflow only when §2.5.2's per-advisory derivation can't produce ≥2 testable Qs from the advisory text alone (the trigger for the §2.5.6 sparse-advisory cap).
2.5.2 Derive the CVE's required preconditions
Read the advisory description fetched in §2.5.1 (and optionally §2.5.1b MITRE supplement) and derive the precondition checklist directly from it. The advisory describes the specific bug with its specific preconditions — that's the source of truth, not a static class-average mapping.
Structure the output as Q1, Q2, … so the audit trail has stable referents. The operator (running this skill via Claude Code, which is the supported mode) reads the advisory text and produces:
- A list of preconditions the attacker needs, extracted from phrases like "An attacker can …", "requires …", "when the application …", "affects code that …" in the advisory description.
- A list of named vulnerable symbols for §2.5.4 (class.method, function names, kwargs, flags — these are typically formatted in backticks or code blocks in the advisory).
- For each precondition, a testable Q phrased against the calling code: "Does the calling code do X?" Reword the precondition into a question the §2.5.3 step can answer
reachable / absent / unknown against this specific codebase.
The questions should be as narrow as the advisory allows. If the advisory names ProxyManager.connection_from_url(...).urlopen(..., assert_same_host=False), don't ask the abstract Q "does the code carry sensitive headers" — ask the sharper "does the code call ProxyManager.connection_from_url with assert_same_host=False and any sensitive header." Sharper Qs produce sharper verdicts.
Write each Q&A answer down. The audit trail must include the derived Q list verbatim — a future reviewer needs to know what was asked, not just the verdict.
Reference patterns (use as a sanity check that you've covered the standard preconditions for the bug class — NOT as a dispatch table). Common CWE classes and the kinds of Qs they typically require:
| CWE ID(s) | Bug class | Typical question shape |
|---|
| CWE-200, CWE-201, CWE-209 | Information / header / token leak | Does calling code carry sensitive state? Does the leak path require attacker-controlled destination? Can the attacker influence that destination? |
| CWE-22, CWE-23 | Path traversal | Does calling code pass external/attacker-controlled paths to the vulnerable API? |
| CWE-78, CWE-77, CWE-88 | Command / argument injection | Does calling code pass external args to subprocess / shell? Is the vulnerable spawn helper invoked? |
| CWE-89 | SQL injection | Does calling code build queries from external strings? Is the vulnerable codepath actually invoked (raw-SQL vs ORM-only)? |
| CWE-94, CWE-502, CWE-915 | Code injection / deserialization RCE | Does calling code deserialize / eval external bytes? Is the unsafe loader invoked, vs the safe variant? |
| CWE-295, CWE-297, CWE-345, CWE-322 | TLS / cert / signature bypass | Does calling code make outbound TLS to non-internal hosts? Is the trust decision based on the vulnerable verification path? Are hostnames user-controlled? |
| CWE-352 | CSRF | Is there a browser-driven request flow? Does the vulnerable endpoint actually exist in our service? |
| CWE-400, CWE-409, CWE-770, CWE-789, CWE-1333 | DoS — memory / regex / compression | Does calling code accept attacker-controlled bytes of arbitrary length? Uptime contract — long-lived service vs one-shot CLI? |
| CWE-918 | SSRF | Does calling code fetch URLs with externally-supplied hosts? Is the deployment environment sensitive to internal-network access? |
| CWE-601 | Open redirect | Does calling code expose redirect targets to user input? Is this a web frontend holding session state? |
| CWE-327, CWE-328, CWE-326, CWE-916 | Weak crypto | Does calling code use the weak primitive for a security-sensitive purpose? Are keys/parameters under our control? |
| CWE-862, CWE-863 | Missing / incorrect authz | Does calling code expose any endpoint gated by the vulnerable check? |
| CWE-203 | Observable / timing discrepancy | Is the vulnerable comparison used in a security-sensitive context? Is timing exploitable across many requests? |
| CWE-674 | Uncontrolled recursion (distinct from CWE-1333 ReDoS) | Does calling code accept attacker-controlled depth-bearing input? Uptime contract? |
| CWE-732 | Incorrect permission assignment | Does calling code create files on a shared filesystem via the vulnerable API? Are files sensitive? |
| CWE-377, CWE-378, CWE-379 | Insecure temp file | Does calling code use vulnerable temp-file API in a shared-tmp context? Are race windows exploitable in practice? |
| CWE-426, CWE-427 | Untrusted search path | Does the process spawn subprocesses / load libs from PATH-resolved names? Is PATH attacker-controllable? |
| CWE-79 | XSS | Does calling code render the vulnerable template / sink to a browser? Is data piped into the sink external? |
| CWE-20 | Improper input validation (class-wide) | Does calling code pass external input to the vulnerable API? Downstream consequence + caller-side shielding? |
The above is a hint about which question shapes typically apply for a CWE class — use it to verify your derived Qs aren't missing a standard precondition. Always derive from the advisory text; never use this table as the sole source.
Why per-advisory derivation beats a static dispatch: the same CWE class can describe wildly different bugs. CWE-22 might be "path traversal in clone_from clone target" in one advisory and "path traversal in os.path.join with .. segments" in another — the preconditions differ. A static dispatch gives you the average; the advisory text gives you the specific. For an autonomous-dismissal decision, specific wins.
2.5.3 Map preconditions against the calling code
For each PROD import path found in Signal C, read the calling file (or the relevant function) and answer:
- Does the calling code carry the sensitive state the CVE targets? (auth headers? session cookies? privileged file paths? OAuth state?)
- Does the calling code accept input from an untrusted source? (network user? HTTP request body? environment variable populated by an external system?) Or is the input developer-controlled (repo source, hardcoded URL, deterministic config)?
- Is the calling process long-lived / network-reachable, or one-shot CLI?
Cross-reference the answers against the checklist Qs from §2.5.2. For each Q, record one of: reachable (precondition satisfied by the calling code), absent (precondition cannot be satisfied given the calling code), or unknown (cannot determine without runtime trace or deeper analysis). If any required precondition is absent, the CVE is candidate not applicable. If any is unknown, the analysis is candidate moderate confidence at best (see §2.5.6).
2.5.4 Vulnerable-symbol call-graph trace (one level)
Signal C in §2.4 confirms the package is imported. That's necessary but not sufficient — the specific vulnerable API named in the advisory must also be reachable. Many CVEs name a concrete function/method/flag (ProxyManager.connection_from_url, assert_hostname, yaml.unsafe_load, Repo.clone_from(multi_options=...)). If the calling code uses the package but never reaches the vulnerable symbol, the CVE is not exploitable in this codebase even though Signal C lit up.
This is a one-level static grep — not full call-graph analysis. It catches the common case where the calling code uses a different entrypoint of the same package.
Step 1 — extract the vulnerable symbol(s) from the advisory description. Look for class.method, function names, kwargs, and CLI flags:
jq -r '.description' "$TMP/advisory_${GHSA_ID}.json" \
| grep -oE '`[^`]+`|\b[A-Z][A-Za-z0-9_]*\.[a-z_][A-Za-z0-9_]*\b|\b[a-z_][A-Za-z0-9_]+\(' \
| sort -u
Step 2 — grep production code for each candidate symbol. Use the same PROD_DIRS array machinery as §2.4 (quoted-each-element expansion, /build/ and /tests/ filtered):
SYM_RE=$(printf '%s' "$SYM" | sed 's/[][\\/.*^$+?{}()|]/\\&/g')
SYMBOL_HITS=$(grep -rlnE "$SYM_RE" "${PROD_DIRS[@]}" --include="*.py" 2>/dev/null \
| grep -v "/build/" | grep -v "/tests/" \
| wc -l | tr -d ' ')
Step 3 — verdict feeds into §2.5.6 confidence:
| Vulnerable symbol in advisory? | SYMBOL_HITS | Effect |
|---|
| Yes, specific symbol named | > 0 | Strong evidence CVE is reachable — confidence shifts toward high for an "applicable" verdict |
| Yes, specific symbol named | 0 | Strong evidence CVE is NOT reachable — confidence shifts toward high for a "not-applicable" verdict (this is the §2.5.3 absent case backed by code evidence, not just inference) |
No specific symbol named (CVE is class-wide — e.g. "any use of requests with proxies") | n/a | Skip this step; confidence ceiling is moderate at best because there's no symbol to grep for |
| Symbol named but obscured by aliasing / dynamic dispatch | n/a | Mark as unknown for §2.5.3 — confidence drops one tier |
This step is the difference between dismissing as not_used (Phase 2.4 found no imports anywhere) and dismissing as inaccurate with high confidence (Phase 2.4 found imports, but Phase 2.5.4 confirmed the vulnerable surface is unreached).
Known limitations — these are the cases where the grep is insufficient and confidence MUST drop:
- Dynamic dispatch —
getattr(obj, method_name) where method_name is computed at runtime. Grep won't see it.
- Framework abstraction — the calling code goes through a framework's HTTP transport (e.g.
web3.HTTPProvider → requests → urllib3); the vulnerable symbol is reached internally, not by direct call. Grep at this level won't see it.
- Aliased imports —
from urllib3 import ProxyManager as PM then PM.connection_from_url(...). The earlier import-graph scan catches the import; this step won't catch the use unless the alias is grepped too.
When any of these patterns is plausible given the package's role, drop the confidence tier and mark the precondition unknown in §2.5.3.
2.5.5 The archetype multiplier
Apply the ARCHETYPE value from Phase 0.1:
| Archetype | Default exploit-surface posture |
|---|
cli-tool | Strict — CVEs requiring browser sessions, long-running listeners, untrusted-remote inputs, server-side state are presumed not-applicable. Memory-DoS CVEs are not-applicable unless uptime is a contractual concern. The CVE must show a concrete exploit chain that fits the CLI usage to count as applicable. |
framework / scaffold | Permissive — consumers may use the framework in any archetype, so a CVE that's exploitable in some downstream consumer is applicable here. Default to PROD-applicable unless the CVE is structurally impossible (e.g. an auth CVE in a library that has no auth code at all). |
service | Permissive — long-running, network-reachable, often handles user input. Default to PROD-applicable. |
unknown | Treat as framework. |
2.5.6 Confidence tier
Derive a confidence tier in {high, moderate, low} from the §2.5.3 precondition map AND the §2.5.4 vulnerable-symbol trace. The tier is what gates dismissal in Phase 3 — inaccurate dismissals require high confidence.
| Confidence | All of these must hold |
|---|
| high | Every Q derived in §2.5.2 was answered reachable or absent (no unknown) AND the §2.5.4 vulnerable-symbol trace produced a definitive yes/no (specific symbol named in advisory + grep returned 0 or >0 hits, no aliasing / framework-internal / dynamic-dispatch caveats applied) AND the archetype is unambiguous from §0.1 heuristics AND §2.5.2 produced ≥2 testable Qs from the advisory (sparse-advisory check) |
| moderate | At least one of the above conditions is borderline: one Q is unknown, OR the advisory names the CVE class-wide with no specific symbol to grep for, OR the archetype detection landed on unknown and defaulted to framework, OR the advisory was too sparse to derive ≥2 testable Qs even with the §2.5.1b MITRE supplement |
| low | Multiple Qs are unknown, OR aliasing/framework-internal dispatch is plausible AND the symbol grep returned 0 hits (false-negative risk), OR the calling code reaches the package through a third-party transport whose behavior isn't audited in this repo |
Two heuristics worth applying mechanically:
- If §2.5.2 derived ≥2 specific testable Qs from the advisory AND §2.5.4 produced a definitive symbol-trace result AND the archetype is
cli-tool or service (unambiguous), default to high.
- If §2.5.2 could only derive ≤1 Q (sparse advisory) OR the archetype is
unknown, ceiling is moderate regardless of how clean the other signals are. The sparse-advisory case is the right signal that we don't know enough to dismiss autonomously — when even MITRE supplementation (§2.5.1b) can't produce a checklist, route to needs-human-review rather than guess.
2.5.6b Structural impossibility check (frameworks / scaffolds only)
The §2.5.5 archetype rule defaults framework and scaffold to "permissive" — consumers may use the framework in ways this repo can't anticipate, so default to PROD-APPLICABLE. But §2.5.5's prose carries an escape clause: "unless the CVE is structurally impossible (e.g. an auth CVE in a library that has no auth code at all)." Without operationalizing it, the matrix routes every framework + precondition-absent case to a needs-human-review issue — which produces issue spam for cases that are genuinely autonomous-dismissible (e.g. a GitPython CVE in a framework that only calls Repo(local_path) and exposes no clone API).
This step makes the escape clause operational. For framework / scaffold archetypes, before falling through to the permissive default in §2.5.7, check whether all three conditions hold:
-
Symbol grep was definitive (§2.5.4) — the advisory named a specific vulnerable symbol AND the grep returned exactly 0 hits in prod code, with no aliasing / dynamic-dispatch / framework-internal caveats applied. If the advisory is class-wide ("any use of requests with proxies"), this condition fails — there's no symbol to grep for, so structural impossibility can't be proven.
-
Existing import sites are orthogonal in usage — walk every prod file from §2.4 Signal C and read what the imported symbols are actually used for. Classify each call site as:
orthogonal — the imported symbols are used only for purposes the advisory doesn't name. Examples: importing a class but using only its constructor / read-only methods / non-vulnerable methods (e.g. Repo(local_path) for HEAD/working-tree reads when the CVE is in Repo.clone_from); importing exception classes; importing types/constants.vulnerable-adjacent — the call site actually invokes a method/attribute named by the advisory, OR uses the symbol in a way that could reach the vulnerable code path under normal control flow (e.g. importing PoolManager and calling .request() when the CVE is in the high-level request path).
All Signal C call sites must be orthogonal. If any is vulnerable-adjacent, the check fails — the existing usage is at least one method-call away from the CVE, and a small caller change could complete the path.
Note: this is a stricter test than §2.5.4 alone. §2.5.4 says "the vulnerable symbol isn't called right now"; condition 2 here says "and the framework isn't using the same subsystem in a way that's one step away from calling it."
-
No public API forwards external inputs to the vulnerable subsystem — enumerate the framework's public API and ask whether any parameter could plausibly become the vulnerable kwarg/argument named in the advisory.
Enumerate via AST, not grep — def name(self, url, ...) with the body using urllib3.PoolManager(...) is the kind of call site that condition 3 cares about, and grep can't reliably extract parameter names from formatted multi-line def signatures. Run this once per skill invocation, cache the result:
import ast, pathlib, json, os
prod_dirs = [
d for d in pathlib.Path(os.environ["TMP"], "prod_dirs.txt")
.read_text().splitlines()
if d
]
public_apis: list[dict] = []
for d in prod_dirs:
root = pathlib.Path(d)
if not root.exists():
continue
for p in root.rglob("*.py"):
parts = str(p).split("/")
if "tests" in parts or "build" in parts:
continue
try:
tree = ast.parse(p.read_text())
except (SyntaxError, UnicodeDecodeError):
continue
for node in ast.walk(tree):
if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
continue
if node.name.startswith("_"):
continue
params = [a.arg for a in node.args.args if a.arg != "self"]
kwonly = [a.arg for a in node.args.kwonlyargs]
public_apis.append({
"file": str(p), "line": node.lineno, "name": node.name,
"params": params + kwonly,
})
print(json.dumps(public_apis))
PY
Match against the advisory's vulnerable inputs. Use the vulnerable-symbol extraction from §2.5.4 step 1 — it already produces a candidate list (function names, kwargs, flags). For each public API in the AST output, check whether any of its parameter names fuzzy-match the advisory's vulnerable inputs. A match is one of:
- exact name (
proxy, url, headers, cookies, target_path, multi_options)
- common alias (
url matches target_url / endpoint / remote; path matches file_path / dest; options matches opts / kwargs)
- the parameter is typed as
**kwargs AND the calling code passes through to the vulnerable API (one-step grep)
If zero matches → condition 3 passes (no public API exposes the vulnerable input). If any match → condition 3 fails — the public API plausibly forwards external input to the vulnerable subsystem, route to needs-human-review.
This is intentionally permissive on the match side (treats common-alias as a hit) so we err toward human review when the surface is ambiguous, but conservative on the enumeration side (only def at module top-level or class methods, ignoring private _name and dunders).
Examples:
def make_request(self, url, headers=None, proxy=None) in a framework + urllib3 proxy/header CVE → params [url, headers, proxy] match url + proxy + headers → fail.def bump_version(self, package: str, version: str) in aea-dev-helpers + GitPython clone CVE → params [package, version] match nothing in [remote, url, path, clone_target, multi_options] → pass.def read_config(self, path: str) in a framework + GitPython path-traversal CVE → param path matches path → fail (even if the function never touches git, the parameter shape is dangerous when combined with the imported Repo class).
When all three conditions hold, the framework's code structurally cannot reach the vulnerable surface AND doesn't expose a public API a consumer could exploit. Mark the verdict NOT-APPLICABLE (structural) — this is high-confidence on par with cli-tool dismissal, eligible for autonomous inaccurate dismissal.
Worked example — passes: open-aea + GitPython CVEs (CWE-20, CWE-94, CWE-22 on clone_from / multi_options). Open-aea imports from git import Repo only in aea-dev-helpers/bump_version.py, uses it as Repo(local_path) to read the working tree, exposes no public API that takes a clone URL or path. (1) symbol grep for clone_from / multi_options = 0. (2) all imports are read-only Repo operations, orthogonal to the clone subsystem. (3) public API of aea-dev-helpers is a CLI bump-version tool — no parameter could become a clone URL. → all three conditions hold → autonomous dismiss.
Worked example — fails: a framework that imports from urllib3 import PoolManager and exposes def make_request(url, headers, proxy=None). Even if the framework's own callsites don't trigger the CVE, condition (3) fails — the public API forwards user-controlled inputs into the vulnerable subsystem. Must route to needs-human-review issue.
The cli-tool archetype doesn't need this check — it already gets autonomous dismissal under §2.5.7 directly. The service archetype also doesn't get this check — services ARE the consumer (long-running, real attack surface), so framework-style structural-impossibility doesn't apply.
2.5.7 Final verdict + action matrix
Combine §2.5.3 (preconditions match), §2.5.5 (archetype posture), §2.5.6 (confidence tier), and §2.5.6b (structural-impossibility check for frameworks) into one action call. The action matrix is conservative on the dismissal side and permissive on the issue side — false-positive issues are cheap, false-negative dismissals are expensive. But for framework / scaffold archetypes, §2.5.6b lets the skill bypass the "default to caution" rule when structural impossibility is provable, so we don't drown the human reviewer in spurious issues.
Dependabot action matrix (unchanged from prior version):
| Preconditions | Archetype | Confidence | §2.5.6b | Verdict | Action (Dependabot) |
|---|
All reachable | any | high or moderate | n/a | PROD-APPLICABLE | Open issue (Phase 3.2). If confidence = moderate, additionally label needs-human-review. |
All reachable | any | low | n/a | PROD-APPLICABLE | Open issue + needs-human-review label. Body must call out the uncertainty source(s). |
Any absent | cli-tool | high | n/a | NOT-APPLICABLE | Dismiss with inaccurate (Phase 3.1b). Comment names which Q failed and references the §2.5.4 symbol-grep result. |
Any absent | cli-tool | moderate / low | n/a | NOT-APPLICABLE (low-conf) | Open issue + needs-human-review label. Do NOT dismiss — the maintainer decides. |
Any absent | framework / scaffold | high | passes | NOT-APPLICABLE (structural) | Dismiss with inaccurate (Phase 3.1b). Comment names which §2.5.6b condition was decisive — typically the missing public-API surface or the orthogonal import pattern. |
Any absent | framework / scaffold | high | fails | PROD-APPLICABLE | Open issue + needs-human-review. Body explains which §2.5.6b condition failed (e.g. "framework exposes make_request(url, headers, proxy) — consumer might satisfy CWE-200 Q1+Q3"). |
Any absent | framework / scaffold | moderate / low | any | PROD-APPLICABLE | Open issue + needs-human-review. The signals aren't reliable enough for structural-impossibility to be safe. |
Any absent | service | any | n/a | PROD-APPLICABLE | Open issue. Services are long-running and themselves the consumer — structural impossibility doesn't apply. |
Any unknown | any | any | n/a | PROD-APPLICABLE | Open issue + needs-human-review. Do NOT dismiss. |
Code-scanning action matrix (new):
| Signal F (file location) | Signal G (rule pattern) | Preconditions | Archetype | Confidence | Verdict | Action (Code Scanning) |
|---|
| in tests/ | any | n/a | any | n/a | TEST-ONLY | Dismiss with used in tests (Phase 3.1d). |
| in prod | force-review | n/a | any | n/a | NEEDS-HUMAN-REVIEW | Open issue + needs-human-review. NEVER auto-dismiss this rule class. |
| in prod | normal | All reachable | any | any | PROD-APPLICABLE | Open issue. If confidence != high, additionally label needs-human-review. |
| in prod | normal | Any absent | cli-tool | high | FALSE-POSITIVE | Dismiss with false positive (Phase 3.1e). Comment names the failing Q + Signal H privacy hint. |
| in prod | normal | Any absent | cli-tool | moderate / low | NOT-APPLICABLE (low-conf) | Open issue + needs-human-review. |
| in prod | normal | Any absent | framework / scaffold | any | PROD-APPLICABLE | Open issue + needs-human-review. Consumers may use the framework's public API in ways that hit the flagged path. |
| in prod | normal | Any absent | service | any | PROD-APPLICABLE | Open issue. Services are themselves the consumer; conservative-by-default. |
| in prod | normal | Any unknown | any | any | PROD-APPLICABLE | Open issue + needs-human-review. |
| unclassifiable file location | any | n/a | any | n/a | SKIP | stderr line for manual review — flagged file outside both prod and test trees. |
The two matrices share the same Phase 2.5 confidence-tier logic. The Dependabot matrix has the extra §2.5.6b structural-impossibility column because consumer-context unknowability matters when the bug is in a third-party dep. The code-scanning matrix is stricter on framework auto-dismissal — the bug is in our own code, so "consumer might hit this" is even more relevant than in the Dependabot case; no autonomous dismissal for framework/scaffold code-scanning alerts.
When dismissing, the dismissed_comment must name:
- Dependabot
inaccurate: which checklist Q failed + (framework/scaffold) which §2.5.6b condition was decisive.
- Dependabot
not_used: where the test/dev imports were found.
- Code-scanning
false positive: which Q failed + Signal H result (function privacy) + reference to the audit issue.
- Code-scanning
used in tests: the test path the alert fired against.
Examples:
- Dependabot cli-tool: "Tomte's link checker carries no auth headers — CWE-200 Q1 absent. §2.5.4 grep for
connection_from_url = 0 hits."
- Dependabot framework structural: "Open-aea: GitPython CVE not reachable. §2.5.6b — only
Repo(local_path) read usage in bump_version.py (orthogonal), no public API forwards URLs/paths to clone subsystem."
- Code-scanning cli-tool false-positive: "
python/HardcodedNonCryptoSecret in tomte/cli.py:142 — the 'secret' is a deterministic content hash (Q1 absent); function is _compute_pkg_hash (private). Not exploitable."
- Code-scanning used-in-tests: "
python/reDOS in tests/test_url_parsing.py:88 — test fixture for a parser; the regex never sees attacker input at runtime."
That comment is the audit trail; a future reviewer needs enough information to challenge the call without re-running the analysis from scratch.
2.5.8 Out of scope: PoC harness
The gold-standard verification of "is this CVE exploitable in this codebase" is to run the public exploit PoC against the actual codebase. This skill does not automate that for three reasons:
- PoCs are CVE-specific and hand-crafted; they don't generalize into a harness.
- The PoC may be destructive (RCE, data exfiltration) — running it in a developer environment is not safe by default.
- The threat model is "exploitable from where the calling code sits," not "the bug still exists upstream" — the PoC tests the latter.
If a moderate or low-confidence dismissal is challenged by the maintainer, PoC verification is the documented escalation path. Note that in the needs-human-review issue body so the maintainer knows what to do next.
2.6 Transitive deps — reverse-resolve the chain
For transitive deps (scope == "unknown" or no pin in pyproject.toml), figure out which direct deps pull the vulnerable package in:
uv tree --package "$PKG" --invert 2>/dev/null
poetry show --why "$PKG" 2>/dev/null
If every reverse-dep root is a dev/test group root, mark DEV. If any reverse-dep root is in [project] dependencies (prod), mark PROD. This is the right call even if production code itself never does import <transitive> — the transitive is in the production install footprint and exploitable at runtime.
Phase 3 — Act on the classifications
For every alert, take exactly one action: dismiss, open issue, or skip. Build a per-alert audit record so the Phase 4 summary is honest about what happened.
When MODE=rerun-dismissed, Phase 3.0 runs instead of 3.1 / 3.1b / 3.2 / 3.3 — the skill produces a report comparing the new verdict to the recorded dismissed_reason for each alert, and takes no actions. Use this after a §2.5 logic change to surface previously-dismissed alerts whose verdict would now differ.
3.0 Rerun-dismissed report mode (read-only)
Walks already-dismissed alerts and reports verdict drift between the current skill logic and the historical dismissal. No gh api PATCH, no gh issue create — the only mutations are to stdout / the report file.
RECORDED_REASON=$(jq -r ".[$i].dismissed_reason" "$TMP/alerts.json")
RECORDED_COMMENT=$(jq -r ".[$i].dismissed_comment // \"\"" "$TMP/alerts.json")
if [[ "$VERDICT" == "DEV" ]]; then
NEW_REASON="not_used"
elif [[ "$VERDICT" == "NOT-APPLICABLE" || "$VERDICT" == "NOT-APPLICABLE (structural)" ]] \
&& [[ "$CONFIDENCE" == "high" ]] \
&& { [[ "$ARCHETYPE" == "cli-tool" ]] || [[ "$STRUCTURAL_IMPOSSIBLE" == "true" ]]; }; then
NEW_REASON="inaccurate"
else
NEW_REASON="open-issue"
fi
case "${RECORDED_REASON}:${NEW_REASON}" in
"${RECORDED_REASON}:${RECORDED_REASON}")
AGREE+=("$ALERT_NUM $PKG $GHSA_ID ($RECORDED_REASON)") ;;
"not_used:inaccurate" | "inaccurate:not_used")
REFINE+=("$ALERT_NUM $PKG $GHSA_ID (was=$RECORDED_REASON now=$NEW_REASON — same action, different reason)") ;;
*":open-issue")
DRIFT+=("$ALERT_NUM $PKG $GHSA_ID (was=$RECORDED_REASON now=open-issue — skill would NO LONGER dismiss; consider reopening manually if applicable)") ;;
*)
OTHER+=("$ALERT_NUM $PKG $GHSA_ID (was=$RECORDED_REASON now=$NEW_REASON)") ;;
esac
End of run, print:
=== triage-security rerun-dismissed report for $REPO (Dependabot-only) ===
Alerts re-evaluated: $N_PROCESS
Agree (skill still endorses the dismissal): ${#AGREE[@]}
Refine (same action, different reason — informational): ${#REFINE[@]}
DRIFT (skill would no longer dismiss): ${#DRIFT[@]}
Other (verdict swap, e.g. not_used → inaccurate or vice versa): ${#OTHER[@]}
DRIFT — review these manually:
...
Other — review these manually:
...
Never auto-reopen a dismissed alert in this mode. A human dismissal is a human decision; the skill's job is to surface drift, not override the call. If DRIFT is non-empty, the operator reviews each entry and decides whether to manually reopen the alert via the GitHub UI or via gh api -X PATCH ... -f state=open.
Exit code in rerun mode: 0 if DRIFT is empty (skill agrees with all historical dismissals), 1 otherwise.
