メニュー
Validate AI/ML models and datasets for bias, fairness, and ethical concerns. Use when auditing AI systems for ethical compliance, fairness assessment, or bias detection. Trigger with phrases like "evaluate model fairness", "check for bias", or "validate AI ethics".
Audit a Node.js project's installed npm dependency tree for known CVEs by wrapping the npm audit JSON output and emitting findings in the canonical penetration-tester schema. Detects direct AND transitive vulnerabilities, normalizes npm's severity scale (info/low/moderate/ high/critical) to the shared Severity enum, and parses both v1 and v2 audit output formats so the skill works against npm 6 and npm 7+ lockfiles. Use when: pre-merge gate on a Node project, post-incident sweep after a transitive package compromise (e.g. event-stream, ua-parser, node-ipc, color.js), SOC2 vendor-management evidence collection, or auditing an inherited or acquired Node codebase. Threshold: any HIGH or CRITICAL CVE in the resolved dependency tree. MODERATE / LOW reported informationally. Trigger with: "audit npm deps", "npm vulnerability scan", "check node packages for CVEs", "npm audit".
Audit a Python project's installed dependencies for known CVEs by wrapping pip-audit (PyPA's official vulnerability auditor) and emitting findings in the canonical penetration-tester schema. Detects vulnerable direct AND transitive packages, normalizes pip-audit's severity output via OSV severity bands, falls back to pip list --outdated when pip-audit isn't installed, and supports requirements.txt, pyproject.toml (PEP 621), Pipfile.lock, and poetry.lock as input sources. Use when: pre-merge gate on a Python project, post-incident sweep after a PyPI compromise (e.g. ctx, request-toolbelt typosquats, ultralytics 8.3.42 compromise), SOC2 evidence collection, or inheriting an unfamiliar Python codebase. Threshold: any HIGH or CRITICAL CVE in the resolved dependency tree. MODERATE / LOW reported informationally. Trigger with: "audit python deps", "pip vulnerability scan", "check pypi packages for CVEs", "pip-audit run".
Audit a project's dependency licenses against an explicit policy (allow-list / deny-list / review-required) and flag incompatibilities before they ship to production. Reads SPDX license identifiers from npm package manifests, Python METADATA / PKG-INFO files, and pyproject.toml; classifies each license by family (permissive, weak-copyleft, strong-copyleft, proprietary, unknown); detects copyleft contamination and SPDX-incompatible license combinations. Use when: pre-release legal review, M&A code-audit due diligence, preparing an OSS attribution NOTICE file, or switching a project's own license. Threshold: any GPL-family license in a project declaring MIT or Apache-2.0; any UNKNOWN-license package; any metadata-vs-source license mismatch. Trigger with: "check licenses", "license compliance audit", "SPDX scan", "GPL contamination check".
Read findings JSONL files from cluster 1-4 skills, deduplicate by fingerprint, group by severity, and compose a deliverable- grade markdown vulnerability report with per-finding sections (title, severity, target, detail, remediation, evidence) and a top-level summary table. The canonical written artifact a customer receives at engagement close; precise, reproducible, machine- checkable against source findings. Use when: closing an engagement, generating an interim report, regenerating after CVE or OWASP enrichment, or producing the input for generating-executive-summary. Threshold: findings missing required fields are dropped. HIGH and CRITICAL findings highlighted in the summary section. Trigger with: "compose vuln report", "write pentest report", "generate vulnerability deliverable", "render findings to report".
Verify that a penetration test has explicit, written, signed authorization before any scanning begins. Reads a Rules-of- Engagement (ROE) attestation file, validates required fields (authorizer, in-scope targets, time window, emergency contact, signature), checks the signer against an allowlist, and emits a CRITICAL finding if anything is missing. Designed as the first skill the orchestrator routes to. Use when: starting a new engagement, after a scope change, or before any cluster 1-4 scan skill runs. Threshold: any missing or unsigned ROE field; any time-window expiry; any in-scope target outside the authorized list. Trigger with: "confirm authorization", "verify ROE", "check pentest authz", "pre-flight authorization".
Parse the ROE scope definition, enumerate every in-scope target (hostnames, IPs, CIDRs, URLs, cloud accounts, SaaS tenants), validate syntax, detect overlap with out-of-scope or known third-party SaaS ranges, and emit a normalized target list plus IP allowlist for scanning tools. Runs after confirming-pentest- authorization and before any cluster 1-4 scan. Use when: starting an engagement, expanding scope mid-engagement, validating that a target list matches the ROE, or generating an allowlist for an external scanner. Threshold: malformed syntax, in-scope overlap with out-of-scope, reserved or third-party SaaS ranges without acknowledgement. Trigger with: "define scope", "enumerate targets", "validate target list", "generate IP allowlist".
| name | validating-ai-ethics-and-fairness |
| description | Validate AI/ML models and datasets for bias, fairness, and ethical concerns. Use when auditing AI systems for ethical compliance, fairness assessment, or bias detection. Trigger with phrases like "evaluate model fairness", "check for bias", or "validate AI ethics". |
| allowed-tools | Read, Write, Edit, Grep, Glob, Bash(python:*) |
| version | 1.0.0 |
| author | Jeremy Longshore <jeremy@intentsolutions.io> |
| license | MIT |
| tags | ["ai","compliance","audit"] |
| compatibility | Designed for Claude Code, also compatible with Codex and OpenClaw |
Validate AI/ML models and datasets for bias, fairness, and ethical compliance using quantitative fairness metrics and structured audit workflows.
pip install fairlearn)pip install aif360) for comprehensive bias analysisExponentiatedGradient from Fairlearn)| Error | Cause | Solution |
|---|---|---|
| Insufficient group sample size | Fewer than 30 observations in a demographic group | Aggregate related subgroups; use bootstrap confidence intervals; flag metric as unreliable |
| Missing sensitive attributes | Protected attribute columns absent from dataset | Apply proxy detection via correlated features; request attribute access under data governance approval |
| Conflicting fairness criteria | Demographic parity and equalized odds contradict | Document the impossibility theorem trade-off; prioritize the metric most aligned with the deployment context |
| Data quality failures | Inconsistent encoding or null values in attribute columns | Standardize categorical encodings; impute or exclude nulls; validate with schema checks before analysis |
| Model output format mismatch | Predictions not in expected probability or binary format | Convert logits to probabilities via sigmoid; binarize at the decision threshold before metric computation |
Scenario 1: Hiring Model Audit -- Validate a resume-screening classifier for gender and age bias. Compute demographic parity across male/female groups and age buckets (18-30, 31-50, 51+). Apply the four-fifths rule. Finding: female selection rate at 0.72 of male rate (critical severity). Recommend reweighting training samples and adjusting the decision threshold.
Scenario 2: Credit Scoring Fairness -- Assess a credit approval model for racial disparate impact. Calculate equalized odds (TPR and FPR) across racial groups. Finding: FPR for Group A is 2.1x Group B (high severity). Recommend in-processing constraint using ExponentiatedGradient with FalsePositiveRateParity.
Scenario 3: Healthcare Risk Prediction -- Evaluate a patient risk model for age and socioeconomic bias. Compute calibration curves per group. Finding: model overestimates risk for low-income patients by 15%. Recommend recalibration using Platt scaling per subgroup with post-deployment monitoring for fairness drift.