security-metrics

name	security-metrics
description	Define and track security programme metrics to measure effectiveness and communicate risk. Outputs KPI definitions, measurement methodology, dashboard design, and reporting cadence.
argument-hint	["security programme maturity","audience","compliance requirements","available tooling"]
allowed-tools	Read, Write

Security Metrics

Security metrics make the invisible visible — they turn security activities into data that leadership can act on and compare over time. The challenge is measuring outcomes (are we more secure?) not just activities (how many patches did we apply?).

Metrics Framework

## Tier 1: Operational Metrics (daily/weekly)
These tell you if security controls are functioning.

| Metric | Target | Measurement |
|--------|--------|-------------|
| Mean time to patch (Critical CVE) | <24 hours | Vuln scanner → patch timestamp |
| Mean time to patch (High CVE) | <7 days | Vuln scanner → patch timestamp |
| Critical CVEs unpatched >24h | 0 | Vuln scanner count |
| MFA adoption rate | 100% | IAM system |
| Certificate expiry warnings | 0 | Certificate monitoring |
| Failed login attempt rate | <5% | Auth system |

## Tier 2: Programme Metrics (monthly)
These tell you if the programme is improving.

| Metric | Target | Measurement |
|--------|--------|-------------|
| Security findings in prod (post-deploy) | <5% of total | Incident tagging |
| % code with SAST scan | >95% | CI/CD pipeline |
| Security champions coverage | 100% of teams | Programme tracking |
| Penetration test findings resolved | >90% in 90 days | Pentest tracker |
| Security debt backlog size | Declining | JIRA/tracker |

## Tier 3: Risk Metrics (quarterly)
These tell you what the residual risk profile looks like.

| Metric | Target | Measurement |
|--------|--------|-------------|
| CVSS severity distribution | Trend downward | Vuln scanner |
| Third-party risk coverage | >90% vendors assessed | Vendor risk programme |
| Incident severity distribution | More P3/P4, fewer P1/P2 | Incident management |
| Attack surface reduction | Quarterly decrease | Asset inventory |

Metrics Collection

import boto3
from datetime import datetime, timedelta
import json

class SecurityMetricsCollector:
    def collect_vulnerability_metrics(self) -> dict:
        """Collect from your vuln scanner (Qualys, Tenable, Snyk)."""
        # Example: AWS Inspector findings
        inspector = boto3.client("inspector2")
        
        findings = inspector.list_findings(
            filterCriteria={
                "severity": [{"comparison": "EQUALS", "value": "CRITICAL"}],
                "findingStatus": [{"comparison": "EQUALS", "value": "ACTIVE"}],
            }
        )["findings"]
        
        now = datetime.utcnow()
        critical_sla_breach = sum(
            1 for f in findings
            if (now - f["firstObservedAt"].replace(tzinfo=None)).total_seconds() > 86400
        )
        
        return {
            "critical_open": len(findings),
            "critical_sla_breach": critical_sla_breach,
            "critical_sla_compliance": f"{100 - 100*critical_sla_breach/max(len(findings),1):.1f}%"
        }
    
     () -> :
        
        iam = boto3.client()
        credential_report = ._get_credential_report(iam)
        
        total = (credential_report)
        mfa_enabled = (  u  credential_report  u[] == )
        
         {
            : total,
            : mfa_enabled,
            : ,
            : [u[]  u  credential_report  u[] == ]
        }

Security Dashboard Template

# Security Dashboard — March 2024

## Executive Summary
Security posture: 🟡 IMPROVING
- 2 critical findings resolved this week (↓ from 4 last week)
- MFA adoption: 98.2% (↑ 1.4pp from last month)
- No P1 incidents in 47 days

## Vulnerability Management
| Severity | Open | SLA Compliant | Avg Age (days) |
|----------|------|---------------|----------------|
| Critical | 1 | 0% ⚠ | 28 |
| High | 12 | 83% | 4.2 |
| Medium | 67 | 91% | 12.1 |

## Identity & Access
- MFA: 98.2% (2 users non-compliant — escalated)
- Privileged access reviews: 100% complete this quarter
- Orphaned accounts: 0 (last review: 2024-03-01)

## Security Debt
- Open security findings in backlog: 23 (↓ from 31)
- P1 security debt items: 0
- P2 security debt items: 3 (due Q2)

Anti-Patterns to Avoid

Anti-Pattern	Problem	Fix
Activity metrics only	Patches applied ≠ risk reduced	Outcome metrics: SLA compliance, breach rate
Too many metrics	Dashboard fatigue; nothing acted on	8-12 core metrics; detailed drilldowns
No trending	Point-in-time snapshot doesn't show direction	13-week rolling trend on all metrics
Metrics without owners	Nobody responsible for improving the number	Named owner for each tier-1 metric
Gaming the metric	Closing vulnerabilities without fixing them	Review metrics for gaming signals quarterly

10 Rules

1. Measure outcomes, not activities — SLA compliance matters more than "scans run".
2. 8-12 core metrics maximum — more causes paralysis.
3. Every metric has a target and an owner — data without accountability doesn't improve.
4. 13-week trend is more valuable than a single data point — direction matters.
5. Mean time to detect and mean time to respond are the most important incident metrics.
6. Vulnerability SLA compliance reveals programme effectiveness better than raw count.
7. Share security metrics with engineering leadership — security is a shared responsibility.
8. Review metrics for gaming quarterly — closing tickets without fixing issues is invisible without review.
9. Benchmark against your previous performance, not industry averages — you don't know their data quality.
10. Automate collection — manually compiled metrics become stale and get skipped under pressure.

Deep dive: applying this in practice

The sections above describe what to produce. This section describes how practitioners actually run this in the field, including the conversations, artefacts, and review loops that turn a one-page recommendation into a sustained outcome.

The 30/60/90 cadence

A recommendation that is never revisited is a recommendation that quietly fails. Bake review checkpoints in from day one:

• Day 0 — Decision committed. Owner, scope, success metrics, and the first-checkpoint date are recorded in the decision log. The artefact is linked from the team's working space so it is discoverable without asking.
• Day 30 — Early-signal review. Look at the leading indicators, not the lagging ones. Has the team actually started? Are the assumed dependencies real? Have any of the named risks materialised? Adjust scope, not the goal.
• Day 60 — Course-correction window. This is the last cheap moment to change direction. If the leading indicators are flat or negative, escalate. Silence at day 60 is the most expensive form of optimism.
• Day 90 — Outcome review. Measure against the success criteria captured on day 0, not against the story the team is telling now. Write the post-mortem (or pre-mortem-confirmed) in the same artefact so the rationale, the outcome, and the lessons live together.

Stakeholder choreography

Decisions stall not because the analysis is wrong but because the choreography is wrong. Use a lightweight RACI on every recommendation:

Role	Meaning	Anti-pattern
Responsible	Does the work	More than two people listed
Accountable	Owns the outcome, signs off	Shared accountability (always becomes no accountability)
Consulted	Two-way input before the decision	Consulted after the decision is made — purely performative
Informed	One-way notification after the decision	Informed people are asked to approve — wastes their time and yours

If you cannot name a single Accountable person in one minute, the recommendation is not ready to ship.

Writing for senior readers

Senior readers scan first, read second, and only re-read the parts they disagree with. Optimise for that pattern:

1. Lead with the recommendation, not the analysis. The reader should know what you want them to do before they finish the first paragraph.
2. One screen, one page, one decision. If the artefact needs scrolling on a laptop, it is too long for the audience it is written for.
3. Tables beat paragraphs for comparing options. Prose hides the trade-off; a table forces it into the open.
4. Numbers beat adjectives. Replace "significant" with the actual number. Replace "soon" with a date. Replace "improved" with a baseline and a target.
5. Name the disconfirming evidence. A recommendation that lists what would change the author's mind is read as honest; one that does not is read as advocacy.

Common failure modes

Failure mode	Symptom	Counter-move
Analysis paralysis	Weeks of investigation, no decision	Time-box the analysis. State the decision quality you can defend in the time available.
HiPPO override	Highest-paid person's opinion wins regardless of evidence	Force the trade-off table into the room before opinions are voiced
Sunk-cost gravity	Team defends the current path because of prior investment	Re-frame: what would we choose today with no prior investment?
Scope creep at the checkpoint	Review becomes a re-planning session	Separate "did this work?" from "what next?" Run them as two meetings.
Stealth de-scoping	Success metrics quietly soften between day 0 and day 90	Lock the day-0 metrics into the artefact; require an explicit amendment to change them.
Owner drift	Accountable person leaves, no one re-assigns	Owner reassignment is a mandatory step in onboarding/offboarding the role

A worked example

A product line is debating whether to invest in a major rewrite of a legacy service that has been failing under peak load.

A weak response: "We should rewrite it because the code is old."

A response that uses this skill:

Recommendation. Do not rewrite. Invest one quarter in targeted performance work on the existing service and a parallel strangler-fig migration of the top two failing endpoints. Confidence: medium. Would change my mind if peak-load incidents continue at the current rate for two consecutive months after the performance work ships.

Options considered. (1) Full rewrite — 9–12 months, ~$1.4M, high risk of partial delivery. (2) Performance fix in place — 6 weeks, ~$120K, addresses 80% of incident volume per last-quarter analysis. (3) Strangler-fig migration — 6 months for the two hottest endpoints, ~$400K, preserves optionality.

Plan. Owner: Platform tech lead. Day 30: performance fix in staging with load test results. Day 60: production rollout and a 30-day incident-rate comparison. Day 90: decision on whether to expand the strangler-fig scope.

Risks. (1) Performance fix masks a deeper architectural issue — mitigated by capturing flame graphs before and after. (2) Strangler-fig endpoints are not in fact the hottest ones — mitigated by re-running the traffic analysis at day 0. (3) Team capacity collides with a separate compliance deadline — escalated to the portfolio review on the next planning cycle.

That is the shape of output this skill should produce: a defensible, time-bound, owner-attached recommendation that respects the reader's time and survives turnover.

Quick reference card

• One paragraph of context, three options with trade-offs, one recommendation with confidence, one plan with an owner and a date.
• If you cannot name the owner, the metric, and the checkpoint date in one breath, the artefact is not done.
• A decision without a written rationale is a rumour. A rationale without a checkpoint is a wish. A checkpoint without a metric is theatre.
• Reversibility matters more than people admit: one-way doors deserve the slow lane, two-way doors deserve the fast lane.
• The best artefacts in this category are short, dated, signed, and easy to find six months later.