codeprobe-trust-model

name

codeprobe-trust-model

description

Explain the codeprobe trust model and verdict engine (signals, tier escalation, blast-radius thresholds, temporal risk) that produces a `VerdictResponse` with one of {auto_merge, single_review, dual_review, expert_review, block}. Use when the user asks about 'trust model', 'change verdict', or 'when to block a PR'. DO NOT use to compute a verdict for a specific PR — run `uv run codeprobe verdict` (or the CI workflow generated by `codeprobe ci-init`) instead. DO NOT use for general scan interpretation — use codeprobe-interpret.

Codeprobe Trust Model

Conceptual framework for autonomous PR review routing based on deterministic risk scoring. The verdict engine runs in under 60 seconds with zero LLM calls, making it suitable for CI/CD pipelines.

Status

The trust model is implemented in two modules:

• src/codeprobe/verdict.py -- Deterministic change verdict engine
• src/codeprobe/temporal.py -- Temporal risk intelligence (trend tracking)

CI/CD workflow generation is available via uv run codeprobe ci-init ..

Verdict Tiers

Tier	Exit Code	Meaning	Action
auto_merge	0	Low risk, well-tested area	Merge without human review
single_review	1	Normal risk	One reviewer required
dual_review	1	Elevated risk	Two reviewers required
expert_review	2	High risk, complex blast radius	Domain expert must review
block	3	Critical risk or security finding	Cannot merge until resolved

Signal Types

The verdict engine aggregates these signal categories:

Signal	Severity Range	Source
blast_radius	info to block	gitnexus_impact -- upstream dependents of changed symbols
churn_rate	warning	git_hotspots -- changed files in top-10 hotspots
bus_factor	escalation	git_contributors -- areas with single contributor
test_gap	warning to escalation	Test coverage analysis
security_finding	escalation to block	semgrep -- critical/high SAST findings
pattern_violation	warning	patterns.json -- deviations from established patterns
cross_community	warning to escalation	gitnexus_cypher -- changes spanning multiple code communities
ownership_gap	warning	No clear code owner for changed area
complexity_spike	warning to escalation	temporal_risk -- degrading risk trend in affected area

Scoring Pipeline

The verdict engine in verdict.py follows this deterministic pipeline:

1. git diff -- Get changed files between base_ref and head_ref
2. GitNexus cypher -- Map files to symbols and communities
3. GitNexus impact -- Compute blast radius for top changed symbols (capped at 10)
4. Risk profile matching -- Cross-reference changed files/communities against pre-computed area risk profiles from the last full analysis
5. Security check -- Check heuristic_summary for critical/high semgrep findings
6. Git signals -- Check if changed files are hotspots or in bus-factor-risk areas
7. Temporal trends -- Check if affected communities have degrading risk trajectories
8. Pattern consistency -- Check for known pattern violations
9. Composite scoring -- Aggregate signals into final verdict tier

Tier Escalation Rules

• Any block signal -> block
• Each escalation signal pushes up 1 tier
• Every 2 warning signals push up 1 tier (rounded)
• Risk area recommendations act as floor (area with expert_review rec can't produce auto_merge)
• Outdated index data (>7 days) -> at least single_review

Confidence Calibration

Confidence starts at 0.8 (with analysis data) or 0.4 (without):

• +0.1 if GitNexus data is available
• +0.05 if 3+ signals collected
• Minus freshness penalty (0.15 for stale, 0.30 for outdated)

Blast Radius Thresholds

Upstream Dependents	Severity	Meaning
>50	block	Will break many consumers
>20	escalation	Likely to cause cascading issues
>5	warning	Should be tested carefully
1-5	info	Manageable scope

Trust Score Formula (Conceptual)

trust_score = blast_radius_weight * blast_radius_signal
            + churn_weight * churn_signal
            + security_weight * security_signal
            + bus_factor_weight * bus_factor_signal
            + pattern_weight * pattern_signal
            + temporal_weight * temporal_trend_signal

The composite trust score maps to a verdict tier via threshold-based escalation. The decision boundary transparency shows how close a verdict is to flipping to the next tier, with specific triggers for escalation and de-escalation.

Temporal Risk Intelligence

The temporal module (src/codeprobe/temporal.py) tracks risk trajectories across analysis runs:

• Persists point-in-time risk snapshots to .codeprobe/history/risk_{timestamp}.json
• Computes per-area trends: improving, stable, degrading, accelerating_risk
• Tracks churn velocity, contributor trends, test coverage trends
• Projects 30-day risk level via linear extrapolation
• Areas with accelerating_risk (3+ consecutive degrading snapshots) trigger escalation signals

CI/CD Integration

Generate CI workflow files:

uv run codeprobe ci-init .                 # Both GitHub Actions + GitLab CI
uv run codeprobe ci-init . --provider github  # GitHub Actions only

Generated workflows include three cadences:

1. Nightly: Full analysis (risk profiles, patterns, temporal snapshots)
2. On-merge: Incremental index (keeps structural graph current)
3. PR: Fast change verdict against cached context (<60s)

Output Format

The VerdictResponse model includes CI-actionable fields:

{
  "verdict": {
    "verdict": "single_review",
    "confidence": 0.85,
    "affected_symbols": ["process_request", "validate_input"],
    "affected_communities": ["request-handling", "validation"],
    "blast_radius": 23,
    "signals": [...],
    "escalation_reasons": ["23 upstream dependents across 2 changed symbols"],
    "reasoning_chain": ["1. Diff: 3 files changed...", "2. Blast radius: 23...", "VERDICT: single_review"],
    "decision_boundary": {
      "current_verdict": "single_review",
      "next_higher_verdict": "dual_review",
      "distance": 0.4,
      "escalation_triggers": ["Blast radius > 50 would trigger expert_review"],
      "de_escalation_triggers": ["Adding test coverage for changed code paths"]
    },
    "recommended_reviewers": [
      {"identity": "[e-mail]", "reason": "recent_contributor", "confidence": 0.9}
    ]
  },
  "exit_code": 1,
  "should_block": false,
  "review_comment_markdown": "## Codeprobe Verdict: `single_review`\n...",
  "github_labels": ["high-blast-radius"]
}

Future Capabilities

The following are planned extensions:

• Cross-repo impact analysis via ServiceContract and ContractConsumer models
• Pattern persistence via ArchitecturalPattern and ConsistencyReport models
• Calibrated confidence with AnalysisConfidence and CalibratedJudgment models
• Auto-approvable path patterns (docs, config, tests) in RiskProfile.auto_approvable_patterns

Acceptance

Success = the user can name the five verdict tiers and their exit codes, identify which signal categories escalate to block vs. escalation vs. warning, and cite the blast-radius thresholds (>50 block, >20 escalation, >5 warning, 1-5 info); they know that codeprobe ci-init generates the three-cadence CI workflow and that the verdict engine runs deterministically in under 60 seconds with zero LLM calls.