| name | codeprobe-trust-model |
| description | Explain the codeprobe trust model and verdict engine (signals, tier escalation, blast-radius thresholds, temporal risk) that produces a `VerdictResponse` with one of {auto_merge, single_review, dual_review, expert_review, block}. Use when the user asks about 'trust model', 'change verdict', or 'when to block a PR'. DO NOT use to compute a verdict for a specific PR — run `uv run codeprobe verdict` (or the CI workflow generated by `codeprobe ci-init`) instead. DO NOT use for general scan interpretation — use codeprobe-interpret. |
Codeprobe Trust Model
Conceptual framework for autonomous PR review routing based on deterministic risk scoring. The verdict engine runs in under 60 seconds with zero LLM calls, making it suitable for CI/CD pipelines.
Status
The trust model is implemented in two modules:
src/codeprobe/verdict.py -- Deterministic change verdict enginesrc/codeprobe/temporal.py -- Temporal risk intelligence (trend tracking)
CI/CD workflow generation is available via uv run codeprobe ci-init ..
Verdict Tiers
| Tier | Exit Code | Meaning | Action |
|---|
auto_merge | 0 | Low risk, well-tested area | Merge without human review |
single_review | 1 | Normal risk | One reviewer required |
dual_review | 1 | Elevated risk | Two reviewers required |
expert_review | 2 | High risk, complex blast radius | Domain expert must review |
block | 3 | Critical risk or security finding | Cannot merge until resolved |
Signal Types
The verdict engine aggregates these signal categories:
| Signal | Severity Range | Source |
|---|
blast_radius | info to block | gitnexus_impact -- upstream dependents of changed symbols |
churn_rate | warning | git_hotspots -- changed files in top-10 hotspots |
bus_factor | escalation | git_contributors -- areas with single contributor |
test_gap | warning to escalation | Test coverage analysis |
security_finding | escalation to block | semgrep -- critical/high SAST findings |
pattern_violation | warning | patterns.json -- deviations from established patterns |
cross_community | warning to escalation | gitnexus_cypher -- changes spanning multiple code communities |
ownership_gap | warning | No clear code owner for changed area |
complexity_spike | warning to escalation | temporal_risk -- degrading risk trend in affected area |
Scoring Pipeline
The verdict engine in verdict.py follows this deterministic pipeline:
- git diff -- Get changed files between base_ref and head_ref
- GitNexus cypher -- Map files to symbols and communities
- GitNexus impact -- Compute blast radius for top changed symbols (capped at 10)
- Risk profile matching -- Cross-reference changed files/communities against pre-computed area risk profiles from the last full analysis
- Security check -- Check heuristic_summary for critical/high semgrep findings
- Git signals -- Check if changed files are hotspots or in bus-factor-risk areas
- Temporal trends -- Check if affected communities have degrading risk trajectories
- Pattern consistency -- Check for known pattern violations
- Composite scoring -- Aggregate signals into final verdict tier
Tier Escalation Rules
- Any
block signal -> block
- Each
escalation signal pushes up 1 tier
- Every 2
warning signals push up 1 tier (rounded)
- Risk area recommendations act as floor (area with
expert_review rec can't produce auto_merge)
- Outdated index data (>7 days) -> at least
single_review
Confidence Calibration
Confidence starts at 0.8 (with analysis data) or 0.4 (without):
- +0.1 if GitNexus data is available
- +0.05 if 3+ signals collected
- Minus freshness penalty (0.15 for stale, 0.30 for outdated)
Blast Radius Thresholds
| Upstream Dependents | Severity | Meaning |
|---|
| >50 | block | Will break many consumers |
| >20 | escalation | Likely to cause cascading issues |
| >5 | warning | Should be tested carefully |
| 1-5 | info | Manageable scope |
Trust Score Formula (Conceptual)
trust_score = blast_radius_weight * blast_radius_signal
+ churn_weight * churn_signal
+ security_weight * security_signal
+ bus_factor_weight * bus_factor_signal
+ pattern_weight * pattern_signal
+ temporal_weight * temporal_trend_signal
The composite trust score maps to a verdict tier via threshold-based escalation. The decision boundary transparency shows how close a verdict is to flipping to the next tier, with specific triggers for escalation and de-escalation.
Temporal Risk Intelligence
The temporal module (src/codeprobe/temporal.py) tracks risk trajectories across analysis runs:
- Persists point-in-time risk snapshots to
.codeprobe/history/risk_{timestamp}.json
- Computes per-area trends:
improving, stable, degrading, accelerating_risk
- Tracks churn velocity, contributor trends, test coverage trends
- Projects 30-day risk level via linear extrapolation
- Areas with
accelerating_risk (3+ consecutive degrading snapshots) trigger escalation signals
CI/CD Integration
Generate CI workflow files:
uv run codeprobe ci-init .
uv run codeprobe ci-init . --provider github
Generated workflows include three cadences:
- Nightly: Full analysis (risk profiles, patterns, temporal snapshots)
- On-merge: Incremental index (keeps structural graph current)
- PR: Fast change verdict against cached context (<60s)
Output Format
The VerdictResponse model includes CI-actionable fields:
{
"verdict": {
"verdict": "single_review",
"confidence": 0.85,
"affected_symbols": ["process_request", "validate_input"],
"affected_communities": ["request-handling", "validation"],
"blast_radius": 23,
"signals": [...],
"escalation_reasons": ["23 upstream dependents across 2 changed symbols"],
"reasoning_chain": ["1. Diff: 3 files changed...", "2. Blast radius: 23...", "VERDICT: single_review"],
"decision_boundary": {
"current_verdict": "single_review",
"next_higher_verdict": "dual_review",
"distance": 0.4,
"escalation_triggers": ["Blast radius > 50 would trigger expert_review"],
"de_escalation_triggers": ["Adding test coverage for changed code paths"]
},
"recommended_reviewers": [
{"identity": "alice@example.com", "reason": "recent_contributor", "confidence": 0.9}
]
},
"exit_code": 1,
"should_block": false,
"review_comment_markdown": "## Codeprobe Verdict: `single_review`\n...",
"github_labels": ["high-blast-radius"]
}
Future Capabilities
The following are planned extensions:
- Cross-repo impact analysis via
ServiceContract and ContractConsumer models
- Pattern persistence via
ArchitecturalPattern and ConsistencyReport models
- Calibrated confidence with
AnalysisConfidence and CalibratedJudgment models
- Auto-approvable path patterns (docs, config, tests) in
RiskProfile.auto_approvable_patterns
Acceptance
Success = the user can name the five verdict tiers and their exit codes, identify which signal categories escalate to block vs. escalation vs. warning, and cite the blast-radius thresholds (>50 block, >20 escalation, >5 warning, 1-5 info); they know that codeprobe ci-init generates the three-cadence CI workflow and that the verdict engine runs deterministically in under 60 seconds with zero LLM calls.
