القائمة
Render per-check Bayesian trust scores from AgentDB reflexion entries. Read-only -- no side effects.
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
استنادا إلى تصنيف SOC المهني
On-demand vision alignment check. Spawns ndp-vision-guardian to review SPARC artifacts against product/vision/ALIGNMENT-CRITERIA.md.
Validate planning swarm output: artifact existence, AC coverage, pattern IDs, stale references, internal consistency. Produces a glass box report.
NDP 4-tier implementation validation. Tier 1: compilation (build+test+anti-stub). Tier 2: process adherence. Tier 3: spec compliance. Tier 4: risk classification. Produces glass box reports.
Retrieve APPLICATION patterns (architecture, procedures, conventions) from AgentDB using multi-signal retrieval: pattern search, causal recall, and RL predictions. Use BEFORE implementing to ensure consistency.
AgentDB pattern lifecycle management: list, get, delete, deprecate, update, stats, search, duplicates. Use when cleaning up stale patterns, removing deprecated entries, finding duplicates, or auditing pattern health. Workaround for missing MCP delete/update tools (GH Issue #42).
Record feedback on pattern effectiveness. Stores episodes that train the recommendation system, feed the RL engine for smarter pattern ranking, build causal knowledge, and enable pattern discovery via learner.
| name | trust-dashboard |
| description | Render per-check Bayesian trust scores from AgentDB reflexion entries. Read-only -- no side effects. |
Queries AgentDB for trust reflexion entries (written by /validate after each run), computes per-check Bayesian trust scores using Beta distributions, and renders a human-readable dashboard. This is a read-only skill with no side effects.
Trust scores start uninformative (0.5) and calibrate over time as more validation runs and /shadow-judge comparisons accumulate.
/trust-dashboard
No arguments required. Queries all trust:validation:* entries from AgentDB.
reflexion_retrieve(task="trust:validation", limit=500)
This returns all reflexion entries with the trust:validation:* prefix, written by /validate (ADR-004) and /shadow-judge (ADR-006).
Parse the task prefix to extract tier and check name:
trust:validation:{tier}:{check_name}
Group entries by {tier}:{check_name}.
For each check, compute the Bayesian trust score:
correct = count of entries where reward = 1.0
incorrect = count of entries where reward = 0.0
Trust(check) = (correct + 1) / (correct + incorrect + 2)
This is a Beta(correct+1, incorrect+1) distribution mean. With no data, Trust = 0.5 (uninformative prior). As evidence accumulates, the score converges to the true reliability.
Composite = 0.30 * avg(Tier1_checks)
+ 0.30 * avg(Tier2_checks)
+ 0.15 * avg(Tier3_checks)
+ 0.15 * (1 - rework_rate)
+ 0.10 * scope_conformance
Where:
rework_rate = proportion of validation runs that required fix iterationsscope_conformance = proportion of file_scope checks that passedIf a tier has no data, use 0.5 (uninformative) for its average.
TRUST DASHBOARD ({date})
============================
COMPOSITE SCORE: {score}
PER-CHECK SCORES:
Tier 1 (Compilation):
build ............... {score} ({correct}/{total})
test ................ {score} ({correct}/{total})
clippy .............. {score} ({correct}/{total})
anti_stub ........... {score} ({correct}/{total})
Tier 2 (Process Adherence):
banned_deps ......... {score} ({correct}/{total})
stub_scan ........... {score} ({correct}/{total})
file_scope .......... {score} ({correct}/{total})
stale_refs .......... {score} ({correct}/{total})
config_valid ........ {score} ({correct}/{total})
Tier 3 (Spec Compliance):
ac_coverage ......... {score} ({correct}/{total})
test_delta .......... {score} ({correct}/{total})
new_deps ............ {score} ({correct}/{total})
Tier 4 (Risk Classification):
risk_score .......... {score} ({correct}/{total})
SHADOW JUDGE CALIBRATION:
Total comparisons: {N}
Human agreed: {M}
Human disagreed: {K}
False negative rate: {K/N}
LAST 5 FEATURES:
| Feature | Tier1 | Tier2 | Tier3 | Tier4 | Overall |
|---------|-------|-------|-------|-------|---------|
| {id} | {r} | {r} | {r} | {risk}| {r} |
TREND: {Improving|Stable|Regressing} (based on last 5 composite scores)
If no trust entries exist yet:
TRUST DASHBOARD ({date})
============================
No trust data available yet.
Trust scores accumulate after each /validate run.
Run /validate on a feature to start building trust data.
Run /shadow-judge after reviewing to add calibration signal.
| Score Range | Meaning |
|---|---|
| 0.90 - 1.00 | High trust -- check is reliable |
| 0.70 - 0.89 | Good trust -- occasional false negatives |
| 0.50 - 0.69 | Uninformative -- not enough data or mixed results |
| 0.30 - 0.49 | Low trust -- check frequently misses issues |
| 0.00 - 0.29 | Very low trust -- check is unreliable |
Important: Scores below 0.5 are unusual and indicate the check is wrong more often than right. This typically means the check itself needs revision, not that the code is bad.
| Source | What It Writes | When |
|---|---|---|
/validate | reward=1.0 for each passing check | After every validation run |
/shadow-judge approve | reward=1.0 for all checks | When human agrees |
/shadow-judge reject | reward=0.0 for missed checks | When human finds issues |
Self-reported trust (from /validate alone) is always 1.0. Only /shadow-judge reject adds real calibration signal (reward=0.0).
The Beta distribution needs approximately 20+ observations per check for statistically meaningful scores. With fewer observations, the score is heavily influenced by the prior (0.5). The dashboard does not attempt significance testing -- it shows raw Beta means and observation counts so the user can judge data sufficiency.
.claude/skills/validate/SKILL.md -- writes trust entries after validation.claude/skills/shadow-judge/SKILL.md -- human calibration inputproduct/features/ops-006/architecture/ARCHITECTURE.md -- ADR-004 (trust storage), ADR-005 (dashboard)