evidence-system

name	evidence-system
description	Skill for managing the evidence knowledge store: creating and querying findings, maintaining belief maps and investigation journals, enforcing data integrity, and running consistency checks across validated findings.
user-invocable	true
disable-model-invocation	false

Evidence System Skill

Managing findings, belief maps, investigation journal, and raw data across the knowledge store.

When to Use

Use this skill when creating, reviewing, or querying evidence artifacts in investigation workflows.

Layer 1: Findings (Structured Knowledge)

Findings are the unit of scientific knowledge. They live in Beads with a strict schema.

Positive Finding

bd create "FINDING: [result with effect size and CI]" -t task --parent <epic>
bd update <id> --notes "TYPE:finding | VALENCE:positive | CONFIDENCE:0.X"
bd update <id> --notes "SOURCE_TASK:<investigation-task-id>"
bd update <id> --notes "EFFECT_SIZE:[d] | CI_95:[lo, hi] | N:[n] | STAT_TEST:[test] | P:[p]"
bd update <id> --notes "DATA_FILE:<path>"
bd update <id> --notes "DATA_HASH:sha256:<hash>"
bd update <id> --notes "SENSITIVITY: [param variations and results] | ROBUST:yes|no"
bd update <id> --notes "REPLICATED:no | STAT_QUALITY:[score] | REVIEWED_BY:[reviewer]"
bd update <id> --notes "CONTRADICTS:[finding-ids] | SUPPORTS:[hypothesis-ids]"

Negative Finding

bd create "FINDING: [variable] has no measurable effect on [outcome]" -t task --parent <epic>
bd update <id> --notes "TYPE:finding | VALENCE:negative | CONFIDENCE:0.X"
bd update <id> --notes "EFFECT_SIZE:[d] | CI_95:[lo, hi] | N:[n] | P:[p]"
bd update <id> --notes "IMPLICATION: [what this rules out]"

Finding States

State	Meaning
pending_review	Awaiting Statistician + Critic review
validated	Passed all review gates
contested	Failed adversarial loop or replication disagreement
quarantined	Data integrity check failed
replicated	Independently confirmed

Layer 2: Raw Data

Storage Convention

Raw data files live in the agent's worktree:

<worktree>/data/raw/      # Untouched experimental output
<worktree>/data/processed/ # Cleaned/transformed data
<worktree>/data/figures/   # Generated visualizations

Data Integrity Chain

1. Compute SHA-256 immediately after collection:

   shasum -a 256 data/raw/<file>.csv
   

2. Record hash in finding:

   bd update <finding-id> --notes "DATA_HASH:sha256:<hash>"
   

3. Reviewers independently verify the hash before analysis
4. Hash mismatch → finding quarantined immediately

Layer 3: Strategic Context

Location: .swarm/strategic-context.md

The orchestrator maintains this living document for strategic continuity across OODA cycles and sessions.

Strategic Context Format

## Decisions and Rationale
| Cycle | Decision | Rationale | Alternatives Considered |
|-------|----------|-----------|------------------------|

## Dead Ends
| Approach | Cycle Abandoned | Why | Key Finding |
|----------|----------------|-----|-------------|

Who Reads/Writes

Role	Access
Orchestrator	Creates and updates each OODA cycle
Synthesizer	Reads when building deliverables
Theorist	Reads when building causal models

Layer 4: Belief Map

A structured representation of all hypotheses and their current status.

Belief Map Entry

bd create "BELIEF_MAP: [investigation name]" -t task --parent <epic>
bd update <id> --notes "UPDATED:cycle-N | HYPOTHESES_TOTAL:X | TESTED:Y | REMAINING:Z"
bd update <id> --notes "H1:[name] | STATUS:confirmed|refuted|untested|contested|abandoned | P:0.X | EVIDENCE:[finding-ids]"
bd update <id> --notes "H2:[name] | STATUS:[status] | PRIOR:0.X | BASIS:[evidence source]"

Information-Gain Formula

The orchestrator selects the next hypothesis to test using:

priority(H) = uncertainty(H) × impact(H) × testability(H)

Where:

• uncertainty(H) = 1 - |prior - 0.5| × 2 (highest at 0.5 prior)
• impact(H) = number of downstream tasks/hypotheses depending on H
• testability(H) = Methodologist's assessment of how cleanly H can be tested

Querying Evidence

Find all validated findings

bd list --json | python3 -c "
import sys, json
for t in json.load(sys.stdin):
    notes = t.get('notes','')
    if 'TYPE:finding' in notes and 'STAT_REVIEW: APPROVED' in notes:
        print(f\"{t['id']}: {t['title']}\")
"

Find belief map

bd list --json | python3 -c "
import sys, json
for t in json.load(sys.stdin):
    if 'BELIEF_MAP' in t.get('title',''):
        print(f\"{t['id']}: {t['title']}\")
"

Find contested findings

bd list --json | python3 -c "
import sys, json
for t in json.load(sys.stdin):
    notes = t.get('notes','')
    if 'ADVERSARIAL_STATUS:CONTESTED' in notes or 'DATA_INTEGRITY:FAILED' in notes:
        print(f\"{t['id']}: {t['title']}\")
"

Consistency Gate

When the Synthesizer integrates a new finding:

1. Pairwise comparison against ALL existing validated findings
2. Check: Do conclusions contradict? Do effect directions conflict?
3. If contradiction detected:

   bd update <finding-id> --notes "CONSISTENCY_CONFLICT:<finding-A> vs <finding-B> | TYPE:[type] | SEVERITY:[level]"
   

4. CONSISTENCY_CONFLICT blocks convergence — must be resolved before investigation closes
5. Resolution: re-run experiments, identify moderating variable, or update causal model

Convergence Requirements by Rigor

Rigor	Requirements
Standard	All build tasks closed, tests passing
Analytical	All questions answered with data, Statistician reviewed all findings, Evaluator score ≥ 0.75
Scientific	All hypotheses resolved, causal model complete, 1+ prediction confirmed, 1+ competing theory ruled out, all findings ROBUST or FRAGILE-with-conditions, zero CONSISTENCY_CONFLICTs, Evaluator score ≥ 0.75
Experimental	Scientific + all high-impact findings replicated, pre-registration verified, meta-analysis complete, all adversarial loops resolved, Evaluator score ≥ 0.75

Layer 5: Strategic Context Document

Location: .swarm/strategic-context.md

A living document maintained by the orchestrator that preserves:

• Original Abstract — verbatim user input, never modified
• Current Interpretation — how the abstract is being read now (may evolve)
• Decisions and Rationale — key forks taken and why
• Dead Ends — approaches abandoned and why (prevents re-exploration)
• Remaining Gaps — what's still weak in the answer
• Progress Velocity — eval scores per cycle for diminishing returns detection

The orchestrator reads this at session start (before journal, before Beads). Workers receive relevant excerpts via the STRATEGIC_CONTEXT field in their task notes.

See the strategic-context skill for the full maintenance protocol.

Layer 6: Final Deliverable

Location: .swarm/deliverable.md

The Synthesizer produces a structured final output artifact that directly answers the user's original request. Unlike the journal (which tracks process), the deliverable IS the answer.

Structure:

• Executive Summary
• Findings by Topic (mapped to original abstract)
• Evidence strength per conclusion
• Gaps and Limitations
• Recommended Next Steps
• Evidence Appendix

The Evaluator scores this against the original abstract before convergence is declared. See evaluator.agent.md for scoring dimensions and thresholds.