| name | review-hypotheses |
| description | Resolves open hypotheses whose horizons have elapsed, extracts lessons from each outcome, calculates per-category accuracy stats, and generates a calibration report. Use whenever the aspirations loop hits the hypothesis-review cadence, the user says "how accurate are my predictions" or "review my hypotheses", or enough hypotheses have reached their horizon to warrant a batch resolve. Modes: --resolve, --learn, --accuracy-report, --full-cycle, --category-comparison. |
| user-invocable | false |
| triggers | ["/review-hypotheses"] |
| parameters | [{"name":"mode","description":"--resolve, --learn, --accuracy-report, --full-cycle, --category-comparison","required":false}] |
| execution_history | {"total_invocations":0,"outcome_tracking":{"successful":0,"unsuccessful":0,"success_rate":0},"last_invocation":null,"known_pitfalls":[],"reconsolidation_trigger":"After 10 invocations with declining success rate, trigger skill review"} |
| conventions | ["pipeline","aspirations","tree-retrieval","reasoning-guardrails","pattern-signatures"] |
| minimum_mode | assistant |
| revision_id | skill-bootstrap-review-hypotheses-9490ec |
| previous_revision_id | null |
/review-hypotheses — Hypothesis Review & Resolution Engine
Two-phase design: resolve (detect outcomes, move records, record results) and learn (reflect on outcomes, extract patterns). These phases are separated so that /boot can resolve without triggering learning, and /aspirations goals can learn from freshly resolved data without re-checking resolution sources.
Parameters
--resolve — Check resolution status, move active→resolved, record outcomes. Does NOT call /reflect. Sets reflected: false on each record.--learn — Find resolved records with reflected: false, call /reflect --on-hypothesis for each, set reflected: true.--accuracy-report — Generate accuracy statistics across all resolved hypotheses--full-cycle — --resolve + --learn + --accuracy-report + /reflect --full-cycle + spark check--category-comparison <cat1> <cat2> — Compare accuracy between two categories--hypothesis <id> — Check a specific hypothesis
- When called as a goal's skill (from aspirations loop), returns one of:
CONFIRMED — hypothesis confirmed (move pipeline file to resolved/)CORRECTED — hypothesis disconfirmed (move pipeline file to resolved/)PENDING — not enough evidence yet (goal stays pending for retry)EXPIRED — past resolves_by deadline (move pipeline file to archived/)
- No args → default to
--resolve
Step 0: Load Conventions
Step 0: Load Conventions — Bash: load-conventions.sh with each name from the conventions: front matter. Read only the paths returned (files not yet in context). If output is empty, all conventions already loaded — proceed to next step.
Mode 1: Resolve (--resolve)
Detects which active hypotheses have resolved, records outcomes, moves records, and updates the memory tree. Does NOT trigger reflection or learning — that is --learn's job.
Step 1: Load Hypotheses to Check
# Step 1.0: Pre-scan discovered-stage records orphaned past resolves_by (g-115-1629).
# review-hypotheses historically loaded ONLY active + measurement-pending, so
# discovered records past resolves_by were invisible -- never resolved, never
# fed accuracy stats (63 of 193 orphaned at filing, oldest >2mo). This sweep
# EXPIRES clearly-unresolvable ones (short/session horizon past the observation
# window -> archived UNRESOLVABLE, gate-exempt) and PROMOTES well-formed
# evaluable ones to active so the resolution loop below catches them THIS run.
# Direct py -3 (not a bash wrapper) per rb-225/rb-247.
Bash: py -3 core/scripts/hypothesis-discovered-overdue-sweep.py --apply --output json
# Parse {expired, promoted, needs_judgment}. `promoted` records are now
# stage=active and WILL appear in the active load below (resolved this run).
# `needs_judgment` (under-formed, recently overdue) are SURFACED, not auto-
# resolved: for each, if evaluable, synthesize a claim from the position
# (guard-798: leaving discovered needs claim>=20 + a resolution method), then
# resolve/expire with judgment; else leave for the next cycle.
# Primary source: all active hypotheses
Bash: pipeline-read.sh --stage active
# Also re-probe measurement-pending hypotheses (g-115-465 / rb-754) — these are
# shelved waiting for measurement infrastructure to appear. Re-checking the channel
# every cycle is cheap and lets them resolve as soon as data shows up. A re-probed
# measurement-pending record now past its hard resolves_by deadline with the channel
# STILL empty is EXPIRED by Step 2.6a (g-115-1584 / rb-2122) — without that pass the
# stage was a one-way shelf with no auto-expiry that grew unbounded (g-001-02 had to
# hand-archive 35 past-deadline records).
Bash: pipeline-read.sh --stage measurement-pending
APPEND to candidate list
# Horizon filter: micro-hypotheses never enter this pipeline.
# Session-horizon hypotheses use self-check verification (Step 2 handles this).
# Filter OUT any records with horizon: micro (shouldn't exist in pipeline, but defensive).
Filter out records where horizon == "micro"
Sort by resolves_no_earlier_than (soonest first), then end_date for legacy records
Step 1.5: Load Domain Context
Before checking resolution, load background knowledge for each category represented.
Collect unique categories from candidate hypotheses
For each unique category:
Bash: retrieve.sh --category {batch_category} --depth medium
# Returns unified JSON with all data stores. Retrieval counters already incremented.
Cache result — reuse for all hypotheses in same category
Use retrieved context to:
- Inform resolution interpretation (understand domain before judging outcome)
- Populate context_consulted manifest on each hypothesis record during Step 2
- Record: tree_nodes_read, pattern_signatures_checked, articles_read
- Leave context_gaps_identified empty (populated later by /reflect)
Step 2: Check Each Hypothesis
For each active hypothesis:
0. Resolution time filter (token-saving skip):
current_time = current date and time (ET)
# Horizon-aware resolution timing:
# - session: resolves_by may be "session_end" — always check if current session
# - short/long: use resolves_no_earlier_than or end_date as before
# - micro: should never be here (filtered in Step 1)
IF hypothesis.horizon == "session":
IF hypothesis.resolves_by == "session_end":
threshold = null # always eligible for resolution check
ELSE:
threshold = hypothesis.resolves_by (if timestamp) or null
ELSE: # short, long, or unset (default to short behavior)
Determine resolution threshold:
IF hypothesis.resolves_no_earlier_than exists:
threshold = resolves_no_earlier_than
ELSE IF hypothesis.end_date OR hypothesis.source_data.end_date exists:
threshold = end_date + 12 hours (legacy fallback)
ELSE:
threshold = null (no date info — proceed with resolution check)
IF threshold is not null AND threshold > current_time:
SKIP this hypothesis — do NOT check resolution status
Log: "Skipped {slug} — resolves no earlier than {threshold}"
Add to resolve_result.skipped_not_due
Continue to next hypothesis
Note: --hypothesis <id> flag bypasses this filter (always checks)
1. Check resolution date:
- Past due? → Likely resolved, check outcome
- Within 48 hours? → Flag as "resolving soon"
- Session horizon with resolves_by: session_end? → Always check
2. Determine resolution status using horizon-appropriate methods:
# Session-horizon: self-check verification (lightweight)
IF hypothesis.horizon == "session":
IF hypothesis.verification == "self_check":
Evaluate the hypothesis outcome using available local state:
- File existence checks
- State file inspection
- Result of the action that was taken
- Inline reasoning about the outcome
If outcome determinable: record it and proceed to Step 3
If outcome NOT determinable: skip (will retry next session)
ELSE:
Fall through to standard methods below
# Short/long horizon: full external verification (in priority order)
a. Domain-specific sub-skill (if available):
Invoke the relevant resolution sub-skill for this hypothesis's category
Sub-skills handle API calls, data fetching, and status interpretation
b. WebSearch (general resolution check):
WebSearch: "{hypothesis question} outcome result"
WebSearch: "{hypothesis question} resolved decided"
Look for authoritative sources confirming the outcome
c. WebFetch (direct source check):
If the hypothesis record includes a resolution_url or source_url,
fetch it directly and check for outcome indicators
d. Resolution timestamp:
If resolves_no_earlier_than has passed and the hypothesis has a
known binary outcome trigger (e.g., scheduled event), check if
the triggering event has occurred
2.9. Subject-liveness gate for zero-event windows (rb-3721, g-115-2403):
IF the hypothesis predicts an EVENT COUNT over a time window AND the
observed count is zero/absent:
# A zero is three-way ambiguous — (a) predicted cause resolved,
# (b) no exposure/traffic, (c) the SUBJECT was dead — and the count
# alone cannot discriminate. Mirrors guard-346 (tool-signal zero).
Check the subject (env/service/process) was alive AND exposed across
the window BEFORE accepting the zero as evidence:
- uptime/heartbeat probe or the record's own measurement_channel
- open restore/outage goals naming the subject
- usage/traffic channel for the window
IF the liveness/exposure precondition collapsed for a material part
of the window:
outcome: UNRESOLVABLE (already in pipeline VALID_OUTCOMES;
excluded from accuracy stats, so calibration stays clean)
outcome_detail: name the collapse evidence (downtime span, restore
goal id) + "re-form post-restore" when the prediction still matters
SKIP Steps 3-4 for this hypothesis (do NOT record CORRECTED/CONFIRMED)
3. If resolution is confirmed:
Record outcome:
actual_outcome: "YES" | "NO" (or the relevant outcome value)
resolution_date_actual: "YYYY-MM-DD"
resolution_source: "How we verified (method + source)"
4. Compare hypothesis to outcome:
our_hypothesis: "YES"
our_confidence: 0.72
actual_outcome: "NO"
outcome: CORRECTED
surprise_level: 7 # calculated from confidence gap
Step 2.5: Source Agreement Check
When resolving a hypothesis using evidence from multiple sources:
sources = [] # collected during resolution research above
For each source consulted during Steps 1-2:
sources.append({
source_id: "{domain or identifier}",
source_type: "{web_search | web_fetch | memory | user_input}",
verdict: "{YES | NO | UNCLEAR}", # does this source support the hypothesis?
snippet: "{relevant excerpt, max 200 chars}"
})
IF len(sources) == 0:
source_validation = null # nothing to validate
ELIF len(sources) == 1:
source_validation = {
sources_consulted_count: 1,
agreement: "single_source",
agreement_note: "",
sources: sources
}
ELSE: # 2+ sources
verdicts = [s.verdict for s in sources if s.verdict != "UNCLEAR"]
IF len(verdicts) > 0 AND all verdicts same: agreement = "unanimous"
ELSE:
agreement = "contested"
agreement_note = "Sources disagree: {summarize disagreement}"
→ Append to outcome_detail: "[SOURCE DISAGREEMENT: {agreement_note}]"
source_validation = {
sources_consulted_count: len(sources),
agreement: agreement,
agreement_note: agreement_note or "",
sources: sources
}
This does NOT block resolution. Contested sources are noted but the agent still makes a judgment call on the outcome.
Step 2.6: Measurement-Pending Transition + Expiry (g-115-465 / rb-754 / g-115-1584)
Catches hypotheses that are past resolves_no_earlier_than but whose
measurement_channel produced no usable data. Without this step, such
hypotheses stay in active status indefinitely, bloating the active pool
and producing recurring INCONCLUSIVE results every cycle.
Step 2.6a (g-115-1584 / rb-2122) closes the back half of that lifecycle: a
record ALREADY shelved in measurement-pending whose hard resolves_by
deadline has now passed with the channel still empty is EXPIRED (moved to
archived, outcome=EXPIRED) instead of being re-shelved. Without it the
measurement-pending stage was a one-way shelf with no auto-expiry — it grew
unbounded (g-001-02 had to hand-archive 35 past-deadline records). The data
model already documents this transition as intended (pipeline.py VALID_STAGES
note: "measurement-pending→archived (at resolves_by absolute deadline)"); this
step wires the trigger. Expiry is symmetric to Step 4.0's active-record expiry:
EXPIRED records land in archived (never resolved), so they are excluded from
accuracy stats (compute_meta counts only CONFIRMED/CORRECTED) and are never
pulled into Mode 2 reflection (--unreflected = stage==resolved AND not reflected).
Five gap subtypes (catalog at world/knowledge/tree/system/system-constraints-loop/hypothesis-measurement-gap.md):
- infra-missing: writer/source emitter doesn't exist
- session-empty: infrastructure exists but no events produced
- schema-missing: source records exist, specific field absent
- log-inaccessible: log path not at world root, agent can't read it
- (catch-all): new shapes flagged for taxonomy update
After Steps 2 + 2.5 complete for a hypothesis:
IF actual_outcome was determined: continue to Step 3 (resolved path)
ELIF threshold (resolves_no_earlier_than) has passed AND
resolution attempt completed AND
no value was observable in measurement_channel:
# ── Step 2.6a: HARD-DEADLINE expiry for already-shelved records (g-115-1584 / rb-2122) ──
# If THIS record is already in measurement-pending (it came in via Step 1's
# measurement-pending re-probe load) AND it carries a hard resolves_by deadline
# that has now passed, the measurement will not arrive in time — expire it to
# archived instead of re-shelving. Records with no resolves_by (null) or a
# still-future resolves_by fall through to the normal (re-)shelve below, so a
# hypothesis without a hard deadline keeps re-probing exactly as before.
# resolves_by may be a date (YYYY-MM-DD) or a datetime — compare the same way
# Step 4.0 does ("resolves_by has passed"). Mirrors Step 4.0's active-record
# expiry; EXPIRED lands in archived (never resolved).
IF record.stage == "measurement-pending"
AND record.resolves_by is not null
AND resolves_by has passed (parse date/datetime; compare to current_time):
expiry_merge = {
"outcome": "EXPIRED",
"outcome_date": "<today ISO>",
"outcome_detail": "Measurement-pending expired: past resolves_by ({resolves_by}); channel never produced data (gap_subtype: {record.measurement_gap_subtype}). Auto-expired by Mode 1 Step 2.6a.",
}
echo '<expiry_merge>' | bash core/scripts/pipeline-move.sh <id> archived
Add to resolve_result.expired_measurement_pending list.
Log: "EXPIRED measurement-pending {id} — resolves_by {resolves_by} passed, channel still empty"
Continue to next hypothesis.
# ── end Step 2.6a — records below have no passed hard deadline; (re-)shelve ──
# Classify the gap subtype based on attempt failure mode
gap_subtype = (
"infra-missing" IF writer/script doesn't exist OR returned no records
"session-empty" IF source schema present but zero matching records
"schema-missing" IF source records exist but expected field absent
"log-inaccessible" IF log file not findable at expected path
"other" otherwise
)
# Build measurement-pending merge JSON
merge = {
"stage": "measurement-pending", # set by pipeline-move
"measurement_pending": true,
"measurement_pending_set_at": "<now-iso>",
"measurement_gap_subtype": gap_subtype,
"measurement_gap_detail": "<one-sentence explanation of what was probed and what came back>",
"context_consulted": <as in Step 4.1>,
}
echo '<merge-json>' | bash core/scripts/pipeline-move.sh <id> measurement-pending
# Skip Step 3, Step 4. The hypothesis is shelved until evidence appears.
Add to resolve_result.measurement_pending list (separate from newly_resolved).
Continue to next hypothesis.
ELSE: leave in active (resolution date arrived but channel returned a value
that could not be classified — judgment call still possible next cycle)
Migration: on first run after this step ships, all 5 hypotheses listed in
g-115-462 investigation report transition automatically. Subsequent runs
re-probe the channel — if data appears, the hypothesis moves through the
normal Step 2/3/4 path (now starting from measurement-pending instead of
active).
Step 3: Record Outcome
For each resolved hypothesis:
1. Determine outcome:
- Compare our_hypothesis against actual_outcome
- outcome: "CONFIRMED" if hypothesis matches outcome, "CORRECTED" if not
2. Calculate surprise level:
# Single source of truth — same formula as reflect --batch-micro Step 3.
- if CORRECTED: surprise_level = round(our_confidence * 10)
- if CONFIRMED: surprise_level = round((1 - our_confidence) * 10)
- High surprise = high confidence + wrong, or low confidence + right
3. Update the pipeline record with:
- outcome: CONFIRMED | CORRECTED
- our_confidence: (preserved from hypothesis)
- surprise_level: (calculated above)
- resolution_summary: "CONFIRMED — predicted {X} with {confidence}% confidence"
or "CORRECTED — predicted {X}, actual was {Y}"
4. Include outcome in resolution output:
"Result: CONFIRMED — predicted YES with 72% confidence"
or "Result: CORRECTED — predicted YES with 72% confidence, actual was NO"
Step 3.5: Broad Re-Retrieve on High Surprise (G3 / R5)
When a resolution carries surprise_level >= 7, the just-recorded outcome likely
falsifies one or more downstream beliefs / reasoning-bank entries / pattern
signatures that the hypothesis was implicitly endorsing. Step 1.5's batch
retrieval was shallow-to-medium — adequate for predicting outcomes, but
insufficient for finding ALL the entries a surprising correction may affect.
Per .claude/rules/retrieve-before-deciding.md decision point 3 ("resolving a
surprising hypothesis"), this step retrieves category-broadly BEFORE the
atomic move so the Tree Update Protocol (Steps 4.5 + Tree Steps 1-5) and any
downstream /reflect calls operate on a complete reconciliation candidate set.
IF surprise_level >= 7:
# Broad retrieve at deep depth — supplementary stores AND tree nodes
Bash: retrieve.sh --category {hypothesis.category} --depth deep
From the returned JSON, mark candidates for reconciliation review:
- beliefs[] whose claim predicts or depends on the just-falsified outcome
- reasoning_bank[] entries whose content endorses the (now-falsified) prediction
- pattern_signatures[] whose trigger matches the resolution shape
- tree_nodes[] whose capability_level was anchored on this outcome class
Append to the merge JSON built in Step 4.1:
reconciliation_candidates:
beliefs: [bel-NNN, ...] # IDs only — /reflect dereferences later
reasoning_bank: [rb-NNN, ...]
pattern_signatures: [sig-NNN, ...]
tree_nodes: [<node.key>, ...]
If retrieve.sh returns nothing relevant to the falsification:
reconciliation_candidates: { beliefs: [], reasoning_bank: [],
pattern_signatures: [], tree_nodes: [] }
(still record the empty manifest — proof retrieval was attempted)
# E10: Cross-reference confidence recalibration on high-surprise CORRECTED
# outcomes. Phase 8 will encode the ONE node it picks; this loop touches
# ALL cited nodes that didn't make it into the encoding bundle but were
# implicitly endorsing the falsified prediction. Without this, the cited
# nodes retain their pre-correction confidence and look authoritative on
# the next retrieve.
IF outcome == "CORRECTED" AND context_consulted.tree_nodes_read is non-empty:
For each node_key in context_consulted.tree_nodes_read:
Bash: bash core/scripts/tree-read.sh --node <node_key>
IF read failed (node missing or error): SKIP this node
Parse result JSON → old_confidence = result.confidence (default 0.0)
new_confidence = max(0.0, round(old_confidence - 0.05, 2))
IF new_confidence == old_confidence: SKIP (already at floor)
Bash: echo '{"operations": [{"op": "set", "key": "<node_key>", "field": "confidence", "value": <new_confidence>}, {"op": "set", "key": "<node_key>", "field": "last_update_trigger", "value": "surprise-recalibration"}]}' | bash core/scripts/tree-update.sh --batch
Log: "RECALIBRATED {node_key}: confidence {old_confidence} → {new_confidence} (hypothesis {hypothesis.id} surprise={surprise_level})"
# Fail-open: if a tree-update call errors, log and continue. Do NOT
# block the atomic resolve in Step 4 on a recalibration failure.
ELIF surprise_level >= 5:
# Medium-surprise — re-use Step 1.5 cached batch retrieval, no new probe
Carry forward the same context_consulted manifest; do not extend it
ELSE:
# Low surprise (≤4): hypothesis was well-calibrated, no broad re-retrieve
Skip this step entirely
This step is fail-open: if the retrieve.sh call errors, log the failure and
record reconciliation_candidates: { error: "<message>" } — proceed to Step 4.
A failed broad retrieve must NOT block the atomic resolve write.
Step 4: Move Resolved Hypotheses (ATOMIC — complete ALL steps for each hypothesis)
For each resolved hypothesis, execute this checklist IN ORDER.
□ 4.0 CHECK EXPIRATION (hypothesis goals only):
IF the record has resolves_by AND resolves_by has passed AND outcome is still undetermined:
- echo '{"outcome":"EXPIRED"}' | bash core/scripts/pipeline-move.sh <id> archived
- Return "EXPIRED" to calling skill
- SKIP remaining steps 4.1-4.2 for this record
□ 4.1 BUILD merge JSON with:
- Outcome data (outcome: CONFIRMED/CORRECTED, surprise, outcome_date)
- outcome_detail: resolution summary text
- horizon: (preserve from original — defaults to "short" if missing)
# Horizon-dependent metadata:
# short/long horizon — full metadata:
- replay_metadata:
last_replayed: null
replay_count: 0
encoding_score: null
reconsolidation_updates: []
- Preserve context_consulted from original record (populated during evaluation):
tree_nodes_read, pattern_signatures_checked, data_sources_used,
articles_read, experiential_matches, context_gaps_identified
- Initialize context_quality:
usefulness: pending
most_valuable_source: null
least_valuable_source: null
chain_note: null
# Compute process_score inline (not deferred to reflect)
IF outcome == "CONFIRMED" AND confidence >= 0.60:
dual_classification = "earned_confirmed"
ELIF outcome == "CONFIRMED" AND confidence < 0.60:
dual_classification = "lucky_confirmed"
ELIF outcome == "CORRECTED" AND confidence >= 0.60:
dual_classification = "unlucky_corrected"
ELIF outcome == "CORRECTED" AND confidence < 0.60:
dual_classification = "deserved_corrected"
process_quality = confidence if CONFIRMED else (1.0 - confidence)
# session horizon — reduced metadata:
# SKIP replay_metadata, context_quality, process_score
# SKIP context_consulted unless already present on the record
- Source cross-validation (ALL horizons):
source_validation: {source_validation object from Step 2.5}
- Learning status (ALL horizons):
reflected: false
reflected_date: null
□ 4.2 ATOMIC MOVE+UPDATE (single script call):
# The merge JSON MUST include ALL fields from Step 4.1 above.
# For short/long horizon, this means: outcome, outcome_detail, outcome_date,
# surprise, replay_metadata, context_quality, process_score, reflected: false.
# DO NOT skip fields — pipeline-move.sh merges them atomically.
# Missing fields here = missing fields forever (reflect can't backfill structure).
# RESOLUTION-EVIDENCE GATE (g-303-27): a CONFIRMED/CORRECTED move to
# resolved is REJECTED (400 resolution_evidence_required) unless
# outcome_detail (or experience_ref / evidence_for) carries >=1
# verifiable pointer -- a goal-id (g-NNN-NN), commit SHA, file:line,
# session-id, an rb-/guard-/exp-/msg- id, a canonical-script name
# (foo.sh/foo.py), or a percentage with measurement context. This is
# almost always already present in a real resolution summary. For a
# genuinely pointer-free resolution (e.g. a math proof where the
# derivation IS the evidence), add "evidence_override": "<reason>" to
# the merge JSON. EXPIRED/UNRESOLVABLE outcomes are exempt.
echo '<merge-json-with-ALL-4.1-fields>' | bash core/scripts/pipeline-move.sh <id> resolved
This atomically: updates all fields, sets stage to resolved, recounts meta.
GATE: Do NOT proceed to the next hypothesis until the move completes successfully.
If the script exits non-zero, STOP and report the error — do not partially resolve.
Step 4.5: Rate Context Quality (Retrieval Protocol Phase 5)
For each hypothesis just resolved in Step 4, rate the context loaded during Step 1.5:
For each resolved hypothesis:
Read its context_consulted section
Rate usefulness:
- Did loaded tree nodes, patterns, guardrails, or beliefs help interpret the outcome?
- Correctly predicted + context supported reasoning → "helpful"
- Context present but didn't influence resolution interpretation → "neutral"
- Context pointed toward wrong interpretation → "misleading"
- Context had no bearing on this hypothesis → "irrelevant"
Identify most_valuable_source:
Which loaded item (tree node, pattern, guardrail, belief, experiential record)
was most useful? Format: "{layer}:{id}"
Identify least_valuable_source:
Which loaded item added least value or was noise? Format: "{layer}:{id}"
Write to resolved record's context_consulted.context_quality:
usefulness: {rating}
most_valuable_source: {layer:id}
least_valuable_source: {layer:id}
chain_note: "{one-sentence explanation}"
If no context was loaded (empty context_consulted): set usefulness: "irrelevant"
If unable to judge: leave usefulness: "pending" (finalized by /reflect)
Step 5: Check for Triggered Reviews
Check if any auto-review triggers are met:
| Trigger | Condition | Action |
|---|
| Streak break | 3+ consecutive corrected | Flag for priority learning |
| High-confidence miss | Confidence > 0.80 AND corrected | Flag for priority violation analysis |
| Low-confidence hit | Confidence < 0.50 AND confirmed | Flag for underconfidence review |
| New category success | First confirmed in a category | Flag as potential strength |
| Stale hypothesis | Active > 90 days, no resolution | Check if hypothesis is still trackable |
| Category drift | 5+ hypotheses in one category, 0 in others | Flag for exploration |
If triggered: write triggered review flags to the resolve_result (consumed by callers for alerting).
Step 6: Return Resolve Result
Return a structured summary (consumed by /boot for reporting, and by --full-cycle for chaining):
resolve_result:
newly_resolved: N
already_resolved: M
still_active: K
skipped_not_due: J
resolving_soon: L
measurement_pending: P
expired_measurement_pending: Q
triggered_reviews: [...]
resolved_hypotheses:
- id: "2026-03-15_record-slug"
question: "Will X happen?"
predicted: "YES"
confidence: 0.72
actual: "NO"
outcome: CORRECTED
surprise_level: 7
reflected: false
Tree Update Protocol (after resolution)
After resolving any hypothesis, update the memory tree:
Tree Step 1: Recalculate Category Accuracy
category = resolved_hypothesis.category # e.g., "crypto", "sports-nhl", "politics"
Recount: confirmed / total resolved in this category
Tree Step 2: Update Affected Node (Dynamic Lookup)
node=$(bash core/scripts/tree-find-node.sh --text "{category}" --leaf-only --top 1)
# Returns: {key, score, file, depth, summary, node_type}
Read node.file
Update _tree.yaml via batch (accuracy, sample_size, confidence, capability_level):
echo '{"operations": [
{"op": "set", "key": "<node.key>", "field": "accuracy", "value": <new-value>},
{"op": "set", "key": "<node.key>", "field": "sample_size", "value": <new-value>},
{"op": "set", "key": "<node.key>", "field": "confidence", "value": <new-value>},
{"op": "set", "key": "<node.key>", "field": "capability_level", "value": "<new-value>"}
]}' | bash core/scripts/tree-update.sh --batch
Tree Step 3: Update Performance Tracking Node (Dynamic Lookup)
# Use dynamic lookup — never hardcode paths at any depth.
perf_node=$(bash core/scripts/tree-find-node.sh --text "performance-tracking" --top 1)
If perf_node exists:
Read perf_node.file
Update outcome record, category outcome breakdown
Tree Step 4: Update Accuracy Tracking Node (Dynamic Lookup)
# Use dynamic lookup — never hardcode paths at any depth.
acc_node=$(bash core/scripts/tree-find-node.sh --text "hypothesis-accuracy" --top 1)
If acc_node exists:
Read acc_node.file
Update overall accuracy, category breakdown, batch data
Tree Step 5: Propagate Capability Changes
If any node's capability_level changed:
result=$(bash core/scripts/tree-propagate.sh <node.key>)
# Returns: {source_node, ancestors_updated: [...], capability_changes: [...]}
IF result.capability_changes is non-empty:
For each changed ancestor: Read ancestor.file (.md)
- Update parent node's capability map and topic summary
If root-level domain summary affected:
bash core/scripts/tree-update.sh --set root summary "<updated>"
Log: "CAPABILITY UNLOCK: {topic} → {new_level}"
Announce in output: "Category {X} unlocked {LEVEL} level!"
Mode 2: Learn (--learn)
Processes resolved hypotheses that have NOT yet been reflected on. This is the learning phase — it calls /reflect to generate ABC chains, violation tracking, and pattern updates.
Step 1: Load Unreflected Hypotheses
Bash: pipeline-read.sh --unreflected
Sort by surprise descending (learn from surprises first)
If no unreflected records: return { hypotheses_learned: 0 } and exit
Step 2: Reflect on Each Hypothesis
For each unreflected hypothesis:
1. Bash: pipeline-read.sh --id {id} (loads the full resolved record)
2. invoke /reflect --on-hypothesis {hypothesis-id}
This generates:
- ABC chain (Antecedent-Behavior-Consequence)
- Textual reflection
- Violation tracking (if wrong)
- Source assessment
- Encoding score
- Pattern signature updates
3. After /reflect completes successfully:
- Bash: pipeline-update-field.sh {id} reflected true
(auto-sets reflected_date to today)
Step 2.5: Differentiated Extraction Handoff
When calling /reflect for each unlearned resolution:
- Pass the hypothesis outcome (CONFIRMED/CORRECTED) to /reflect
- /reflect will use differentiated extraction (confirmed → strategy validation, corrected → preventive guardrails)
- After /reflect completes, verify that:
- A reasoning bank entry was created via
reasoning-bank-read.sh --id rb-NNN
- For corrected outcomes: a guardrail was added via
guardrails-read.sh --id guard-NNN
- If /reflect did not create expected entries, create them directly:
- Reasoning bank entry: pipe JSON to
reasoning-bank-add.sh (stdin).
Required JSON fields: title, content, category, type, when_to_use,
tags, applies_to (one of any|framework|domain|specific).
Hypothesis-derived entries: pick applies_to by hypothesis subject —
framework-quality / pipeline-coordination hypotheses → framework;
deployment-domain hypotheses (the specific services, products, or
workflows this agent is deployed into) → domain; methodology
insights (calibration, pattern-recognition) → any; single-record
diagnostic with no transferable shape → specific.
- Guardrail entry: pipe JSON to
guardrails-add.sh (stdin)
Step 3: Return Learn Result
learn_result:
hypotheses_learned: N
hypotheses_already_learned: M
violations_recorded: K
patterns_updated: J
learned_hypotheses:
- id: "2026-03-15_record-slug"
outcome: CORRECTED
violation_type: "high-confidence miss"
encoding_score: 0.72
Mode 3: Accuracy Report (--accuracy-report)
Step 1: Load All Resolved Hypotheses
Bash: pipeline-read.sh --stage resolved (all resolved records)
Bash: meta-read.sh meta-knowledge/_index.yaml # existing meta-data
# Include micro-hypothesis batch stats from pipeline metadata.
# Micro stats are stored in pipeline-meta.json under micro_hypothesis_stats.
# These aggregate across sessions — each session-end consolidation appends batch totals.
Bash: pipeline-read.sh --meta → micro_hypothesis_stats (if exists)
Step 2: Calculate Metrics
accuracy:
overall:
total: 25
confirmed: 18
corrected: 7
accuracy_pct: 72.0
trend: "improving"
by_category:
politics: {total, confirmed, accuracy, confidence_avg, calibration_error}
crypto: {total, confirmed, accuracy, confidence_avg, calibration_error}
by_hypothesis_type:
high-conviction: {total, confirmed, accuracy}
calibration: {total, confirmed, accuracy}
exploration: {total, confirmed, accuracy}
contrarian: {total, confirmed, accuracy}
by_evaluation_method:
system-1: {total, confirmed, accuracy}
system-2: {total, confirmed, accuracy}
by_research_depth:
quick: {total, confirmed, accuracy}
moderate: {total, confirmed, accuracy}
deep: {total, confirmed, accuracy}
by_time_horizon:
micro: {total, confirmed, accuracy}
session: {total, confirmed, accuracy}
short_term_7d: {total, confirmed, accuracy}
medium_term_30d: {total, confirmed, accuracy}
long_term_90d: {total, confirmed, accuracy}
confidence_calibration:
bins:
"50-59%": {predicted, actual, count, error}
"60-69%": {predicted, actual, count, error}
"70-79%": {predicted, actual, count, error}
"80-89%": {predicted, actual, count, error}
"90-100%": {predicted, actual, count, error}
source_reliability:
- source: "source-id"
times_used: 12
led_to_confirmed: 9
reliability: 0.75
streaks:
current: 3
longest_confirmed: 5
longest_corrected: 2
Step 3: Hypothesis Testing
If 20+ resolved hypotheses, test hypotheses:
H1: "Deep research outperforms quick evaluations"
H2: "Category X outperforms category Y"
H3: "Hypotheses with source X outperform those without"
H4: "Higher-confidence hypotheses are more predictable"
H5: "Shorter time horizons are more predictable (compare micro→session→short→long)"
H6: "Micro-hypothesis accuracy correlates with self-model accuracy (are we good at predicting our own behavior?)"
For each: compare groups, note significance, write to
world/knowledge/strategies/hypothesis-results.md
Step 4: Update Meta-Memory
Bash: meta-set.sh meta-knowledge/_index.yaml # update with all accuracy figures
Update aspirations meta via Bash: `aspirations-meta-update.sh --source agent <field> <value>`
Step 5: Strategy Recommendations
1. Categories to focus on (accuracy > 70%)
2. Categories to reduce (accuracy < 40%)
3. Confidence calibration adjustments
4. Evaluation weight adjustments
5. Research depth recommendations
6. Source reliability actions
Mode 4: Full Cycle (--full-cycle)
The comprehensive weekly review. Chains all modes plus deep reflection and replay.
1. invoke /review-hypotheses --resolve (detect outcomes — idempotent if already done)
2. invoke /review-hypotheses --learn (reflect on unlearned — idempotent if already done)
3. invoke /review-hypotheses --accuracy-report
4. invoke /reflect --full-cycle (pattern extraction, calibration, replay)
5. Run spark check for new aspirations/goals
6. Propose strategy adjustments to /aspirations evolve
Bash: echo "review-hypotheses phase documented"
Idempotency note: If /boot already ran --resolve this session, step 1 is a no-op (nothing new to resolve). If aspiration goals already ran --learn, step 2 is a no-op (all records already reflected). The full-cycle still adds value via steps 3-6 which operate at the aggregate level.
Return Protocol
See .claude/rules/return-protocol.md — last action must be a tool call, not text.
Chaining
- Called by:
/aspirations (as goal skill for hypothesis goals, --resolve/--learn for batch ops), /boot (--resolve only for catch-up)
- Calls (by mode):
--resolve: None (standalone — detects outcomes, records results)--learn: /reflect --on-hypothesis (for each unreflected resolution)--full-cycle: --resolve, --learn, --accuracy-report, /reflect --full-cycle (which includes --batch-micro)
- Feeds into:
/aspirations evolve (strategy recommendations)
- Updated by:
/reflect (provides patterns and strategies back)
