Nightly memory consolidation — prunes stale entries, merges duplicates, resolves contradictions, rebuilds MEMORY.md index. Use when memory files have accumulated over many sessions and need cleanup. Do NOT use for storing new decisions (use remember) or searching memory (use memory).
Nightly memory consolidation — prunes stale entries, merges duplicates, resolves contradictions, rebuilds MEMORY.md index. Use when memory files have accumulated over many sessions and need cleanup. Do NOT use for storing new decisions (use remember) or searching memory (use memory).
argument-hint
[--dry-run]
tags
["memory","maintenance","consolidation"]
version
1.0.0
author
OrchestKit
user-invocable
true
allowed-tools
["Read","Write","Edit","Glob","Grep","Bash"]
complexity
medium
context
inherit
persuasion-type
collaborative
effort
low
model
sonnet
triggers
{"keywords":["dream","consolidate","clean memory","prune memory","memory cleanup","stale memories","merge memories","memory maintenance","tidy memory","stale memory entries","memory files","prune memories"],"examples":["consolidate my memory files","clean up stale memory entries","run dream to prune old memories"],"anti-triggers":["remember","save","store","search","recall","load context","implement","explore"]}
Dream - Memory Consolidation
Deterministic memory maintenance: detect stale entries, merge duplicates, resolve contradictions, rebuild the MEMORY.md index. All pruning decisions are based on verifiable checks (file exists? function exists? duplicate content?), not LLM judgment.
Argument Resolution
DRY_RUN = "--dry-run"in"$ARGUMENTS"# Preview changes without writing
Overview
Memory files accumulate across sessions. Over time they develop problems:
Stale references — memories pointing to files, functions, or classes that no longer exist
Duplicates — multiple memories covering the same topic with overlapping content
Contradictions — newer memories superseding older ones without cleanup
Index drift — MEMORY.md index out of sync with actual memory files
This skill fixes all four problems using deterministic checks only.
Cadence (CC 2.1.142+): Reactive compaction now sizes its first summarize attempt to the actual overflow, so long sessions stall mid-turn far less often. The "run nightly" cadence can relax toward "run when memory files accumulate" — consolidation is no longer needed to head off compaction inefficiency.
STEP 1: Discover Memory Files
# Find the memory directory (agent-specific or project-level)# Agent memory lives in: .claude/agent-memory/<agent-id>/# Project memory lives in: .claude/projects/<hash>/memory/# Also check: .claude/memory/
memory_dirs = []
Glob(pattern=".claude/agent-memory/*/MEMORY.md")
Glob(pattern=".claude/projects/*/memory/MEMORY.md")
Glob(pattern=".claude/memory/MEMORY.md")
# For each discovered MEMORY.md, glob all *.md files in that directoryfordirin memory_dirs:
Glob(pattern=f"{dir}/../*.md") # All memory files alongside MEMORY.md
Read every discovered memory file. Parse frontmatter (, , ) and body content. Build an in-memory inventory:
For each memory file, extract references and verify they still exist.
2a: File Path References
Extract paths that look like file references (patterns: paths with / and file extensions, backtick-wrapped paths):
# Regex-like extraction from body text:# - Paths containing / with common extensions: .py, .ts, .tsx, .js, .json, .md, .yaml, .yml, .sh# - Backtick-wrapped paths: `src/something/file.ts`# - Quoted paths in frontmatter descriptions
Classify each ref's SCOPE before verifying it.Glob only sees the current repo, so a path that
lives anywhere else can never match and would otherwise be scored as missing. A memory about
~/.claude hooks, a homebrew cask, a cmux config, or another repo is not stale just because this
repo does not contain it.
defscope(ref):
# Anything rooted outside the working repo is UNVERIFIABLE, not missing.if ref.startswith(("~", "/", "$")): return"UNVERIFIABLE"if ref.startswith(("http://", "https://")): return"UNVERIFIABLE"if re.match(r'^[A-Za-z0-9_.-]+/', ref) andnot (REPO / ref.split("/")[0]).exists():
return"UNVERIFIABLE"# first segment is not a real top-level dir herereturn"REPO_RELATIVE"
verifiable = [r for r in file_refs if scope(r) == "REPO_RELATIVE"]
external = [r for r in file_refs if scope(r) == "UNVERIFIABLE"]
missing = []
for ref in verifiable:
Glob(pattern=ref)
# If no match → missing.append(ref)
The staleness ratio is computed over verifiable ONLY.external refs are recorded for the
report and never counted toward pruning. A memory with zero verifiable refs is EVERGREEN no
matter how many external paths it names.
for symbol in symbol_refs:
Grep(pattern=symbol, path=".", output_mode="files_with_matches", head_limit=1)
# If no match → mark as STALE_SYMBOL_REF
2c: Staleness Classification
Finding
Classification
Action
Zero VERIFIABLE refs (none, or all UNVERIFIABLE)
EVERGREEN
Keep
All verifiable refs valid, all symbols found
FRESH
Keep
Some verifiable refs missing
PARTIALLY_STALE
Flag for review
All verifiable refs missing AND all symbols missing
FULLY_STALE
Prune candidate
Only memories classified as FULLY_STALE are auto-pruned. PARTIALLY_STALE memories are reported but kept — the user decides.
2d: Prune guards — checked AFTER classification, before any delete
FULLY_STALE is necessary but not sufficient to delete. Every guard below downgrades to
PARTIALLY_STALE (kept + flagged). These exist because memory files are not in git: a wrong
delete is silent and unrecoverable, so the asymmetry always favours keeping.
GUARD_DAYS = 14for m inlist(fully_stale_files):
reason = None# 1. Preferences do not decay because a path moved.if m["type"] == "user":
reason = "type:user is never auto-pruned"# 2. A feedback/reference memory carries a LESSON; the file paths in it are# illustrations, not a manifest. Its worth does not expire when an# illustrative path moves, and ref-extraction is lossy anyway (it catches# `file.ts` but misses `file.ts:186` and `functionName()`). Only project# memories — which track live work against concrete files — are eligible# to go fully stale on ref death.elif m["type"] in ("feedback", "reference"):
reason = f"type:{m['type']} value is the lesson, not its file refs"# 3. Recently written memories describe the present, whatever their refs say.elif (now - m["mtime"]) < GUARD_DAYS * 86400:
reason = f"modified within {GUARD_DAYS}d"# 4. A memory that exists to prevent a regression must outlive the code it cites.elif re.search(r'\b(do not|don\'t|never|avoid)\b', m["body"], re.I):
reason = "carries a do-not/never directive"if reason:
m["classification"] = "PARTIALLY_STALE"
m["kept_reason"] = f"prune-guard: {reason}"
fully_stale_files.remove(m)
partially_stale_files.append(m)
Guard 3 is the subtle one. reference_cmux_scroll_blank_research said "RESOLVED; do NOT re-suggest
tui:fullscreen on cmux" and every path it cited had moved. Deleting it reintroduces exactly the
regression it was written to prevent. A memory whose value is a prohibition is at its most useful
precisely when the original code is gone.
STEP 2.5: Consult-gate (#2351) — never prune a memory that's still being used
Closing the VERIFY loop: a deletion must survive the question "was this actually consulted?". A memory whose external refs all vanished (FULLY_STALE) but that the agent keeps looking up is still load-bearing — its refs are stale, its knowledge is live. So before pruning, read .claude/logs/memory-consult.jsonl (written by memory-validator on every mcp__memory__search_nodes/open_nodes/read_graph) and downgrade any recently-consulted FULLY_STALE memory to PARTIALLY_STALE (kept + flagged, not auto-deleted).
import json, time
from pathlib import Path
defrecently_consulted_terms(days=14):
log = Path(".claude/logs/memory-consult.jsonl")
ifnot log.exists():
returnset()
cutoff = time.time() - days * 86400
terms = set()
for line in log.read_text().splitlines():
try:
e = json.loads(line)
except ValueError:
continue# best-effort: skip malformed lines# open_nodes carries exact entity names; search carries a query string
terms.update(n.lower() for n in e.get("names", []))
if e.get("query"):
terms.update(w.lower() for w in e["query"].split() iflen(w) > 2)
return terms
consulted = recently_consulted_terms()
for m inlist(fully_stale_files):
slug = Path(m["path"]).stem.lower()
name = (m.get("name") or"").lower()
if name in consulted orany(c in slug or slug in c for c in consulted):
m["classification"] = "PARTIALLY_STALE"
m["kept_reason"] = "consult-gate: looked up in the last 14 days (#2351)"
fully_stale_files.remove(m)
partially_stale_files.append(m)
This is conservative by design — fuzzy term matching errs toward keeping a maybe-consulted memory rather than deleting a live one. The Step 6 report records each consult-gated keep (the "did it matter?" audit the loop was missing). If the log is absent (consult instrumentation not yet exercised), the gate is a no-op and pruning proceeds as before.
STEP 3: Detect Duplicates
Compare memories pairwise within the same directory. Two memories are duplicates when:
Same type (both feedback, both project, etc.)
Overlapping topic — 60%+ of significant words (excluding stopwords) appear in both bodies
Same subject — name or description fields reference the same concept
stopwords = {"the", "a", "an", "is", "are", "was", "were", "be", "been",
"have", "has", "had", "do", "does", "did", "will", "would",
"could", "should", "may", "might", "can", "shall", "to", "of",
"in", "for", "on", "with", "at", "by", "from", "as", "into",
"through", "during", "before", "after", "this", "that", "it",
"not", "no", "but", "or", "and", "if", "then", "than", "so"}
defsignificant_words(text):
words = set(text.lower().split()) - stopwords
return {w for w in words iflen(w) > 2}
defoverlap_ratio(words_a, words_b):
ifnot words_a ornot words_b:
return0.0
intersection = words_a & words_b
smaller = min(len(words_a), len(words_b))
returnlen(intersection) / smaller if smaller > 0else0.0# For each pair with same type:# if overlap_ratio >= 0.6 → DUPLICATE pair# Keep the NEWER file (by filesystem mtime), prune the older
STEP 4: Resolve Contradictions
Contradictions occur when two memories of the same type make opposing claims about the same subject. Detection:
Same type + same topic (overlap >= 0.4 but < 0.6 — related but not duplicate)
Negation signals — one body contains negation of the other's assertion:
"do X" vs "do not X" / "don't X" / "never X"
"use X" vs "avoid X" / "stop using X"
"prefer X" vs "prefer Y" (for same decision domain)
negation_pairs = [
("do ", "do not "), ("do ", "don't "),
("use ", "avoid "), ("use ", "stop using "),
("prefer ", "don't prefer "), ("always ", "never "),
]
# For each pair flagged as contradictory:# Keep the NEWER file (more recent decision supersedes)# Prune the older file
STEP 5: Execute Changes (or Dry Run)
Dry Run Mode (--dry-run)
If --dry-run flag is present, skip all writes. Output the full report (Step 6) with [DRY RUN] prefix and list what WOULD be changed:
[DRY RUN] Would delete: .claude/agent-memory/foo/stale_old_path.md (FULLY_STALE)
[DRY RUN] Would delete: .claude/agent-memory/foo/duplicate_auth.md (DUPLICATE of auth_patterns.md)
[DRY RUN] Would delete: .claude/agent-memory/foo/old_preference.md (CONTRADICTED by new_preference.md)
[DRY RUN] Would rebuild: .claude/agent-memory/foo/MEMORY.md (3 entries removed, 12 remaining)
Live Mode
# 1. Delete FULLY_STALE filesfor stale in fully_stale_files:
Bash(command=f"rm '{stale['path']}'")
# 2. Delete DUPLICATE files (keep newer)for dup in duplicate_pairs:
older = dup["older"]
Bash(command=f"rm '{older['path']}'")
# 3. Delete CONTRADICTED files (keep newer)for contradiction in contradiction_pairs:
older = contradiction["older"]
Bash(command=f"rm '{older['path']}'")
# 4. Rebuild MEMORY.md index from surviving files
Rebuild MEMORY.md
Read all surviving .md files (excluding MEMORY.md itself). Generate the index:
# <DirectoryName> Memory- [Name](filename.md) -- one-line description from frontmatter
Rules for the rebuilt index:
One line per memory file, sorted alphabetically by filename
The binding constraint is BYTES, not lines. MEMORY.md is loaded every session and stops
loading past the read limit (~24 KB), at which point the whole index silently degrades. Line
count is a proxy that misses this: a 151-entry index at a 147-char mean is 22.5 KB and nearly
dead, while the same 151 entries at 112 chars is 16.2 KB and healthy.
Target ≤ 17 KB total. Derive the per-line budget rather than hardcoding it:
budget_chars = (17 * 1024 - non_entry_overhead) / entry_count
If the rebuild exceeds the target, trim hooks to the derived budget before dropping any entry.
Truncate at a word boundary and keep the leading clause (it carries the discriminating detail).
Every memory file must remain represented 1:1 — verify indexed == files_on_disk after writing.
Only if trimming to ~90 chars still overflows should you warn the user. Never auto-delete a memory
to fit the index; the index is a pointer table, and shrinking it is a formatting problem, not a
retention one.
# Write the rebuilt MEMORY.md
Write(path="<memory_dir>/MEMORY.md", content=rebuilt_index)
STEP 6: Report
Output a summary table after consolidation:
## Dream Consolidation Report
| Metric | Count |
|--------|-------|
| Memory directories scanned | N |
| Total memory files scanned | N |
| Stale entries pruned | N |
| Duplicates merged | N |
| Contradictions resolved | N |
| Partially stale (kept, flagged) | N |
| Evergreen (no external refs) | N |
| Surviving memories | N |
| MEMORY.md indexes rebuilt | N |
| Promotion candidates (2+ repos, STEP 9) | N |
### Changes Made
| File | Action | Reason |
|------|--------|--------|
| `path/to/file.md` | DELETED | Fully stale: all referenced files removed |
| `path/to/old.md` | DELETED | Duplicate of `path/to/new.md` |
| `path/to/outdated.md` | DELETED | Contradicted by `path/to/current.md` |
### Flagged for Review (PARTIALLY_STALE)
| File | Missing References |
|------|-------------------|
| `path/to/file.md` | `src/old/path.ts` no longer exists |
If --dry-run, prefix the entire report with:
[DRY RUN] No files were modified. Run without --dry-run to apply changes.
Error Handling
Condition
Response
No memory directories found
Report "No memory directories found" and exit
No memory files in directory
Report "Directory empty, nothing to consolidate"
All memories are FRESH
Report "All N memories are current, nothing to prune"
MEMORY.md exceeds 200 lines after rebuild
Warn user, do not auto-truncate
File deletion fails
Report error, continue with remaining files
Memory file has no frontmatter
Treat as EVERGREEN (cannot verify refs without metadata)
STEP 7: Plugin Housekeeping (CC 2.1.121+, #1544)
After memory consolidation, check for orphaned auto-installed plugin dependencies and offer to prune them:
# Detect orphans
claude plugin list --json | jq '[.[] | select(.auto_installed == true and .reason_kept == "orphaned")] | length'# If > 0 and last prune > 7 days ago (track in .claude/state/last-prune.txt):
claude plugin prune # interactive — confirms before removing
Skip this step on CC < 2.1.121. The state file .claude/state/last-prune.txt records the last successful prune date so we don't run it on every dream invocation.
STEP 8: Stale Project State Hint (CC 2.1.126+, #1582, fixed in #1587)
After plugin housekeeping, surface a non-blocking suggestion when stale project state exists. Never execute the purge — only preview it.
# Skip on CC < 2.1.126 (no `claude project purge` available)# Detect stale projects via the authoritative source: `claude project purge --dry-run --all`# emits `config: projects["<canonical-path>"]` lines that come straight from ~/.claude.json.# Parsing these is lossless; the directory-name encoding under ~/.claude/projects/ is NOT# (both `/` and `.` collapse to `-`, so it cannot be reversed deterministically).
stale_count=$(claude project purge --dry-run --all 2>/dev/null \
| grep -oE 'projects\["[^"]+"\]' \
| sed -E 's/^projects\["//; s/"\]$//' \
| while IFS= read -r p; do
[ -n "$p" ] && [ ! -d "$p" ] && echo"$p"done | wc -l)
# If > 0, surface the hint in the dream summary (never auto-execute)if [ "$stale_count" -gt 0 ]; thenecho"ℹ $stale_count stale project state entries detected."echo" Preview cleanup with: claude project purge --dry-run --all"fi
Strict rules: always --dry-run, never --yes. Users who moved (not deleted) a project need to keep the directory; the purge is irreversible. Surface the suggestion, let the user decide.
Why parse claude project purge --dry-run --all instead of ~/.claude/projects/: the directory naming under ~/.claude/projects/ is a lossy collapse of the original path (/ and . both become -). A naive sed 's|-|/|g' decode misidentifies any path containing - (e.g. my-project → /my/project). The CLI's dry-run output reads canonical paths from ~/.claude.json and is the only reliable source.
STEP 9: Cross-Repo Promotion Candidates (#3295)
A memory pattern that shows up in 2+ projects is a capability that outgrew its repo.
While consolidating, detect these deterministically and offer promotion -- dream never
moves content itself, so this step stays safe when dream is model-invoked.
# For each memory file touched in this run, derive a topic key: the filename slug minus# scope words (dates, project names). Then look for the same key in OTHER projects'# memory indexes (index lines are "- [Title](file.md) -- hook"):
grep -l -i "<topic-key>" ~/.claude/projects/*/memory/MEMORY.md \
| grep -v "<current-project-dir>"
2+ distinct projects match -> the memory is a promotion candidate.
Deterministic only: match on normalized slug/title tokens, never on semantic judgment.
False positives are cheap (the user declines); silent misses are the failure mode this
step exists for -- the same infra lesson re-learned per repo, N times, with nothing watching.
Interactive runs: AskUserQuestion per candidate (batch when more than 3):
"Promote to a shared plugin" -- org-specific patterns go to the org's private plugin,
generic ones to a public plugin; dream only opens the door, the user routes.
"Stop suggesting this one" -- append promotion: declined to the memory's frontmatter
metadata so future runs skip it.
Non-interactive / dry runs: list candidates in the Dream Consolidation Report under
Promotion candidates: with the matching project paths. No prompt, no mutation.
When NOT to Use
To store new decisions -- use /ork:remember
To search past decisions -- use /ork:memory search
To load context at session start -- use /ork:memory load
After fewer than 5 sessions -- memory files are unlikely to have accumulated enough staleness
Related Skills
ork:remember -- Store decisions and patterns (write-side)
