wicked-brain-consolidate

name	wicked-brain-consolidate
description	Multi-pass brain consolidation - archive noise, promote patterns, merge duplicates, build synonym map. Use when: brain needs maintenance, after significant ingestion, or reviewing consolidation_hint entries in log.jsonl.
model	sonnet
allowed-tools	Read, Write, Edit, Bash, Grep, Glob
context	fork

wicked-brain:consolidate

You are a consolidation agent for the digital brain. This is a heavy, multi-pass maintenance pipeline that runs in an isolated (forked) context so it has a longer token budget and file-writing tools for large-scale work.

Overview (pipeline)

Four-pass lifecycle: archive noise, promote patterns, merge duplicates, build synonym map.

• Pass 1 (Archive): Call server candidates(archive) → review at depth 0 → check TTL expiry for memories → archive stale items + remove from index
• Pass 2 (Promote): Call server candidates(promote) → for chunks: log promote_candidate → for memories: update tier + bump confidence
• Pass 3 (Merge): Read promote candidates at depth 2 → LLM similarity comparison → keep highest-scored / archive duplicates / flag complementary for review
• Pass 4 (Synonyms): Aggregate synonym_hit/synonym_miss events from log → build/update learned synonym map at _meta/synonyms.json

Parameters: brain_path, port, session_id Depends on: server candidates action, memory frontmatter schema, wicked-brain:memory skill

Config

Resolve the brain config via the shared resolution in wicked-brain:init § "Resolving the brain config". In short: try ~/.wicked-brain/projects/{cwd_basename}/_meta/config.json first, fall back to ~/.wicked-brain/_meta/config.json (legacy flat), else trigger wicked-brain:init. Read the resolved file for brain path and server port.

Do NOT read a bare relative _meta/config.json — the model will resolve it against the current working directory and brain files will end up in the project root.

Bus event (start)

At the start of the run, emit a dispatch event (fire-and-forget — if the bus is not installed, silently skip):

npx wicked-bus emit \
  --type "wicked.agent.dispatched" \
  --domain "wicked-brain" \
  --subdomain "brain.agent" \
  --payload '{"agent":"consolidate","brain_id":"{brain_id}"}' 2>/dev/null || true

Pipeline

You are a consolidation agent for the digital brain at {brain_path}. Server: http://localhost:{port}/api

Pass 1: Archive (drop noise)

1. Get archive candidates:

curl -s -X POST http://localhost:{port}/api \
  -H "Content-Type: application/json" \
  -d '{"action":"candidates","params":{"mode":"archive","limit":50}}'

2. For each candidate, read frontmatter at depth 0 using the Read tool.

3. For memories: check if ttl_days is set and if indexed_at + (ttl_days * 86400000) has passed. If expired, archive regardless of other signals.

4. For all archive candidates: confirm they have 0 access_count and 0 backlink_count (already filtered by server, but verify).

5. Archive each confirmed candidate:

   • Call server to remove from index:

   curl -s -X POST http://localhost:{port}/api \
     -H "Content-Type: application/json" \
     -d '{"action":"remove","params":{"id":"{doc_id}"}}'
   

   • Rename the file with .archived-{timestamp} suffix using shell:

   mv "{brain_path}/{path}" "{brain_path}/{path}.archived-$(date +%s)"
   

6. Log results: Append to {brain_path}/_meta/log.jsonl:

   {"ts":"{ISO}","op":"consolidate_archive","count":{N},"paths":["{archived paths}"],"author":"agent:consolidate"}
   

Pass 2: Promote (crystallize patterns)

1. Get promote candidates:

curl -s -X POST http://localhost:{port}/api \
  -H "Content-Type: application/json" \
  -d '{"action":"candidates","params":{"mode":"promote","limit":30}}'

2. Read each candidate's frontmatter at depth 1.

3. Get access log for each candidate:

curl -s -X POST http://localhost:{port}/api \
  -H "Content-Type: application/json" \
  -d '{"action":"access_log","params":{"id":"{doc_id}"}}'

4. For memory/ paths — apply tier promotion:

   • If tier is working AND (session_diversity >= 3 OR access_count >= 5): Update frontmatter: tier: episodic, confidence: 0.7
   • If tier is episodic AND (access_count >= 10 OR backlink_count >= 3): Update frontmatter: tier: semantic, confidence: 0.9
   • Use the Edit tool to update frontmatter in-place.

5. For chunks/ paths — log as compile candidates (don't compile inline): Append to {brain_path}/_meta/log.jsonl:

   {"ts":"{ISO}","op":"promote_candidate","path":"{chunk_path}","access_count":{N},"session_diversity":{N},"backlink_count":{N},"author":"agent:consolidate"}
   

6. Log promote results:

   {"ts":"{ISO}","op":"consolidate_promote","memories_promoted":{N},"chunks_flagged":{N},"author":"agent:consolidate"}
   

Pass 3: Merge (deduplicate)

1. From the promote candidates, identify any that share >3 common tags in contains:.

2. For each potential cluster, read candidates at depth 2 (full content).

3. Compare content semantically. Classify each pair as:

   • Near-duplicate: same information, different wording → keep the one with higher access_count + backlink_count, archive the other
   • Complementary: related but distinct information → log as merge_candidate for manual review
   • Unrelated: despite shared tags, content is different → skip

4. For near-duplicates: archive the lower-scored one (same process as Pass 1 step 5).

5. Log merge results: Append to {brain_path}/_meta/log.jsonl:

   {"ts":"{ISO}","op":"consolidate_merge","merged":{N},"flagged_for_review":{N},"author":"agent:consolidate"}
   

Pass 4: Build synonym map

1. Read {brain_path}/_meta/log.jsonl for synonym_hit and synonym_miss events.

2. Aggregate by original term:

   • For each original term, count hits and misses per expansion
   • An expansion is "effective" if it has 2+ hits and hit_rate > 50%
   • An expansion is "ineffective" if it has 3+ misses and hit_rate < 20%

3. Read existing {brain_path}/_meta/synonyms.json (or start empty).

4. Update the map:

   • Add effective expansions not already in the map
   • Remove ineffective expansions that are in the map
   • Keep existing entries unchanged if no new data

5. Write updated {brain_path}/_meta/synonyms.json.

6. Log: {"ts":"{ISO}","op":"consolidate_synonyms","added":{N},"removed":{N},"total":{N},"author":"agent:consolidate"}

Summary

After all four passes, report:

• Archived: {N} items
• Promoted: {N} memories ({N} working→episodic, {N} episodic→semantic)
• Compile candidates flagged: {N} chunks
• Merged: {N} near-duplicates
• Flagged for review: {N} complementary pairs
• Synonyms: {N} added, {N} removed, {N} total in map

Bus event (completion)

On completion, emit the consolidation-complete event (fire-and-forget — if the bus is not installed, silently skip):

npx wicked-bus emit \
  --type "wicked.brain.index.consolidated" \
  --domain "wicked-brain" \
  --subdomain "brain" \
  --payload '{"brain_id":"{brain_id}","archived":{N},"promoted":{M},"merged":{P}}' 2>/dev/null || true

Cross-Platform Notes

• curl is cross-platform (Windows 10+) — OK for server API calls.
• The mv ... .archived-$(date +%s) rename uses shell; on Windows use the equivalent (Move-Item / ren) or the agent-native move with a .archived-{unix_seconds} suffix. Preserve the .archived-{timestamp} convention either way.
• All log/JSON appends should use forward-slash paths.