Strategic Forgetting — Not remembering everything is a feature
Hierarchical Organization — Navigate categories, not scan linearly
From PageIndex
Vectorless Retrieval — LLM reasoning instead of embedding similarity
Tree-Structured Index — O(log n) navigation, not O(n) scan
Explainable Results — Every retrieval traces a path through categories
Reasoning-Based Search — "Why relevant?" not "how similar?"
Cloud-First (EvoClaw)
Device is replaceable — Soul lives in cloud (Turso)
Critical sync — Hot + tree sync after every conversation
Disaster recovery — Full restore in <2 minutes
Multi-device — Same agent across phone/desktop/embedded
Memory Tiers
🔴 Hot Memory (5KB max)
Purpose: Core identity and active context, always in agent's context window.
Structure:
{"identity":{"agent_name":"Agent","owner_name":"User","owner_preferred_name":"User","relationship_start":"2026-01-15","trust_level":0.95},"owner_profile":{"personality":"technical, direct communication","family":["Sarah (wife)","Luna (daughter, 3yo)"],"topics_loved":["AI architecture","blockchain","system design"],"topics_avoid":["small talk about weather"],"timezone":"Australia/Sydney","work_hours":"9am-6pm"},"active_context":{"projects":[{"name":"EvoClaw","description":"Self-evolving agent framework","status":"Active - BSC integration for hackathon"}],"events":[{"text":"Hackathon deadline Feb 15","timestamp":1707350400}],"tasks":[{"text":"Deploy to BSC testnet","status":"pending","timestamp":1707350400}]},"critical_lessons":[{"text":"Always test on testnet before mainnet","category":"blockchain","importance":0.9,"timestamp":1707350400}]}
Auto-pruning:
Lessons: Max 20, removes lowest-importance when full
Events: Keeps last 10 only
Tasks: Max 10 pending
Total size: Hard limit at 5KB, progressively prunes if exceeded
Generates:MEMORY.md — auto-rebuilt from structured hot state
🟡 Warm Memory (50KB max, 30-day retention)
Purpose: Recent distilled facts with decay scoring.
Entry format:
{"id":"abc123def456","text":"Decided to use zero go-ethereum deps for EvoClaw to keep binary small","category":"projects/evoclaw/architecture","importance":0.8,"created_at":1707350400,"access_count":3,"score":0.742,"tier":"warm"}
score < 0.05 → Frozen (delete after retention period)
Eviction triggers:
Age > 30 days AND score < 0.3
Total warm size > 50KB (evicts lowest-scored)
Manual consolidation
🟢 Cold Memory (Unlimited, Turso)
Purpose: Long-term archive, queryable but never bulk-loaded.
Schema:
CREATE TABLE cold_memories (
id TEXT PRIMARY KEY,
agent_id TEXT NOT NULL,
text TEXT NOT NULL,
category TEXT NOT NULL,
importance REALDEFAULT0.5,
created_at INTEGERNOT NULL,
access_count INTEGERDEFAULT0
);
CREATE TABLE critical_state (
agent_id TEXT PRIMARY KEY,
data TEXT NOT NULL, -- {hot_state, tree_nodes, timestamp}
updated_at INTEGERNOT NULL
);
Retention: 10 years (configurable)
Cleanup: Monthly consolidation removes frozen entries older than retention period
Tree Index
Purpose: Hierarchical category map for O(log n) retrieval.
Input: Raw conversation text
Output: Structured JSON
{"fact":"User decided to use raw JSON-RPC for BSC to avoid go-ethereum dependency","emotion":"determined","people":["User"],"topics":["blockchain","architecture","dependencies"],"actions":["decided to use raw JSON-RPC","avoid go-ethereum"],"outcome":"positive"}
# Rule-based (default)
distiller.py --text "Had a productive chat about the BSC integration..." --mode rule
# LLM-powered
distiller.py --text "..." --mode llm --llm-endpoint http://localhost:8080/complete
# With core summary
distiller.py --text "..." --mode rule --core-summary
Stage 2→3: Distilled → Core Summary
Purpose: One-line summary for tree index
Example:
Distilled: {
"fact": "User decided raw JSON-RPC for BSC, no go-ethereum",
"outcome": "positive"
}
Core summary: "BSC integration: raw JSON-RPC (no deps)"
Target: <30 bytes
LLM-Powered Tree Search
Purpose: Semantic search through tree structure using LLM reasoning.
How it works:
Build prompt with tree structure + query
LLM reasons about which categories are relevant
Returns category paths with relevance scores
Fetches memories from those categories
Example:
Query: "What did we decide about the hackathon deadline?"
Keyword search returns:
projects/evoclaw (0.8)
technical/deployment (0.4)
LLM search reasons:
projects/evoclaw/bsc (0.95) — "BSC integration for hackathon"
You are a memory retrieval system. Given a memory tree index and a query,
identify which categories are relevant.
Memory Tree Index:
projects/evoclaw — EvoClaw framework (warm:6, cold:45)
projects/evoclaw/bsc — BSC integration (warm:3, cold:12)
...
User Query: What did we decide about the hackathon deadline?
Output (JSON):
[
{"path": "projects/evoclaw/bsc", "relevance": 0.95, "reason": "BSC work for hackathon"},
{"path": "active_context/events", "relevance": 0.85, "reason": "deadline tracking"}
]
{"status":"warning","warnings_count":2,"warnings":["Tool 'Z-Image' mentioned without URL/documentation link","Action 'install' mentioned without command example"],"suggestions":["Add URLs for mentioned tools/services","Include command examples for setup/installation steps","Document next steps after decisions"]}
Extract Metadata (v2.1.0)
memory_cli.py extract-metadata --file PATH
Purpose: Extract structured metadata (URLs, commands, paths) from a file.
memory_cli.py search-url --url FRAGMENT [--limit 5] [--agent-id default]
Purpose: Search facts by URL fragment.
Example:
# Find all facts with comfy.org URLs
memory_cli.py search-url --url "comfy.org"# Find GitHub repos
memory_cli.py search-url --url "github.com" --limit 10
Output:
{"query":"comfy.org","results_count":1,"results":[{"id":"abc123","text":"Z-Image ComfyUI model for photorealistic images","category":"tools/image-generation","metadata":{"urls":["https://docs.comfy.org/tutorials/image/z-image/z-image"],"commands":["huggingface-cli download Tongyi-MAI/Z-Image"],"paths":[]}}]}
# Rule-based distillation
memory_cli.py distill --text "User: Let's deploy to testnet first. Agent: Good idea, safer that way."# LLM distillation
memory_cli.py distill \
--text "Long conversation with nuance..." \
--llm --llm-endpoint http://localhost:8080/complete
Output:
{"distilled":{"fact":"Decided to deploy to testnet before mainnet","emotion":"cautious","people":[],"topics":["deployment","testnet","safety"],"actions":["deploy to testnet"],"outcome":"positive"},"mode":"rule","original_size":87,"distilled_size":156}
Hot Memory
# Update hot state
memory_cli.py hot --update KEY JSON [--agent-id default]
# Rebuild MEMORY.md
memory_cli.py hot --rebuild [--agent-id default]
# Show current hot state
memory_cli.py hot [--agent-id default]
Keys:
identity — Agent/owner identity info
owner_profile — Owner preferences, personality
lesson — Add critical lesson
event — Add event to active context
task — Add task to active context
project — Add/update project
Examples:
# Update owner profile
memory_cli.py hot --update owner_profile '{"timezone": "Australia/Sydney", "work_hours": "9am-6pm"}'# Add lesson
memory_cli.py hot --update lesson '{"text": "Always test on testnet first", "category": "blockchain", "importance": 0.9}'# Add project
memory_cli.py hot --update project '{"name": "EvoClaw", "status": "Active", "description": "Self-evolving agent framework"}'# Rebuild MEMORY.md
memory_cli.py hot --rebuild
Tree
# Show tree
memory_cli.py tree --show [--agent-id default]
# Add node
memory_cli.py tree --add "path/to/category""Description" [--agent-id default]
# Remove node
memory_cli.py tree --remove "path/to/category" [--agent-id default]
# Prune dead nodes
memory_cli.py tree --prune [--agent-id default]
Examples:
# Add category
memory_cli.py tree --add "projects/evoclaw/bsc""BSC blockchain integration"# Remove empty category
memory_cli.py tree --remove "old/unused/path"# Prune dead nodes (60+ days no activity)
memory_cli.py tree --prune
Use cheaper models for frequent operations (distill, search)
Batch distillation — Queue conversations, distill in batch
Cache tree prompts — Tree structure doesn't change often
Skip LLM for simple — Use rule-based for short conversations
Example LLM Endpoint
from flask import Flask, request, jsonify
app = Flask(__name__)
@app.route("/complete", methods=["POST"])defcomplete():
data = request.json
prompt = data["prompt"]
# Call your LLM (OpenAI, Anthropic, local model, etc.)
response = llm_client.complete(prompt)
return jsonify({"text": response})
if __name__ == "__main__":
app.run(port=8080)
Performance Characteristics
Context Size:
Hot: ~5KB (always loaded)
Tree: ~2KB (always loaded)
Retrieved: ~1-3KB per query
Total: ~8-15KB (constant, regardless of agent age)
Retrieval Speed:
Keyword: 10-20ms
LLM tree search: 300-600ms
Cold query: 50-100ms
5-Year Scenario:
Hot: Still 5KB (living document)
Warm: Last 30 days (~50KB)
Cold: ~50MB in Turso (compressed distilled facts)
Tree: Still 2KB (different nodes, same size)
Context per session: Same as day 1
Comparison with Alternatives
System
Memory Model
Scaling
Accuracy
Cost
Flat MEMORY.md
Linear text
❌ Months
⚠️ Degrades
❌ Linear
Vector RAG
Embeddings
✅ Years
⚠️ Similarity≠relevance
⚠️ Moderate
EvoClaw Tiered
Tree + tiers
✅ Decades
✅ Reasoning-based
✅ Fixed
Why tree > vectors:
Accuracy: 98%+ vs. 70-80% (PageIndex benchmark)
Explainable: "Projects → EvoClaw → BSC" vs. "cosine 0.73"
Multi-hop: Natural vs. poor
False positives: Low vs. high
Troubleshooting
Tree size exceeding limit
# Prune dead nodes
memory_cli.py tree --prune
# Check which nodes are largest
memory_cli.py tree --show | grep "Memories:"# Manually remove unused categories
memory_cli.py tree --remove "unused/category"
Warm memory filling up
# Run consolidation
memory_cli.py consolidate --mode daily --db-url "$TURSO_URL" --auth-token "$TURSO_TOKEN"# Check stats
memory_cli.py metrics
# Lower eviction threshold (keeps less in warm)# Edit config.json: "eviction_threshold": 0.4
Hot memory exceeding 5KB
# Hot auto-prunes, but check structure
memory_cli.py hot
# Remove old projects/tasks manually
memory_cli.py hot --update project '{"name": "OldProject", "status": "Completed"}'# Rebuild to force pruning
memory_cli.py hot --rebuild
LLM search failing
# Fallback to keyword search (automatic)
memory_cli.py retrieve --query "..." --limit 5
# Test LLM endpoint
curl -X POST http://localhost:8080/complete -d '{"prompt": "test"}'# Generate prompt for external testing
tree_search.py --query "..." --tree-file memory/memory-tree.json --mode llm --llm-prompt-file test.txt
Migration from v1.x
Backward compatible: Existing warm-memory.json and memory-tree.json files work as-is.
New files:
config.json (optional, uses defaults)
hot-memory-state.json (auto-created)
metrics.json (auto-created)
Steps:
Update skill: clawhub update tiered-memory
Run consolidation to rebuild hot state: memory_cli.py consolidate
v2.1.0 — A mind that remembers everything is as useless as one that remembers nothing. The art is knowing what to keep. Now with structured metadata to remember HOW, not just WHAT. 🧠🌲🔗
