engram

# Engram Memory System > v3.6.5. Repository-owned source; changes require an explicit maintenance request. > Use the canonical installed scripts; do not make divergent workspace copies. > Changelog: [CHANGELOG.md](CHANGELOG.md) · Script reference: [references/scripts.md](references/scripts.md) · Watchdog: [references/watchdog.md](references/watchdog.md) ## Quick Start ```bash # Initialize complete memory system bun skills/engram/scripts/init.js --agent-id main --qmd-variant auto # With a disabled heartbeat canary (recommended first step) bun skills/engram/scripts/init.js --agent-id main --qmd-variant auto --with-cron # Enable the cron only after watchdog/canary verification bun skills/engram/scripts/init.js --agent-id main --qmd-variant auto --with-cron --enable-cron # Provision an additional workspace without shared hooks/config/restart/cron bun skills/engram/scripts/init.js --workspace /path/to/workspace --agent-id work --workspace-only # Force merge into existing workspace bun skills/engram/scripts/init.js --force # Read-only workspace audit (no fixes) bun skills/engram/scripts/watchdog.js --workspace /path/to/workspace --json ``` Fresh full init installs the complete 11-hook managed set, restarts the gateway, and requires the OLL rule-context and rollback hooks to be eligible/loadable in the runtime read-back. `--workspace-only` intentionally skips every shared runtime mutation. Cron provisioning is disabled by default and requires explicit `--enable-cron` after preflight. Fresh workspaces now start with `nightly.enabled=true`, `adaptation.mode=active`, and a matching fresh-init rollout projection. The single fleet registry and scheduler remain deployment-owned; init does not create a competing scheduler. QMD coordinator bootstrap is explicit: `init.js --qmd-manifest /private/migration.json` uses the canonical deterministic `script -> managed Gateway exec` installer, creates a fresh job disabled, and records its read-back declaration. No model, command-runner fallback, secret copying, backfill or auto-activation. Additional workspaces enroll in the existing coordinator; `--workspace-only` forbids shared scheduler provisioning. [Setup](references/setup.md) · [Scheduler contract](references/qmd-global-maintenance.md). Watchdog can audit a shared declaration with `--qmd-scheduler <path>` and reports Workshop containment errors without trying to repair host-owned jobs. ## Engram QMD CLI CLI запускается на Bun прямо из TypeScript-исходников и использует тот же QMD core, что и будущие внутренние интеграции. Runtime dependencies и build step не нужны. ```bash bun bin/engram --help bun bin/engram --workspace /path/to/workspace qmd resolve bun bin/engram --workspace /path/to/workspace qmd doctor --strict bun bin/engram --workspace /path/to/workspace \ qmd search "query" -c workspace-memory ``` When QMD uses a cloud provider whose credential is kept in the OpenClaw protected secret store, run `query`, `vsearch`, and cloud-backed maintenance through OpenClaw managed Gateway exec. Native shell/terminal execution does not receive protected secret injection by design. Do not copy the credential into `.env`, command arguments, or an explicit command environment. Plain BM25 `search` and credential-free diagnostics may run locally. Правила первой версии: - `resolve`, `capabilities`, `status` и `doctor` ничего не изменяют; - `search`, `query` и `vsearch` требуют хотя бы одну явную `-c` и проверяют коллекции против workspace allowlist; - `--json` пишет в stdout ровно один envelope; verbose details появляются только с `--verbose`; - CLI не публикует generic passthrough, `update` или `embed`; - operator CLI не считается OS security boundary и не принимает пользовательский `--scope`; - agent-facing режим не включён до появления trusted caller context. Внутренний maintenance coordinator доступен как core API и через закрытый operator entrypoint `scripts/qmd-maintenance-coordinator.ts`: он хранит dirty generations вне индексируемого workspace, сериализует один physical index lease и запускает index-wide `update` перед scoped incremental `embed`. Обычный цикл никогда не использует `-f`. Hooks, cron и production topology переключаются отдельным rollout после canary. Workspace heartbeat в `coordinated` mode не обслуживает QMD и делегирует единому scheduler; см. [`references/qmd-global-maintenance.md`](references/qmd-global-maintenance.md). Безопасная установка launcher: ```bash bun scripts/install-cli.js --dry-run bun scripts/install-cli.js engram --version ``` По умолчанию installer использует `$BUN_INSTALL/bin` или `~/.bun/bin`. Он идемпотентен и удаляет только launcher с точным managed body, созданным из этого checkout. Полный контракт: [references/qmd-cli.md](references/qmd-cli.md). Rollout и rollback: [references/qmd-cli-rollout.md](references/qmd-cli-rollout.md). ## Architecture Overview ``` workspace/ ├── life/ # Knowledge Graph (entities, facts, summaries) │ ├── people/{name}/ │ │ ├── items.json # Facts array (v2 schema) │ │ └── summary.md # Quick-context (Hot/Warm/Cold tiers) │ └── projects/{slug}/ ├── memory/ # Operational memory │ ├── agent-{id}/ # Per-agent session silo (e.g. agent-main) │ │ └── {session}/YYYY-MM-DD.md # Daily notes (main, telegram-…) │ ├── domains/ # Subagent domains │ │ ├── registry.json │ │ └── {slug}/ │ ├── heartbeat-state.json # Phase tracker │ └── memory-state/ # Dedup hashes ├── ops/ # OLL: observations + tensions └── engram.json # Workspace config (models, QMD) ``` Three-layer storage: **Daily Notes** (ephemeral, rotated) → **Knowledge Graph** (durable facts, tiered retrieval) → **QMD** (hybrid search: BM25 + embeddings + rerank). For full architecture: [references/architecture.md](references/architecture.md) ## Memory Rules ### Session Isolation **Golden Rule: Memory is isolated by session.** | Session Type | Memory Scope | KG Access | Daily Notes | |---|---|---|---| | Main (direct chat) | Full KG + all collections | ✅ Write | ✅ | | Group (topic) | Domain context only | ❌ | ✅ (own session) | | Subagent (cleanup:delete) | Domain only | ❌ | ❌ (uses changelog) | - **NEVER** cross-reference one project's memory from another project's session. - Group chats **CANNOT** see `MEMORY.md` or `life/` — only main session has full KG access. - **Always** specify `-c <collection>` in QMD queries — never query without a collection flag. ### Every Session Startup (automated by hooks) - `engram-session-start` → creates sessionDir + daily note, writes `<!-- session:start -->` - `engram-bootstrap-qmd` → declares scheduler ownership; performs no QMD maintenance - `engram-daily-note` → reconciles existing note state on `gateway:startup`; creates nothing - For topic-thread: `engram-topic-domain-load` injects Domain Context + AGENTS via system-event ### Writing Memory **NEVER write `items.json` directly.** In a KG v3-enabled workspace, use only `engram_memory_save` / `engram_memory_retract` inside the trusted source turn. The typed pipeline validates source-turn ownership, registry admission, replacement, projection and receipt before commit. Fleet rollout is complete. Legacy v2 mutation entrypoints were physically removed; `items.json` is an immutable historical archive. If typed ingress is unavailable or rejects an unregistered assertion, use daily/domain storage and extend the registry separately. Never recreate or bypass the typed writer. ### Daily Notes (Three-Layer Rotation) - Daily notes capture session activity in sections: `events`, `decisions`, `learnings`, `threads`, `next` - **Three-Layer Rotation**: when daily note >1000 lines → Archive (QMD-indexed) → Stub (10-20 lines) → QMD reindex - Write via `daily-note-append.js` — never edit daily notes directly - `--workspace`, when supplied, must be an absolute workspace path. - `--session` must be a stable canonical contour (`main`, `telegram-direct-…`, `telegram-group-…`, or a full canonical agent key). Never pass a runtime/turn UUID; the writer rejects bare UUID session segments. - Record as you go, not at session end ```bash bun skills/engram/scripts/daily-note-append.js \ --session main --section events --text "Fixed 44 semantic duplicates in KG" ``` For an explicitly high-signal event (a decision, material fix, or delivered artifact), optionally add a concise retrieval card alongside the canonical daily-note entry: ```bash bun skills/engram/scripts/daily-note-append.js \ --session main --section events --text "Fixed stale heartbeat lock" \ --retrieval-id heartbeat-lock-repair \ --retrieval-title "Heartbeat lock repair" ``` The card is an opt-in QMD retrieval aid, not a second source of truth: it links to the daily note and is indexed by the same memory collection. Do not generate cards automatically, backfill ordinary notes, or add a cron for them. ### Knowledge Graph - **Tiered retrieval**: `summary.md` first (quick context), `items.json` on demand (full facts) - **Entity creation**: when 2+ facts reference same entity, create KG entity - **No-Deletion Rule**: facts are NEVER deleted. Set `status: "superseded"` and link via `supersededBy`. - Fact Schema: [references/fact-schema.md](references/fact-schema.md) - Topic/project sessions remain domain-only. Automatic heartbeat promotion to `life/` is retired and cannot be restored with configuration. ### Memory Decay | Tier | Recency | In summary? | |------|---------|-------------| | Hot | ≤7 days | ✅ Prominent | | Warm | 8-30 days | ✅ Lower priority | | Cold | 30+ days | ❌ (searchable via QMD) | Full decay rules: [references/decay-rules.md](references/decay-rules.md) KG v3 records actual use through `engram_memory_access` once per source turn, with exact assertion UUIDs that materially influenced the answer. The tool writes append-only events; the daily coordinator applies them idempotently to a separate access-state overlay and rebuilds `life/v3/current-summary.md`. Canonical assertion bodies and the historical v2 archive remain immutable. ### QMD Search QMD = hybrid search engine (BM25 + embeddings + rerank). Two providers: local (Vulkan), jina (cloud). **Query triggers:** 🔴 **Always search QMD when:** - User asks "what do you know about…" / "найди…" / "remember when…" - User references a person/project/entity by name - User asks to recall, find, or check memory 🟡 **Consider searching QMD when:** - User mentions a topic that might have related facts - You need context for a decision - Current conversation might benefit from historical context **Commands:** ```bash qmd search "query" -c <collection> # BM25, <1s qmd query "query" -c <collection> # Hybrid (embeddings + rerank), ~3s qmd query "query" -c <coll1> -c <coll2> # Multi-collection ``` For QMD installation: [references/qmd-setup.md](references/qmd-setup.md) ## Real-Time Extraction > **KG v3 cutover note:** when the runtime exposes `engram_memory_save` and > the workspace has an enabled `memory-state/kg-v3/live-ingress.json`, that > typed tool is the sole inline KG writer. Use it at most once for an explicit > durable user assertion. Do not run `memory-signal.js` or mechanical extraction > for the same KG intent. The legacy writer is physically absent and must not be > recreated. Operational/progress/test/status > material still goes to daily/domain stores. If the tool is absent or rejects > an unregistered entity/predicate, route the intent to daily/domain memory. Automatic session/daily/domain/OLL → KG promotion is retired. High-signal facts enter canonical memory only through the typed tool inside the source turn. ``` Message → explicit durable intent? ├── YES + registered typed assertion → engram_memory_save/retract → KG v3 ├── YES but unregistered → daily/domain store; extend registry separately └── operational/progress/test/status/casual → daily/domain store or skip ``` ### Signal Detection (diagnostic only) ```bash bun skills/engram/scripts/memory-signal.js --text "Я предпочитаю TypeScript" # → { "signal": "high", "categories": ["preference"], "confidence": 0.88 } ``` Supports **Russian and English**. Six categories: `correction`, `preference`, `decision`, `identity`, `instruction`, `milestone`. This classifier does not authorize or perform a KG write. ### Contradiction Detection ```bash # Intra-entity (fast, no QMD) bun skills/engram/scripts/memory-contradict.js --fact "Uses Node.js" --entity "people/alice" # Cross-entity (via QMD BM25) bun skills/engram/scripts/memory-contradict.js --fact "Uses Node.js" --entity "people/alice" \ --cross-entity --collections "life,openclaw-memory-agent-main-main" ``` ### Rules for Inline Extraction - Only an authorized primary/meta session receives the typed KG capability. - One source turn may perform at most one typed KG mutation. - Missing entity/predicate admission is not a reason to use a legacy writer. - Mechanical extraction only advances watermarks and reports suppressed counts. - Duplicate/replacement behavior belongs to the v3 writer and operation journal. ## Session Recording Workspace AGENTS.md owns operational capture policy. For an active observer-owned binding, Memory Worker saves Events/Decisions: do not duplicate those sections, wait for the worker, or read this skill merely to finish a reply. This also applies to compaction and memory-flush. Learnings and handover remain selective foreground operations. For sessions without an active observer binding, use the foreground recording rules below with an explicit absolute workspace and trusted canonical session key. The following two-level recording fallback applies only to foreground-owned sections: ``` Level 1: Agent inline (primary) — best quality, during session ↓ forgot? Level 2: Compaction memoryFlush — safety net before context compression ``` ### When to record - **Topic completed** — a task/discussion block finishes (every 5-10 messages) - **Decision made** — any explicit decision → `--section decisions` - **Topic shift** — conversation moves to a new subject - **Significant result** — something was built, fixed, or discovered ```bash bun skills/engram/scripts/daily-note-append.js \ --session main --section events --text "Fixed 44 semantic duplicates in KG" ``` ### Rules - Keep entries brief (1-2 lines each) - Record facts, not feelings ("Fixed X" not "Had a great session") - Operational events → `events`, explicit choices → `decisions`, insights → `learnings` - Do NOT wait for session end — record as you go ## Heartbeat Для legacy workspace heartbeat остаётся 10-фазным оркестратором. После PR 2 cutover чистая установка оставляет в heartbeat только не-OLL обслуживание; weekly synthesis и OLL ownership переходят единому nightly coordinator. | Phase | Kind | Назначение | |-------|------|-----------| | 0 | inline | Fast Init: state, lock, check | | 0.5 | inline | Rotation: daily notes >1000 lines | | 1 | inline | Cursor maintenance only; no content classification or KG writes | | 1.5 | inline | Stub Summary | | 2 | legacy-only subagent | Synthesis: hb-synthesis (Mon only) | | 3 | subagent | Domains Status: hb-domains | | 3.5 | subagent | Domains Write: hb-domains-write (primary durable path for topics) | | 4 | inline | Maintenance: validate, qmd update/embed | | 5 | legacy-only inline | OLL Check: rethink triggers | | 5.5 | legacy-only inline | OLL Spawn Queue | | 6 | inline | Report + Unlock | For full heartbeat flow, cron provisioning, and subagent model resolution: [references/heartbeat-flow.md](references/heartbeat-flow.md) · [references/HEARTBEAT.md](references/HEARTBEAT.md) ## Operational Learning Loop (OLL) System observes its own friction and accumulates observations for review. **Write observations:** ```bash bun skills/engram/scripts/memory-observe.js --observation "KG extraction missed facts" --category friction ``` **Categories:** `friction` (weight ×3), `surprise` (weight ×2), `pattern` (weight ×1) **Tensions** are auto-created when contradiction check finds Jaccard ≥0.5 + ≥3 common keywords, or manually: ```bash bun skills/engram/scripts/memory-tension.js --tension "..." --fact1 <id> --fact2 <id>