# andenken
> *An-denken* — to think toward what has been. Recollective thinking.
> Heidegger's term for the kind of memory that doesn't merely retrieve,
> but lets the past gain meaning in the present.
Semantic memory for humans and AI agents. Not a corporate RAG pipeline — an
embedding lens that lets the meaning around one existence's canonical time axis
return to the present.
> **Agent-in-charge doc** — [§andenken: 존재의 뜻새김, 시맨틱 메모리를 넘어서](https://notes.junghanacs.com/botlog/20260319T110800.html)
> (Denote `20260319T110800`, Korean). The public standing report of the agent
> who owns this repo: what andenken is for, the two turns it must make, where
> its boundaries are, and why it is named after Heidegger's *Andenken*.
**Project home:** [English](https://junghanacs.com/projects/andenken/) ·
[한국어](https://junghanacs.com/ko/projects/andenken/). A public introduction beside
Entwurf; the steward's report above keeps dated judgments, while this README
owns implementation and usage.
## Who owns the memory
The agent-memory field mostly answers *how do we store memory intelligently*.
beads, Letta, Hermes are one family: the DB is the authority and the system is
the subject that curates. andenken sits on the opposite vertex — files are the
authority, and a human sets the coordinates. That is not a claim of superiority;
it answers a different question.
Three operating consequences follow, and each one is visible in this repo:
- **No automatic dreaming.** Memory refresh is split into four surfaces.
This repo schedules none; agent-config can run `--local` at Pi session start
on the authority. The first post-recovery run needs the estimate gate below,
while *the moment two machines come to hold the same memory* stays an explicit
human call (`--global`). The owner is not whoever holds every beat; it is
whoever decides the moment of coherence.
- **Memory follows the person, not the machine.** The session corpus is a
device-merged, append-only lifetime folder with its own roster
(`~/repos/gh/session`). If the files that own memory live on exactly one
machine, then that machine is the real owner.
- **The doc is the only thing standing between a request and a command.** With
no automation, a stale skill doc silently stops the memory axis. Owning the
surface means owning its verification.
→ [에이전트 기억층 — 누가 기억의 주인인가](https://notes.junghanacs.com/botlog/20260408T120252.html)
is the shared reference where andenken, agent-config, dictcli, nixos-config and
aionsclubs each answer that question from their own repo (Korean).
## How to Read This
andenken is one layer of a larger memory architecture. Before diving in,
understand where it fits.
### Memory axes in the harness
```
active memory │ Blocking pre-reply recall. Harness side (pi-extensions /
(recall) │ openclaw plugin pattern). Bounded timeout, graceful
│ degrade on failure. Consumes andenken as a backend.
│ ── Not implemented in this repo. ──
embedding │ THIS REPO.
(semantic) │ Vector + BM25 hybrid retrieval over curated corpora.
│ Sessions: Qwen3-Embedding-8B (4096d).
│ MD (public garden export): Qwen3-Embedding-8B (4096d).
│ Org: 4B/2560d — currently disabled (upstream R&D).
│ Own embeddings: sessions and md.
│ OpenClaw: a separate existing-vector harvest.
dream │ Overnight consolidation. Compacts recent memory into
(consolidation)│ distilled units. Separate axis; separate roadmap.
│ ── Not implemented in this repo. ──
```
The canonical time axis itself is not implemented here. The harness-side
[`timeline`](https://github.com/junghan0611/agent-config/tree/main/skills/timeline) collector owns KST
coordinates, event identity, source status, and provenance across timelog,
journal, agenda, notes, and git. andenken supplies the semantic turn around
that factual spine:
- **time → meaning:** retrieve session decisions and durable garden context
around a timeline slice;
- **meaning → time:** find candidate evidence semantically, then let the
timeline confirm when it happened and what surrounded it.
Sidecars (required but not axes):
- **dictcli** — personal vocabulary graph (Korean↔English expand, Kiwi stem)
- **denotecli** — structural graph / dblock navigation over org notes
If you want to know about the whole memory stack of the harness, andenken is
the right place to ask. The implementation here is one axis; the documentation
covers all three.
### Tracks inside andenken
| Track | Quality bar | Scope |
|-------|------------|-------|
| **sessions** | Recover decisions and continuity inside canonical time windows | Core. pi + Claude Code JSONL, indexed from the device-merged corpus (`~/repos/gh/session`) with stored timestamp/project/role/source/file signals, plus closed pi-durable snapshots admitted into that corpus by an explicit act (source `pi-durable`). OpenClaw parity is a technical baseline. |
| **md (public garden)** | Recover durable interpretation attached to dated notes and events | Current production knowledge axis. Direct Markdown embedding over exported `~/repos/gh/notes/content` (~2,200 md / ~27MB). OpenClaw builtin md memory logic + LanceDB backend. |
| **openclaw (harvest)** | Recover what the bots said and kept, as its own nameable axis | Landed 2026-09-03. OpenClaw already embeds its agents' sessions and memory with `qwen/qwen3-embedding-8b` at 4096d — the same model we use, chosen independently — so the rows arrive carrying both text and vector and the import costs **zero embedding API calls**. Append-only, authority-harvested then explicitly published to the private Oracle replica, and never a search fallback. |
| **org** | Currently disabled | 3,000+ Denote notes. Source track. Doctor/chunker/incremental work is upstream R&D, **not** what agents consume right now. |
**Split of effort.** The agent-in-charge separates *what we ship to agents now*
from *upstream development*:
- **Now:** sessions + md are the two directly embedded tracks. OpenClaw is
also queryable as a separate, explicitly named harvest of existing vectors.
Garden export is controlled, easier to tune, and immediately useful.
- **Upstream:** org track. Org notes are richer but messier; few people actually
retrieve them well today. Doctor WARN cleanup, chunker improvements, and
incremental sync all live here. Disabled in production until it earns its keep.
The previous qmd-over-garden-MD plan was retired on 2026-05-12 (issue #8).
qmd's local-GGUF rerank/expand stack was too heavy for interactive retrieval
(~53s/query on 90-file smoke). The current md track keeps the same garden
source but uses the same simple **embed → LanceDB → hybrid retrieve** contract
as the sessions track.
## What It Does
andenken is not a harness that keeps sessions alive, and it is not a search
benchmark that finds a few concept words in a garden. It is the embedding memory
axis that brings the judgements and inventions made inside a session — long after
that session has flowed away — back around the events on the canonical time axis,
so they become the next move in the present.
> The bridge calls, the session flows away, the time axis keeps the facts, and
> andenken helps those facts gain meaning again.
**The time axis is the skeleton; the embedding is the lens.** `timeline` owns
*when* and *what*; andenken recovers *why*, *which judgement*, and *where it
continues* out of sessions and md. That runs in both directions:
1. **time → meaning** — given a date or window, read the timeline's events and
refs, then find the session judgements and the durable garden interpretation
around them.
2. **meaning → time** — start from today's question, find candidate sessions and
notes, then send the timestamps, files and identifiers they yield back to the
timeline to confirm the real coordinate and the depth around it.
An explicit date is read first, structurally. **An embedding result never becomes
the truth about time.**
```
andenken search-sessions "why did we stop the paid full rebuild"
andenken search-sessions "last week's decision" --date-from ... --date-to ...
andenken search-sessions "corpus 결정" --mode recent --project andenken
andenken search-md "2026-07-11 장염 복통"
andenken status
./run.sh corpus:manifest verify # is the corpus itself intact?
./run.sh doctor --md # production md triage / gap explainability
./run.sh accept # user-facing acceptance (API 0 by default)
```
The north-star question is not whether a generic concept term returns
something. Two days out of the real time axis define it better than any
vocabulary probe:
- **2026-02-07** — commits, stamps, notes and journal are all silent, yet the day
holds 611.6 min of family, 358.6 min of reading, 484.6 min of sleep. Reading
only the artifacts and answering "an empty day" is a failure.
- **2026-07-11** — sleep, reading and family time plus the journal's `장염 복통`
and `인간 환멸`, with zero depth-2/3 artifacts. The answer is not "did
nothing"; it is to distinguish the *depth* of the record.
So the question the system must answer is: **what did I live, what did I make,
why did I do it, and where do I continue now?**
### What andenken does not own
The boundaries are as load-bearing as the features. Each line names another
agent-in-charge who does own it.
| Not ours | Whose | Why the line matters |
|---|---|---|
| KST coordinates, event identity, source status, provenance, natural-language dates | `timeline` (agent-config) | A similarity score must never become the truth about time. andenken does not merge `empty / partial / stale / unreadable` on its own. |
| Calling, waking, resuming a session; transport between garden citizens | `entwurf` / `meta-bridge` | andenken searches what a session left behind; it does not keep sessions alive. |
| Exposing recall to each harness, composing time facts with embedding evidence | `agent-config` | One capability, but its reach differs per surface — a doc that claims three axes from the CLI misleads a sibling inside pi. |
| Korean morphology, ko↔en vocabulary pairs | `dictcli` | Layer 1 does not absorb Layer 3. Particle stripping lives here only because it is BM25 preprocessing. |
| Note structure, backlinks, dblocks | `denotecli` | Layer 2 is navigation, not retrieval. |
| Bot memory quality, chunking, model, retention — and their `openclaw-agent.sqlite` itself | OpenClaw (nixos-config) | The tier-4 track is a **harvest**, not a sync. We import; we do not tune their axis. Their databases are opened `sqlite3 -readonly` and snapshotted; we never write to them and never compact them. |
| `data/openclaw.lance` — the store those vectors land in | **andenken** | The mirror image of the row above: their vectors, our LanceDB schema, ids and index maintenance. Authority-side import writes can fragment it, so its justified compaction is ours (`compact openclaw`). The completed store can be explicitly published to a private query replica; neither copy is OpenClaw's upstream SQLite database. |
| The garden content itself | GLG | andenken reads the notes; it never curates the source. |
And one rule that reads like a feature but is a boundary: an overlay or a bridge
mailbox is **never** embedded as a third session source. That work is already
represented in the pi sessions; indexing it would manufacture duplicate memory.
## Architecture
```text
INDEXING — sources stay outside the index
Device-merged Pi / Claude Code corpus ─→ extract / chunk / embed ─┐
Admitted closed pi-durable snapshots ──→ policy B.1 / checkpoint ─┴→ sessions.lance
Exported public garden Markdown ──────→ chunk / embed ────────────→ md.lance
OpenClaw's existing text + vectors ────→ append-only harvest ──────→ openclaw.lance
QUERYING — choose a named surface and source filters
semantic / hybrid → query embedding + vector search / BM25 as appropriate
→ weighted fusion or RRF → MMR diversity → bounded candidates
recent (sessions) → stored-signal scan + timestamp DESC, no query embedding
READING — choose evidence before opening more context
candidate → source / file / entry reference → selected excerpt or original
```
Sessions and md use Qwen3-Embedding-8B / 4096d; OpenClaw harvest accepts compatible
existing vectors without re-embedding them. Raw Org indexing remains disabled
upstream R&D, not another production input. BM25 includes Korean particle
preprocessing; `dictcli` supplies the separate vocabulary-expansion capability.
Indexing, retrieval and original reading have different admission/cost boundaries;
a search query does not re-index the corpus.
`applyRecencyDecay` still exists in `retriever.ts`, but both live tracks pass
`recencyHalfLifeDays: 0` (`index.ts`, `cli.ts` session path, `md-search.ts`).
The 14-day half-life removed on 2026-09-03 multiplied scores by
`exp(-ln2 x age/14)` and then applied `minScore 0.001`, so an ordinary hit fell
below the floor at roughly 49 days and a strong one at roughly 85. On a memory
axis built to reach back years, that was a hard delete wearing the costume of a
ranking signal. Recency intent belongs to `--mode recent`, which is a primary
sort over stored timestamps, not a multiplier on relevance.
### Three-layer search
```
Question: "When did the embedding cost incident happen, why, and what changed?"
Timeline — canonical KST coordinates, event identity, source status
Layer 1 — andenken Session + md semantic evidence
Layer 2 — denotecli Structural graph / exact note navigation
Layer 3 — dictcli Personal vocabulary bridge
```
The timeline is not Layer 0 of the embedding stack; it is the factual spine the
stack must not overwrite. The harness composes these capabilities.
Layer 1 keeps retrieval quality high on its own. Layer 2 provides navigation.
Layer 3 reflects the human's vocabulary. Each catches what the others miss.
## Multi-Harness
Same core serves pi (extension), Claude Code (skill), OpenCode (skill). Tools:
`session_search`, `knowledge_search`. In the current production surface,
`knowledge_search` points at the md track; CLI `search-knowledge` is kept as a
compatibility alias to `search-md`. See `index.ts` (pi) and `cli.ts` (CLI
harnesses).
The md path is deliberately **one shared core**: since 2026-08-11 the pi
extension's `knowledge_search`, the CLI's `search-md`, and the `golden` gate all
call `searchMdCore()` in `md-search.ts`, so the gate and the agent-facing tool no
longer measure different pipelines. A copy of a production path is not a
shortcut — `golden` once kept its own inline decay constant and silently held the
old value after production changed it.
**A capability is not equally reachable from every surface.** OpenClaw search
currently uses CLI `search-openclaw`; there is no dedicated Pi OpenClaw tool.
The Pi extension exposes `session_search` and `knowledge_search`. A harness may
use the CLI through a skill, but that is not native tool parity. Where surfaces
differ, this repo says so rather than letting a sibling hunt for a missing tool.
### Is it any good? — three layers, and only one of them is automatic
`./run.sh accept` is the user-facing acceptance surface, deliberately separate
from `golden`:
1. **index / operator health** — API 0.
2. **retrieval behaviour** — canonical evidence rank, document-level diversity,
and an honest `stale-index` / `corpus-miss` / `ranking-miss` classification.
`--retrieval` opts into paid query embeddings.
3. **a human verdict** — `usable` / `partial` / `not-improved`. **No automated
run may set this.** A green tally is not user acceptance.
Cases live in `acceptance-cases.json`, so adding one never touches code.
### Full-text leg diagnostics (#16 follow-up)
The source API reports additive `degraded` and `diagnostics.fts` fields on CLI
JSON and pi tool details. A normal zero-match FTS call is `ok`; a missing table
or index is `unavailable`; a failed native query is `error`. If FTS did not run
(for example, `--mode recent`), it is `not-run`, not a health certificate.
Diagnostics carry fixed safe codes, never native messages, queries or row text.
A malformed stored row is a typed error, not an empty search result.
`degraded` covers every FTS leg that ran, including an MD fallback's separately
reported `diagnostics.fallbackFts`. It is independent of the existing
cross-axis `fallback` flag; a later successful FTS call cannot erase an earlier
failure within the same request. Pi also adds a model-visible note when a leg
is degraded. **`degraded: false` does not certify whole-search quality**: it can
also mean FTS never ran, and it says nothing about human usefulness.
Run `./run.sh test:diagnostics` for the fenced synthetic store, consumer-report
and table-shape tests (API 0, no optimize). `./run.sh test:writer` runs the writer
type fence and small synthetic recovery/window tests (API 0, no LanceDB). Maintenance reports distinguish
logical fragments, physical data files, manifest-reported bytes, disk usage
and an optimize call's own statistics; none alone guarantees space reduction.
These follow-up changes are source and temporary-build validated; they do not
update the existing deployed `dist` or any running Pi automatically.
### Explicit maintenance, separate from retrieval
Readable legacy FTS does not guarantee its automatic rewrite will succeed.
An explicitly approved FTS rebuild followed by one optimize mitigated the tested
legacy-reindex failure without vector re-embedding; it is not automatic migration.
Future maintenance needs compatible readers, byte-backup/rollback scope, shared
writer exclusion, capacity checks and before/after row/ID/source/dimension/FTS
invariants. Keep cleanup defaults: on panic or timeout, stop and verify rather
than retrying blindly, removing partial files or forcing `deleteUnverified`.
Logical compaction, physical space recovery and loaded-runtime applicability are
separate claims. Dated measurements and the completed recovery are in
[CHANGELOG Unreleased](./CHANGELOG.md#unreleased) and [#16](https://github.com/junghan0611/andenken/issues/16), not live operating counts in this introduction.
## Stack
- **Embeddings** Sessions + md: Qwen3-Embedding-8B via OpenRouter (4096d). Org (disabled): Qwen3-Embedding-4B (2560d).
- **Vector store** LanceDB (`@lancedb/lancedb` pinned to `0.40.0`; file-based, one file per track)
- **Retrieval** Weighted merge + RRF + MMR. Temporal decay is implemented but off in both live tracks; recency is `--mode recent`.
- **Chunking** Track-specific. Sessions: message-aware (pi-durable rows reuse the same sanitize + split). MD: Markdown-aware. Org (disabled): org 2-tier (heading + direct body).
- **Query expansion** dictcli personal vocabulary graph
- **Runtime** TypeScript on Node 24. `run.sh` runs sources through `tsx`; pi loads the compiled `dist/` (`pnpm run build`). The Nix flake (`nixos-26.05`) provides Node and pnpm; dependencies install with `pnpm install --frozen-lockfile`.
A rebuilt `dist/` or a new native module reaches **new processes only**. A
running pi keeps the code and the LanceDB module it loaded at start; nothing
here reloads a harness. A type check of the main config must use
`tsc -p tsconfig.json --noEmit` if it must not overwrite `dist/`; that config's
ordinary invocation emits compiled files.
## Provider split — directly embedded tracks and harvest
| Track | Model / dim | Endpoint | Status |
|-------|-------------|----------|--------|
| **Sessions** | OpenRouter `qwen/qwen3-embedding-8b` / 4096d | `ANDENKEN_SESSION_*` → `data/sessions.lance` | Live. Device-merged Pi/Claude Code corpus plus explicitly admitted closed pi-durable snapshots. |
| **MD** | OpenRouter `qwen/qwen3-embedding-8b` / 4096d | `ANDENKEN_MD_*` → `data/md.lance` | Live production knowledge axis. `doctor --md` accounts for the manifest↔indexed gap. |
| **OpenClaw** | `qwen/qwen3-embedding-8b` / 4096d, **embedded by OpenClaw** | `sqlite3 -readonly` harvest → `data/openclaw.lance` | Live harvest axis; import reuses vectors without embedding calls. CLI search is explicitly named, never a fallback; query cost is separate from import. |
| **Org** | Qwen3-Embedding-4B / 2560d | `ANDENKEN_ORG_*` → `data/org.lance` | **Disabled.** Upstream R&D only. Do not run `index:org` in production. |
For current counts use `./run.sh status`; dated verification snapshots belong in
CHANGELOG, not the provider contract. Sessions indexed-file read failures or
count changes make status fail explicitly, rather than inventing counts; sessions
sync then stops with a diagnostic before indexing or publication. A disabled org
indexed-read failure is shown as unknown, not as missing-row work. Org init/count
failures outside that narrow handler can still fail status.
LanceDB stores are dimension- and track-separated. Each track's search must use
its own provider env. The 4096d sessions/md providers and the 2560d org provider
are not cross-compatible.
### Session sources — the device-merged corpus
Since `v2026.9.3` the sessions track does **not** index a single machine's live
store. Its input is a device-merged lifetime corpus:
```
~/repos/gh/session/<device>/<the harness's own storage path>
```
The layout is the live path with one device segment prepended, and that is the
contract: `detectSource` still decides by `/.claude/`, `extractProjectName`
still reads the `projects` / `sessions` segment, so both pass unmodified and
**the lance schema did not change**.
| Source | Live path under each device | Notes |
|--------|-----------------------------|-------|
| `pi` | `~/.pi/agent/sessions` | pi-native harness sessions |
| `claude` | `~/.claude/projects` | standalone Claude Code sessions |
| `pi-durable` | `.pi/agent/experimental/durable-sessions/<bucket>/<session>/session.sqlite` | closed pi-durable snapshot (SQLite, schema v1); **admitted, never gathered** — see below |
Two rules govern the corpus: **append-only** (never `rsync --delete`) and **real
copies** (no hardlinks — a shared inode cannot move to another machine).
Why it was needed, measured on 2026-09-02: after filtering, thinkpad held 1,137
files and oracle 1,008, overlapping on 553 — **455 files (0.69GB) existed only on
oracle**, including 64 openclaw workspace sessions with no counterpart on the
laptop. An oracle agent searching its own memory could not find its own work.
Integrity is a checksum manifest, not git. One trial commit of the corpus cost a
**806MB pack and 47m47s of gitleaks**, and append-only data has no readable diff
and no history worth rewriting. `MANIFEST.json` is the SSOT and
`MANIFEST.sha256` is `sha256sum -c` compatible — **verifiable without andenken**.
`DEVICES.json` is a separate roster (`state: active|retired`,
`transport: local|ssh|push`) because using the corpus directory listing as the
roster means trying to reach a retired machine forever.
Surfaces: `corpus:gather` / `corpus:manifest` / `corpus:replicate`. `gather`
runs as Step 0 of `sync:sessions` and `rebuild:sessions` and **refuses to index
when it fails** — an index is never built on a corpus of unknown freshness.
Do **not** index `~/.pi/agent/claude-config-overlay/projects`. It is the
pi-shell-acp Claude overlay and would duplicate work already represented in pi
sessions / entwurf messages. JSONL discovery (indexing) accepts `pi`, `claude`
and `all` (`pi + claude`); search filters add `pi-durable`. Invalid values are
rejected with an explicit error.
`session_search` / `search-sessions` source semantics:
| `source` value | Candidate pool | MD cross-track fallback |
|---------------------|-----------------------|-------------------------|
| omitted / `all` | every row: pi + claude + pi-durable | enabled when results are thin |
| `pi` | pi JSONL rows only — `pi-durable` rows are **not** included | disabled (sessions-only intent) |
| `claude` | claude only | disabled (sessions-only intent) |
| `pi-durable` | admitted pi-durable rows only | disabled (sessions-only intent) |
The source filter is an exact match pushed down into LanceDB at the vector /
FTS / substring query level, so the candidate pool is source-specific. A
specific `pi`, `claude` or `pi-durable` source also suppresses the md knowledge
fallback — if the caller named a session source, they asked for sessions, not
garden notes (`all` is not a specific source; it means no filter). So does any
other stored-signal filter or `--mode recent`.
### pi-durable — closed snapshots only
pi's durable sessions live in a SQLite store, not a JSONL file, and the live
store is never read. A durable session reaches andenken as **one closed
snapshot** made beforehand by an approved procedure outside this repo, and
enters the corpus only through an explicit act. The rules are INVARIANT §7.7.
- **No live gather.** `corpus:gather` admits `.jsonl` only, and a JSONL sync
never collects a durable store. `./run.sh corpus:durable-admit <snapshot>
--device <roster id>` copies one closed snapshot into the corpus (real copy
0600, atomic rename). A replacement must keep the lineage and the old last
entry and must not shrink. The reader refuses a recognizable live store by
name — that refusal is not proof that a store is closed.
- **Own policy, own checkpoint.** `./run.sh index:durable` embeds policy `B.1`:
human input runs and their answers, with no length floor on human turns
(only empty text is excluded). A run still open in the snapshot is **held**
for a later one; a run that also holds a non-human input is reported as
**ambiguous** and not embedded. The checkpoint is `data/durable-manifest.json`,
never `session-manifest.json`. `--dry-run` is API 0; the real run is paid and
authority-only.
- **Reading is not admission.** `./run.sh durable:read` can read entries of a
closed snapshot independently of B.1 admission — by window, range or
conversation, including entries B.1 does not embed (`durable:census` counts
them; `durable:plan` shows what each policy would decide, API 0). The raw
reader and the embedding filter are separate surfaces.
- **Excerpts name their unit.** `withExcerpt` / `--with-excerpt` returns file
lines (`jsonl-line`) for a JSONL hit and neighbouring entries of the same
conversation (`durable-entry`, line numbers are entry ids) for a pi-durable
hit. A SQLite snapshot is not a JSONL file, and a refused read comes back as
a named `unsupported` with the `durable:read` command, never as an empty
excerpt.
- **A full rebuild drops it.** `rebuild:sessions` and `index:sessions --force`
delete the durable checkpoint with the rows; run `index:durable` afterwards.
## md track — public garden direct embedding
The md track starts from the exported public garden, not raw org. Garden export
is a controlled surface (already shaped by Hugo + Denote conventions) and
therefore much easier to tune than the raw Denote tree.
| Surface | Path | Notes |
|---------|------|-------|
| md source | `~/repos/gh/notes/content` | exported Markdown; ~2,200 md / ~27MB |
| md store | `data/md.lance` | LanceDB, dimension 4096d |
| md manifest | `data/md-manifest.json` | mtime/size based incremental |
| provider env | `ANDENKEN_MD_*` | OpenRouter qwen/qwen3-embedding-8b |
| md doctor | `./run.sh doctor --md` | provider / DB / manifest / gap explainability |
The store held `11,078` rows on 2026-10-15. The breakdown below is the
2026-09-03 measurement:
- `10,704` chunks / `2,230` indexed files / `2,245` manifest entries
- the manifest↔indexed gap is accounted for by `doctor --md` through
`analyzeMdFile` skip reasons (`noembed_tag`, `min_body`, `all_chunks_short`,
`deleted`, `unclassified`); the first production cut on 2026-05-12 closed with
`18 = 3 noembed_tag + 15 min_body`, `unclassified=0`
- chunk count is intentionally much lower than org (`44,916`) because the md
chunker emits larger, denser OpenClaw-style CJK-weighted chunks rather than
org's heading/body two-tier fragments.
The implementation ports OpenClaw's builtin md memory logic
(`~/repos/3rd/openclaw/packages/memory-host-sdk/src/host/`) onto the same
LanceDB backend used by the sessions track. This is the exact pattern the org
track was built with originally — OpenClaw core + LanceDB substitute for
sqlite. See [COMPARISON.md](./COMPARISON.md) for the track-by-track matrix.
## Scope and safety policy
- `journal`: only files with identifier `>= 20250101T000000`
- Exclusion tags / skip reasons are surfaced through `analyzeMdFile` and
`doctor --md` (`noembed_tag`, `min_body`, `all_chunks_short`, `deleted`,
`unclassified`)
- Content chunking uses **direct body only** (no parent/child duplicates)
- Policy changes are treated as **full rebuild events**, not incremental syncs
## Rebuild / sync
```bash
cd ~/repos/gh/andenken
./run.sh sync:sessions --local # this device only, zero ssh, no replica touch
./run.sh sync:sessions --global # gather every device → embed → verify → publish
./run.sh rebuild:sessions # sessions full rebuild (estimate + confirm)
./run.sh corpus:gather [--strict] # collect admitted sessions from every device
./run.sh corpus:manifest verify # sha256 integrity over the corpus
./run.sh corpus:durable-admit <snapshot> --device <id> # admit ONE closed pi-durable snapshot
./run.sh index:durable --dry-run # what the next durable run would embed (API 0)
./run.sh index:durable # PAID, authority-only: embed admitted snapshots (B.1)
./run.sh durable:read <session.sqlite> --around <entry> # raw entries of a closed snapshot
./run.sh index:md # md incremental / full (with gate when needed)
./run.sh search:md "<query>" # md search
./run.sh sync:openclaw [--full] # harvest OpenClaw's own index (API 0)
./run.sh sync:openclaw:oracle # publish its private store to the Oracle query replica
./run.sh search:openclaw "<query>" # the harvest axis — by name, never a fallback
./run.sh compact openclaw # defrag our harvest DB (not part of `all`)
./run.sh doctor --md # md production triage / gap explainability
./run.sh accept # user-facing acceptance (API 0 by default)
./run.sh golden # transitional component baseline; see NEXT.md
```
`scripts/sync-sessions.sh` is the operating heartbeat for the sessions track and
what the agent-config `memory-sync` skill calls. Since 2026-09-03 it has two
modes. `--local` (the default) indexes only this device's live sessions, uses no
ssh, and does not touch the replica. **First use after the writer-recovery
release is different:** missing-row recovery can pay a backlog of unknown size.
Hold the authority's agent-config warm path before pulling; pull, run API-0
`./run.sh estimate:sessions`, and accept the whole `to_index` scope and cost
before resuming warm or sync. See the `andenken-embed` skill's first-run gate.
`--global` gathers every device with `--strict`, embeds, verifies, and publishes
the index, the manifest **and** the corpus in one act; that is the moment a human
is meant to hold. The publish runs in a fixed order: verify → corpus
(`replicate-corpus.sh --strict`, which refuses a corpus failing the source-form
check and, in archive mode, also carries the archive — see *Corpus archive
mode* below) → `sessions.lance` with `session-manifest.json` and its paired
files (`durable-manifest.json`, `session-policy.json`, `s0-held.json`; each
removed on the replica when the authority has none). A failure before the index
step leaves the replica's index as it was, and a 0-row index is never
published. Admitted snapshots travel with the corpus. A failed step fails the
whole publish.
`--push` remains as a deprecated alias. This repo schedules nothing; agent-config's
Pi warm caller can run `--local` automatically at session start on the authority.
**One writer at a time, across processes.** Every `sessions.lance` writer —
sync, rebuild, `index:sessions` / `index:durable`, sessions cleanup / compact,
`corpus:durable-admit`, pi `/memory reindex` — first takes
`data/.sessions-write.lock`, without waiting. A held lock exits 75
(`sessions-writer-busy`, or `sessions-writer-stale-lock` when the owner is gone)
and is never reclaimed automatically; dry-runs take no lock. `sync-sessions.sh`
keeps its own older flock (`data/.sync-sessions.lock`) on top of that, so a
`--local` request that finds another sync running coalesces (exit 0) and a
`--global` one waits for it. The guarantee belongs to the code that runs: a pi
process started before the lock existed does not take it. See
[INVARIANT.md](./INVARIANT.md) §7.6.
**Only the index authority writes.** `ANDENKEN_INDEX_AUTHORITY` (default
`thinkpad`) gates the indexer itself, not just the push — see
[INVARIANT.md](./INVARIANT.md) §7.1. The gate sits *after* Step 0, so a refused
call still completes its gather: a replica's sessions travel to the authority as
source files and come back inside the index that is pushed to it.
`ANDENKEN_ALLOW_REPLICA_INDEX=1` is a fork, not a way to catch up.
The current golden fixtures still contain vocabulary-heavy probes; they are
retained only as transitional component checks while the gate is rebuilt around
canonical timeline evidence. The md track follows the same shape (manifest-driven
incremental). The legacy mixed-track `scripts/rebuild-incremental.sh` /
`scripts/rebuild-full.sh` paths are deprecated; each track is run on its own. Org
indexing is currently disabled in production — do not invoke `index:org` outside
upstream R&D.
### Candidate build — unattended runs and resume (issue #18)
`candidate:sessions:build` embeds a reviewed API-0 plan
(`candidate:sessions:plan`) into a **separate** candidate directory under
`~/.local/state/andenken/candidates/`. It never writes production `data/`,
manifests, the corpus or backups, and there is no cutover. It holds the
production `data/.sync-sessions.lock` (`flock -n`, exit 75 when busy) and its own
candidate writer lock, and is pinned to at most 8 CPUs.
```bash
# fresh target (PAID; interactive "yes" required, no --yes)
ANDENKEN_ALLOW_PAID_FULL_REBUILD=1 INDEX_CONCURRENCY=4 ANDENKEN_SESSION_TIMEOUT_MS=120000 \
./run.sh candidate:sessions:build --plan <plan.json> --plan-sha <sha256> --target <candidates>/<name>
# continue a partial target of the same plan (same human gate)
ANDENKEN_ALLOW_PAID_FULL_REBUILD=1 INDEX_CONCURRENCY=4 ANDENKEN_SESSION_TIMEOUT_MS=120000 \
./run.sh candidate:sessions:build --plan <plan.json> --plan-sha <sha256> --target <candidates>/<name> --resume
./run.sh candidate:sessions:measure --target <candidates>/<name> --out <report.json> # API 0
```
What keeps a long run going, and what stops it:
- **Transient provider faults are absorbed in-process.** Timeout, 429, 5xx and
network failures are retried per request with exponential backoff and jitter
(2 s → 300 s, `Retry-After` as a floor), at most 12 attempts and 30 minutes
from the first attempt. One pause is shared by every worker, so a 429 or an
outage slows the whole run instead of letting other workers keep hammering.
- **Everything else stops new requests at once**: other 4xx, wrong dimension,
an S0 hit in outbound text, the candidate 5 GiB cap / 8 GiB free-space reserve
(checked around each write, not a filesystem quota), or a source file that no
longer matches the plan. Files complete at that moment are checkpointed
(manifest written only after their rows are flushed), except after a space,
S0 or dimension stop. The receipt becomes `partial`; nothing is deleted.
- **`--resume` reuses what was paid for.** An API-0 reconciliation reads each
stored file once and requires DB ids and text hashes to equal its extraction.
Files complete in the DB but missing from the manifest are adopted; incomplete
ones are deleted from the candidate only; any manifest↔DB mismatch, unknown
file, duplicate id or changed source is refused. Only the remaining files are
sent. Every attempt's receipt, manifest and ledger is archived byte-for-byte
under `<target>/attempts/`.
- **Adoption ends resume.** Once a candidate has been adopted as the live data
directory, its historical build target is no longer a `--resume` target: the
live address is governed by its adoption receipt, and any further build is a
new candidate. Resume applies only to a target that has not been adopted.
- **Operator stops.** A stale candidate lock (SIGKILL / OOM) is never removed
automatically. A changed plan source voids the plan (re-plan, never resume into
a different input). `--resume` still needs a human `yes`: a run that stops
overnight waits for the morning, with its completed work kept.
- A request that timed out may already have been billed; the receipt's
`api.retriedTexts` is the upper bound on double billing.
The run prints one bounded progress line per minute — files, rows (new vs
reused), requests / attempts / in flight, retries by class, pause, last success
and an ETA range — and the receipt records per-request latency percentiles,
failure classes, backoff, and preflight / human-wait / active / wall time. No
message text, credential values or provider error bodies are emitted in logs
or receipts; selected text remains in the candidate index.
Measured on 2026-11-02 (thinkpad, OpenRouter Qwen3-Embedding-8B 4096d, 8 CPUs,
API concurrency 4, control-v1 plan of 3108 JSONL files + 1 durable snapshot):
| | attempt 1 (fresh) | attempt 2 (`--resume`) |
|---|---|---|
| active time | 29 min 55 s | 45 min 25 s (preflight reconciliation 16 s) |
| outcome | partial: provider-timeout with a 60 s setting; timeouts were not retried then | built: 114,889 rows, unique ids, S0 hits 0, FTS, manifest == DB |
| rows stored | 52,288 (441 computed row vectors were unpersisted after the checkpoint; source originals intact) | 52,288 reused + 62,498 new + 103 durable |
| requests | 1,794 | 2,171 (0 retries) |
| latency per provider call (up to 64 texts) | — | p50 3.3 s, p90 10.0 s, p99 27.8 s, max 63.3 s |
| estimated cost | $0.169 | $0.215 |
The slowest successful provider call took 63 s; the configured timeout in
attempt 2 was 120 s. This does not prove that the same call would have failed
with 60 s: the provider clears its abort timer after response headers arrive,
before body decoding finishes, whereas call latency includes the latter too.
Total embedding time
for about 115k rows was about 75 minutes of active run; the candidate occupies
about 2.05 GB allocated, against 9.28 GB for the production `sessions.lance`
(directory metadata only, which also carries the production version history).
### Selection policy pin — which writers follow it (issue #18)
A data directory says which selection policy its sessions index was built with
in two places that must agree: `data/session-policy.json` and a `selectionPolicy`
field at the top level of `data/session-manifest.json`, both exactly
```json
{"kind":"andenken-sessions-policy","version":1,"selection":"control-v1","s0":"whole-message-v1"}
```
Both absent means legacy — every index written before the pin, unchanged
byte for byte. Both present and identical means control-v1. Anything else — one
side only, the two differ, malformed JSON, an unknown key, version or value, a
pin file or manifest that is a symlink or not a regular file, a manifest that is
not a JSON object — refuses every sessions writer (exit 2,
`REFUSED sessions-policy …`) before a provider, store or corpus is touched. A
lost pin file therefore cannot turn a control-v1 index back into a legacy one.
**The pin is a declaration, not a verification.** It says which policy the
rows *were built with*; pinning reads and re-selects no row. Pin only a data
directory built under control-v1 (a verified candidate). Pinning an existing
legacy index does not upgrade it — it would mislabel legacy rows and unheld S0
matches as control-v1.
```bash
./run.sh policy:sessions # read-only: the pin of data/ as JSON
./run.sh policy:sessions:pin --data <abs dir> # cutover step: pin a dir (API 0)
```
`policy:sessions:pin` writes the manifest field, then the pin file, under that
directory's writer lock, and refuses a directory whose `sessions.lance` is not a
real directory, whose manifest is not a regular JSON-object file, or that
already carries any pin entry (a broken symlink at the pin path counts). It is a
cutover step; nothing runs it automatically.
In a pinned directory the writers read `data/s0-held.json` before the provider
and refuse (exit 2, fixed code, row index and field name only — never a value)
anything but the candidate's exact shape: `{note,count,rows}` with the
candidate's note, rows of exactly `{id,kinds,chars}`, a unique `<file>.jsonl:<line>[#<part>]`
id without `.`/`..` segments, known S0 kinds in canonical order, and a length
the splitter can emit (1–2200). A file's ledger rows move at the same point as
its manifest entry, so a provider failure leaves both as they were. A message
whose retained text fails sanitize (`held-sanitize-conflict`, explicit in the
candidate plan) refuses its whole file: that file's rows and manifest entry stay,
the run fails, and `estimate:sessions` counts it. Conservative limit: the file's
clean messages wait too, until an operator resolves the conflict.
| writer | legacy dir | pinned (control-v1) dir |
|---|---|---|
| `sync:sessions` → `indexer.ts sessions` | unchanged | new and appended files embed the control-v1 send set; whole messages with an S0 match are held, never sent, and their ids / kinds / lengths go to `data/s0-held.json` (saved before the manifest at each checkpoint) |
| `index:sessions --force`, `rebuild:sessions` | unchanged | refused — the wrapper refuses before gather, estimate, preflight or destroy; a pinned index is rebuilt as a candidate, never in place |
| `estimate:sessions` | unchanged | control-v1 send count; S0-held reported apart; `--full` refused |
| `index:durable` | unchanged | a snapshot whose B.1 rows hold any S0 match stops the run before its delete / embed / checkpoint |
| `cli.ts reindex`, pi `/memory reindex` | unchanged, plus the guards below | refused (legacy-only writers); use `sync:sessions` |
| `compact sessions` / `all` | pin and authority read first | same; rows and pin unchanged |
| `cleanup sessions` / `all` | pin and authority read first | refused (`pinned-cleanup-unsupported`): it deletes rows whose source file is absent, which is not decided here for a pinned index; `--dry-run` stays read-only and ungated |
| `estimate.ts sessions` (`./run.sh estimate`) | unchanged | refused — it counts legacy selection; use `estimate:sessions` |
| `sync-sessions-legacy-4b.sh` | retired — refuses (exit 2) | retired |
Independent of the pin: a paid run into an index that holds **none of the
discovered sessions** — missing, empty, or holding only pi-durable rows — is a
full rebuild in all but name, so `indexer.ts sessions`, `cli.ts reindex` and pi
`/memory reindex` refuse it without `ANDENKEN_ALLOW_PAID_FULL_REBUILD=1`;
`sync-sessions.sh` refuses a 0-row index before its 1-call preflight. That
wrapper guard is coarse (total rows): an index holding only pi-durable rows passes
it, costs the one preflight call, and is then refused by the indexer before any
embedding. The fresh-paid guard does not fire while any discovered file remains
indexed: a partial missing-row backlog still needs the first-run estimate gate.
Reprocessing currently embeds the selected file's entire send set, including
unchanged text; production vector reuse and metadata-only migration are not
implemented. Expected-positive files with no rows are rediscovered; partial rows
and legacy zero/missing chunk counts are not repaired by that rule. Delete errors
propagate, including through shared legacy CLI/Pi reindex paths; those two caller
paths were not end-to-end tested by this repair. The index
authority rule (INVARIANT 7.1) now also gates `indexer.ts sessions`, `cli.ts
reindex`, pi `/memory reindex` and `rebuild-sessions-full.sh` (before it gathers
or destroys anything), as it already gated `sync-sessions.sh` and
`index:durable`. `cli.ts reindex` now takes the writer lock, and its `--force`
against a paid endpoint takes the same flag. The shell wrappers also refuse
when `ANDENKEN_DATA` names a different directory from the `data/` they destroy
or publish.
**Corpus source form.** `corpus-manifest.sh` walks the corpus without its
root `.git` (device trees such as `<device>/.pi` and `<device>/.claude` are
the corpus and are walked as before). `form`, `update` and `verify` refuse with
exit 3, writing nothing, when a selected session file is a symlink (a locked
git-annex file or any link) or a git-annex pointer file. `gather-corpus.sh`
runs that check before copying anything, and `replicate-corpus.sh` runs it
before the manifest update and the transfer, ships with `--exclude=/.git/`,
and stops when the manifest update fails. Nothing here gets, unlocks or rewrites
corpus files.
What a source change does not do: a running pi process keeps the `dist/` it
loaded, so `/memory reindex` refuses a pinned dir only after `dist/` is rebuilt
and pi is reloaded.
### Corpus archive mode (issue #18 / agent-config #32)
When the corpus root has a `.git` (git-annex, owned by agent-config), the corpus
is in **archive mode**. The archive itself is not andenken's: andenken calls the
owner's helper (`agent-config/skills/memory-sync/scripts/corpus-archive.py`,
override `ANDENKEN_CORPUS_ARCHIVE_HELPER`) as
`python3 -I HELPER PHASE --corpus ABS --lock-fd 8 [--annex-bin DIR] [args]`,
logs its one-line JSON, and propagates its exit code (`0` ok … `75` lock)
through every wrapper unchanged — `sync:sessions` and `rebuild:sessions` exit
with the failing child's code, not a folded `1`. Exit `19` (a phase failed AND
its index restore failed) is printed as `REFUSED corpus-archive-restore-failed`
and needs an operator; nothing after it runs. A missing helper fails closed
(exit 10) before anything is gathered. `--annex-bin` is passed only when
`ANDENKEN_CORPUS_ANNEX_BIN` is set, and then it must be an absolute existing
directory (else exit 2, `corpus-annex-bin-invalid`, before any helper call);
unset means the helper's normal `PATH`. andenken installs nothing. Without a
`.git` nothing below happens and every path behaves as before.
**Custody.** `<corpus>/.git/corpus-custody.lock`, held with flock on **FD 8** by
the outermost wrapper (order: sync flock 9 → custody 8 → sessions writer lock),
exported as `ANDENKEN_CORPUS_CUSTODY_FD=8`. Children re-enter on the inherited
descriptor (same dev/inode, `flock -n` on the same open file description); none
unlocks or reopens it. A separate open that finds it busy exits 75. A set
`ANDENKEN_CORPUS_CUSTODY_FD` is a claim of inherited custody: if FD 8 is missing
or is another file, the wrapper refuses (75, `corpus-custody-fd-lost`) instead
of opening the lock afresh; only an unset variable means a standalone run that
takes it. What the inherited check proves is narrower than ownership: the
descriptor is the lock file and no other open file description holds it
(`flock` also succeeds on a matching descriptor nobody held yet). That the
outermost wrapper takes it before any child runs is the wrappers' order. Node writers
are launched with `node --import tsx`, because `pnpm exec tsx` does not pass FD
8 through (measured); `corpus-custody.ts` refuses a Node writer without it.
**Commits.** Only two explicit human calls commit the archive, once each:
`sync:sessions --global` and a full `rebuild:sessions` (not `--dry-run`). In
archive mode either of them with `SKIP_GATHER=1` is refused before the lock,
custody or any probe (exit 2, `archive-skip-gather-refused`, API 0): without its
gather there is no checkpoint, and they would publish or rebuild from a ledger
the archive never committed. Their gather runs with `--checkpoint`, and the checkpoint runs
on the index authority (`~/.current-device` only) after the copy, the ledger
refresh and a complete gather. A rebuild refused for its pin or authority is
refused before it gathers, so it commits nothing. A local / warm sync and a
standalone `corpus:gather` hold custody and run the owner's `preflight` (safety
and form readiness) but never commit; the corpus may then be ahead of the
archive HEAD, by design. The first commit (`checkpoint --bootstrap`) is a
one-time operator step outside andenken; no andenken path bootstraps.
**Writers without a gather** — `index:sessions`, `index:durable` (not
`--dry-run`), `corpus:durable-admit`, and `sync:sessions --local` with
`SKIP_GATHER=1` — run the owner's `preflight` and the read-only source
`form` under the held custody before Node starts; either failing exits with its
code and no writer runs. This is readiness, not a HEAD binding: a corpus ahead
of HEAD is still accepted.
**The marker.** Only a `.git` that is entirely absent is legacy. A gitfile, a
symlink (even to a directory) or a `.git` whose lstat fails for another reason
is refused with exit 10 (`corpus-archive-marker-invalid`), API 0, by the shell
wrappers (also under `--dry-run`) and by every Node writer alike. The custody
lock path must be a regular file (a symlink is refused, 75).
`./run.sh test:archive-real` runs this glue against the real owner helper,
pinned by SHA-256 (a different helper is refused before it runs; a private copy
is what runs), on a synthetic temp corpus and replica — never the real ones.
**Replicas.** `replicate-corpus.sh` (non-dry) runs, per replica, each step only
after the one before it succeeded:
1. the replica's `preflight` under the replica's custody, before any byte moves
(one ssh: `exec 8>>LOCK && flock -n 8 || exit 75; python3 -I HELPER preflight
… --lock-fd 8`). It is a safety and form check, not proof that the incoming
transfer fits or is complete;
2. the corpus rsync with the replica's lock held for the transfer
(`--rsync-path "flock -n LOCK rsync"`, root `.git` excluded);
3. the run's one local `bundle`, in a job-owned `mktemp -d` (0700) as
`canonical.bundle` (0600), its sha256 printed;
4. a job-owned 0700 dir on the replica (`mktemp -d ~/.cache/andenken/archive.XXXXXXXX`);
the bundle is copied in, made 0600 and its sha256 must equal (3), else exit 18;
5. the replica's `receive --bundle` under its custody;
6. the location log back: the replica bundles its refs into the same job dir,
that bundle comes back (sha256 checked, else 18) and the canonical side runs
`merge-log --bundle … --from <roster id>` under local custody, so both
hosts' git-annex branches record both copies. Bundles carry Git refs only —
never annexed source content — and nothing is pushed to any other remote;
7. this run's payload files and job dirs are removed (nothing else). A failed
step keeps the replica's job dir and prints its path; the local one is
always removed.
In archive mode `replicate` does **not** refresh the ledger first (legacy mode
still does): the ledger is refreshed by gather and committed by its checkpoint,
and the MANIFEST that ships must be the committed one. `update` always rewrites
it (`generated` moves even when no file changed), so refreshing it here made
every `receive` refuse (13, "worktree MANIFEST.json is not the canonical one") —
measured against the real helper. A standalone `corpus:replicate` after an
uncommitted local gather is therefore refused by the replica's `receive`, fail
closed; `sync:sessions --global` is the path that commits and ships together.
Any non-zero step stops the run with that step's code, so `sync:sessions
--global` ships no index. The roster id must be a valid location-log name
(`^[A-Za-z0-9][A-Za-z0-9_-]{0,63}$`), else exit 2 before the replica is
touched. The ssh calls are not atomic together; `receive` re-verifies what it is
given and fails closed. `ANDENKEN_REPLICA_ARCHIVE_HELPER` and
`ANDENKEN_REPLICA_ANNEX_BIN` (absolute; it must exist on the replica, else the
replica's preflight exits 2 before the rsync) override the replica side.
| entry | archive mode |
|---|---|
| `sync:sessions` (`--local` / `--global`), `rebuild:sessions` | supported: custody, preflight, `node --import tsx` indexer; `--global` and a full rebuild also commit; `--global` replicates the archive |
| `corpus:gather`, `corpus:manifest update`, `corpus:replicate` | supported: custody (taken, or re-entered under a wrapper); no commit |
| `index:sessions`, `index:durable`, `corpus:durable-admit` via `./run.sh` | supported: `run.sh` takes custody, runs preflight + form, then launches one `node --import tsx` |
| `corpus:manifest form` / `verify`, `estimate:sessions`, `index:durable --dry-run`, `status`, `verify` | read-only: no custody |
| direct `pnpm exec tsx indexer.ts sessions` / `durable`, `cli.ts reindex`, pi `/memory reindex`, direct `durable-admit.ts` | refused, exit 75 (`corpus-custody custody-absent` / `custody-fd-lost`), API 0 |
## Why the name
`geworfen` — the human is thrown into the world.
`andenken` — the thrown being thinks back toward what has been.
In Heidegger, *Geworfenheit* and *Andenken* form a pair. 이기상 rendered
*Andenken* as 뜻새김 — "engraving meaning through recollection."
→ The naming record lives in the ARCHIVE of the agent-in-charge doc:
[§andenken: 존재의 뜻새김](https://notes.junghanacs.com/botlog/20260319T110800.html)
(Denote `20260319T110800`, Korean) — including the rejected candidates and how
the Greek *mnemo* turned into the German *Andenken*.
## Further reading
- **[ROADMAP.md](./ROADMAP.md) — core document.** OpenClaw vs andenken comparison summary, change history, maintenance signals, role boundaries. Korean.
- **[COMPARISON.md](./COMPARISON.md) — detailed sessions + md matrix vs OpenClaw.** English.
- **[NEXT.md](./NEXT.md) — the single next thing this agent is doing.** Korean.
- **[§andenken 담당자 문서](https://notes.junghanacs.com/botlog/20260319T110800.html)
— the public standing report of this repo's agent-in-charge.** Denote
`20260319T110800`. Korean.
- [AGENTS.md](./AGENTS.md) — agent-in-charge doc (axes, boundaries, ownership)
- [INVARIANT.md](./INVARIANT.md) — rules that must stay true across changes
- [CHANGELOG.md](./CHANGELOG.md) — CalVer snapshots of what actually closed
- `./run.sh` — living command catalogue (what this repo can actually do)
## Recent milestones
- **2026-10-15** — closed pi-durable snapshots became a session source
(`pi-durable`): explicit admission, policy `B.1`, a separate
`durable-manifest.json` checkpoint, and a raw reader that is not the embedding
filter. One cross-process writer lock now guards `sessions.lance`. The dev
shell moved to Nix `nixos-26.05` / Node 24 and LanceDB to `0.40.0`. Sessions
At that day's earlier maintenance checkpoint, both hosts reported sessions
109,190 rows (103 `pi-durable`), md 11,078 and OpenClaw 8,076. These are
historical checkpoint counts; later increments and recovery are in CHANGELOG.
- **2026-09-03** — sessions sync split into `--local` / `--global`, and the
device corpus, index and manifest now publish as one act. Sessions reached
76,044 chunks / 1,627 files; md 10,704 / 2,230.
- **2026-09-02** — `v2026.9.3`: the sessions input became a **device-merged
lifetime corpus** (`~/repos/gh/session`) governed by a checksum manifest and a
device roster, not git. Two long-standing quality defects closed with it — the
2K embedding truncation (30.3% of user turns were over 2,000 chars; 51.1% of
all user characters sat outside the index) and the 14-day recency decay that
was deleting anything older than a season. `ANDENKEN_INDEX_AUTHORITY` now
gates indexing itself, per INVARIANT §7.1.
- **2026-08-10** — `v2026.8.10`: pi corpus admission moved to the current native
id suffix `_<UUIDv7>.jsonl`. pi had emitted UUIDv7 since 2026-04-15 and only
UUIDv7 from 2026-08-07, so the old garden-id filter had been admitting zero new
pi sessions — no error, just the pi half of the corpus going quiet.
- **2026-05-12** — md first production cut closed. Live knowledge surface
pivoted from org to md; `knowledge_search` now points at the public garden
track, Oracle sync handoff is in place, and `doctor --md` explains the
manifest↔indexed gap (`18 = 3 noembed_tag + 15 min_body`, `unclassified=0`).
- **2026-05-12** — qmd path retired (issue #8). md track redefined as direct
Markdown embedding over the public garden export, ported from OpenClaw
builtin md memory logic onto the sessions LanceDB backend. Org track marked
disabled in production — agents consume sessions + md only.
- **2026-05-11** — Sessions track stabilized and closed: OpenRouter
Qwen3-Embedding-8B / 4096d full rebuild (28,537 chunks, ~$0.065, verify
pass), C2.1a excerpt readback, and C2.1c `session_search.withExcerpt`
opt-in.
- **2026-04-17** — OpenRouter query path + provider split. Indexing stays on
local GPU; queries run from any host.
- **2026-03-30** — Korean particle stripping for BM25 (ported from openclaw).
- **2026-03-21** — 2-step search strategy: abstract → read top-3 → re-search
with concrete terms. Encoded in `promptGuidelines`.
## Related
- [geworfen](https://github.com/junghan0611/geworfen) — existence data dashboard
- [agent-config](https://github.com/junghan0611/agent-config) — harness infra
- [dictcli](https://github.com/junghan0611/dictcli) — vocabulary graph
- [denotecli](https://github.com/junghan0611/denotecli) — Denote KB CLI
## License
MIT