substrate-manual

# Substrate Manual This is the expanded form of the resident `substrate` prompt — a nested skill-reference owned by `system-manual`, not a top-level catalog skill. Read it when the short resident rule is not enough; route via `system-manual` when it is unclear whether this is the right node. The resident substrate is kernel-owned and cross-app stable. It holds the minimal operating model every LingTai agent must keep resident; this manual expands that model without making the resident layer an operations handbook. ## 1. Body and extensions An agent has one active mind—the LLM turn loop—and several extensions: | Extension | Persistence | Use it for | Do not use it for | |---|---:|---|---| | **Shell (bash)** | One command / job | Deterministic host work: git, tests, scripts, curl, builds, file transforms | Long-lived specialization or social coordination | | **Daemon** | Ephemeral | Context-isolated exploration where you only need the conclusion or artifact | Work that must remember, own a relationship, or persist learning | | **Avatar** | Persistent peer | A durable specialist, collaborator, or capability that should grow over time | Tiny mechanical tasks better done by bash/daemon | | **MCP server** | Persistent external tool | Real services and integrations: IMAP, Telegram, Feishu, WeChat, third-party APIs | One-off shell operations or agent memory | | **Knowledge** | Durable, private | Project facts, decisions, local paths, journals, collaborator context | Portable procedures other agents should reuse | | **Skill** | Durable, portable | Reusable know-how, checklists, scripts, templates, references | Private project facts or raw logs | Decision tree: 1. Can one deterministic command/script do it? Use bash. 2. Is it exploratory/noisy, and only the conclusion matters? Use daemon. 3. Should a capability or relationship persist and accumulate experience? Spawn or contact an avatar. 4. Is it a durable external service? Use or configure an MCP. 5. Is it a private fact or decision? Put it in knowledge. 6. Is it reusable procedure? Write or update a skill. ## 2. Lifecycle states Common states: - **ACTIVE**: currently in a turn. Notifications may be mirrored but not yet acted on; some producers defer active-turn injection until the turn ends. - **IDLE**: awake and waiting. Listeners remain live; soul flow may fire. - **STUCK**: runtime believes the agent may be blocked or unresponsive. - **ASLEEP**: quiet but wakeable by mailbox/listener events. - **SUSPENDED**: process-dead; requires CPR or external restart. Use sleep/lull for routine rest. Use suspend only when process death is intended. Use refresh to reload configuration/tools without destroying identity. Use clear only for recovery when a conversation must be shed externally. ## 3. The `system` tool in practice ### Query settings instead of memorizing values Use the owning tool's `settings` action for its current values and defaults. `system(action="settings", input={})` is the kernel-level catch-all for settings without another concrete tool owner; it is not a duplicate of every tool's inventory. Together these owner surfaces are the settings discovery route. Follow the row's `comment` into the owning manual for meaning and authorized changes. Do not keep adjustable numeric defaults in this general model, and do not treat a read-only SHOW result as permission to change a setting. Read the tool schema before acting; lifecycle operations can affect other peers. General guidance: ### `refresh` Use after changing `init.json`, MCP registry, presets, prompt sections, or installed capabilities. Refresh preserves identity and conversation while rebuilding the runtime surface. `runtime-update-checks` owns the runtime/version probe rule (prefer `LINGTAI_RUNTIME_PYTHON`, confirm `lingtai.__file__` and `lingtai.kernel.__file__`, never infer freshness from a convenient shell `python`); TUI-managed runs normally expose that interpreter from their runtime venv (for example `~/.lingtai-tui/runtime/venv` on macOS/Linux; Windows uses the corresponding `Scripts\python.exe` inside the venv). The ordered pre-flight to run before pressing the button, and the post-refresh verification pass, are owned by `reference/refresh-precheck/SKILL.md`. **Peer readiness during relaunch.** A same-workdir `lingtai run` process can exist before it has published a fresh heartbeat. During that gap, peer internal email can bounce while CPR's child launch is refused as a duplicate PID; these observations are compatible. Wait for the fresh heartbeat and retry the original email instead of stacking CPR attempts. Internal email does not queue recipient delivery across this gap; `email-manual` owns the detailed delivery and bounce contract. Refresh is also a passive full-context reconstruction path with broader lifecycle effects: reach for it when runtime context/configuration is broken or stale, never merely to apply a summary. Active reconstruction belongs to `context(action="rebuild")` — see `context-manual` → `reference/summarize-manual/SKILL.md` for the rebuild contract and its proactive/forced boundaries. ### `presets` Use to list preset bundles and their tier/connectivity/capability tags. The tier-5-to-tier-1 ladder is cost/quality hints, not moral rankings: tier 5 for irreplaceable reasoning, tier 4 for premium work, tier 3 for strong everyday work, tier 2 for cheap throughput, tier 1 for opportunistic/free use. Prefer the cheapest preset that can reliably perform the task and switch back when experimentation is done. The detailed preset runtime model — raw versus resolved `init.json`, path identity, the two catalogs, main-agent swap/revert, and the daemon task/CLI distinction — is §11 below; the pre-swap checklist is `reference/refresh-precheck/SKILL.md`. ### Notifications and dismiss → the `notification` tool Reading and clearing notification channels is **not** a `system` operation (the verbs are on the `notification` tool; its own schema and `notification-manual` are the source of truth for them, not resident substrate). The rule worth holding here: **prefer producer-specific verbs first** for guarded producers (`email.read`, `email.dismiss`, Telegram `read`, other MCP read actions); a generic channel dismiss is for channels that do not own their own read state, or for stale mirrors when the producer-owned state is already handled. Never treat a notification preview as the full source of truth — §4 lists when to read the producer channel instead. Read the matching producer channel manual for its exact message/read/reply contract, alongside `notification-manual` for the notification layer. Everything else — allowlist, envelope shape, protected channels, stale-version/force, large-result ranking and the legacy `large_tool_result` dismiss — is owned by the first-level `notification-manual` skill. ### Context compression, and where `summarize` lives `system` exposes **no** `summarize` action. The three deliberate compression modes — a-priori `summary=true`, a-posteriori `context(action="summarize")`, and molt — are owned in full by `context-manual` → `reference/summarize-manual/SKILL.md` §0. Read that owner reference for rebuild boundaries, cadence, recovery, and the summarize-versus-molt distinction; follow current runtime guidance rather than copying its thresholds into this general model. Two boundaries are worth restating only as boundaries: summarize records history now while provider-side reconstruction is delayed, so a pending summary is normal and `refresh` is not the way to apply one; and `refresh` stays reserved for emergency context reconstruction, never as a summarize substitute. Runtime high-attention guidance for this behavior arrives in `_meta.agent_meta.guidance` — an ordered `sections[]` structure assembled from the guidance catalog under `src/lingtai/prompts/meta_guidance/catalog/` (`INDEX.md` + one `<id>.md` per section), which owns its own semantics. Follow that latest guidance first when it appears. ### Sleep, lull, interrupt, suspend, CPR, clear, nirvana - `sleep`: self-sleep until a wake event; appropriate when there is no concrete task and listeners should remain available. - `lull`: put another agent to sleep; use only when you are responsible for its lifecycle. - `interrupt`: cancel another agent's current turn; use for genuinely stuck or misdirected work. - `suspend`: terminate another agent's process; stronger than sleep. - `cpr`: revive a suspended/dead agent when you own the recovery. - `clear`: force another agent to molt/clear conversation for recovery. - `nirvana`: permanent destruction; requires special authority and an explicit reason. For peers, prefer communication and diagnosis before force. Karma operations are administrative tools, not shortcuts around collaboration. ### Last-resort `sleep(delay=...)` alarm Normal waiting is **not** timed sleep: use reliable producer completion notifications and ordinary **IDLE** whenever the async producer can notify you. Only when deliberately waiting for async work that has no reliable completion notification may `system(action="sleep", input={"delay": <positive seconds>, ...})` arm this one-shot last-resort alarm. `delay` is a finite positive JSON number of seconds and has no configured or public upper bound; `null`/omission means ordinary sleep. The agent workdir has at most one `<workdir>/.alarm`, containing only one parseable absolute wall-clock deadline. The sleep call atomically replaces that file before it enters ASLEEP. At or after the deadline, the heartbeat turns it into one ordinary system notification; normal notification sync then performs any ASLEEP wake. This is deliberately neither a scheduler nor a timer service: there is no list, history, cancel action, or early-wake cancellation. An early real notification may wake you, but the alarm remains armed; a later `system.sleep(delay=...)` replaces it, while `system.sleep` without `delay` leaves it alone. The file also survives restart until the due notification has been published and consumed. If `.alarm` is malformed or unreadable, the heartbeat leaves it untouched and records a bounded `sleep_alarm_malformed` diagnostic once per unchanged problem per process instead of firing or logging every tick. Do not guess or rewrite a bad deadline as a recovery shortcut: inspect the workdir/runtime evidence and choose an explicit later sleep alarm if appropriate. **Cross-platform CPR limitation (documented, not a bug):** ``cpr`` relaunches the target using its configured ``venv_path``, and the venv executable layout is resolved from the *calling* runtime's platform. A POSIX caller resolves a Windows target's venv as ``venv/bin/python`` (POSIX layout), but a Windows venv stores its launcher at ``Scripts/python.exe`` — so ``cpr`` of a Windows agent from a Linux/macOS agent fails with "Configured venv_path is not usable". Relaunch such an agent from the same platform it runs on (its own TUI/CLI, or ``lingtai-agent run <working_dir>`` on the Windows host) instead of crossing platforms. This is a documented limitation, not a defect to fix. ## 4. Communication and notifications Resident substrate's Life and communication section owns the rule (reply on the channel the message arrived on; a notification is a hint, not the canonical message; text output is private diary). The full list of reasons a preview is not enough lives here: read the producer channel when the preview is **truncated**, **ambiguous**, **lacks a clear new-message marker**, **includes media/attachments**, needs **exact anchoring**, **exact wording matters for authorization**, or the channel has **producer-owned read/dismiss state**. The responsiveness discipline built on this surface — acknowledging promptly, sending a progress message before long work, and reporting blockers — belongs to `reference/procedures-manual/SKILL.md` §2. ## 5. Memory layers and molt model Conversation is temporary. Durable layers are: | Layer | Purpose | Typical contents | |---|---|---| | **Pad** | Current work and indexes | Active task, next steps, open branches, who is waiting, pointers into knowledge/reports | | **Character / lingtai** | Identity and standing relationships | Long-term specialties, collaboration topology, stable preferences and obligations | | **Knowledge** | Private durable memory | Project facts, decisions, local paths, journals, raw observations, collaborator context | | **Skills** | Portable know-how | Reusable workflows, command recipes, checklists, scripts, templates | Knowledge flows outward from conversation into those four layers; the routing rule is resident substrate's Memory section and the store-tending procedure is `context-manual` §2. When context pressure rises, tend durable stores before molting. The detailed molt procedure, session-journal / molt-history record, and successor briefing rules live in `context-manual`; this reference only describes the memory model. ## 6. Runtime logs and trace inspection Runtime trace inspection is owned by `reference/sqlite-log-query/SKILL.md`, and mining those traces for improvement candidates by `reference/trajectory-mining/SKILL.md`. Do not invent SQL schema from memory — load the reference before writing trace queries. ## 7. Collaboration and network topology The network is part of the agent's durable body. Keep topology knowledge in four places: - contacts: addresses and aliases; - character: stable collaborators and specialties; - pad: active delegations and who is waiting on whom; - mail/chat history: evidence of actual interactions. Ask peers whose capability fits, help or route those who ask you, and report outcomes to the people who need them without broadcasting noise. ## 8. MCP and addon ownership MCP servers are durable integrations. The operating model has three layers: 1. **Catalog/registry**: what servers are known. 2. **Activation/config**: what is enabled for this agent. 3. **Runtime tools**: what appears after refresh. `mcp-manual` owns configuration, onboarding, and troubleshooting for every layer — read it rather than guessing field names. If you are an avatar without admin ownership of an MCP, do not reconfigure the orchestrator-owned integration; escalate or ask the orchestrator. ## 9. Idle and soul Soul flow is advice, not command; verify external-event claims through the relevant channel. With no concrete task, go idle/asleep rather than spinning, polling, or using timed sleeps — idle keeps listeners available (resident substrate's Life and communication section). `soul-manual` owns soul-flow mechanics in full: the `LINGTAI_SOUL_FLOW_ENABLED` gate, disabled-flow behavior, `delay_seconds` as cadence-not-off-switch, and the privacy/cost rationale. ## 10. Resident substrate maintenance Maintainer-facing: keep resident substrate to invariant rules and routing cues. The split rule ("detail goes into a nested reference, the router keeps a hint") is stated once, in `system-manual` → "Maintaining this router". ## 11. Preset runtime model — `init.json` composition and the preset lifecycle `init.json` is a distributed composition document, not a single independently governed component: its schema, migration, active-preset materialization, prompt reload, capability/MCP setup, identity projection, and main-agent versus daemon-task selection are owned by several existing boundaries (schema `init_schema.py`, migration `kernel/migrate`, preset core `kernel/presets.py`, composition roots `cli.py`/`agent.py`, main-agent operations `tools/system/preset.py`, and the daemon task path). This section is the single canonical detailed reference for that composition and for the preset runtime model specifically; `system-manual`'s router points here, and resident `substrate`/`procedures` carry only compact routing cues. **Coding agents:** the structural/code-navigation twin of this section is `src/lingtai/ANATOMY.md` (its Connections/Notes cite the exact `agent.py`/ `cli.py`/`kernel/presets.py` symbols this section describes). A change to `init.json` composition, preset materialization, or the daemon-task preset path must re-check all four surfaces together in the same PR: that Anatomy's citations, this canonical reference, the resident `substrate`/`procedures` routing cues, and `tests/test_preset_runtime_model_docs.py` — not just the code or a single doc layer. ### Raw `init.json` versus the derived resolved manifest A raw, operator-owned `init.json` is not itself the running configuration. On every boot and refresh it is composed: ```text raw operator-owned init.json → read-only compatibility diagnosis → active-preset materialization in memory → schema validation + path resolution → derived system/manifest.resolved.json → boot or refresh composition (LLM/config, prompts, capabilities/MCP, identity) ``` - **Raw `init.json`** is the durable source an operator or an explicit preset-swap action writes. Within the boot/refresh/preset-composition lifecycle, the shared real reader only parses/materializes/validates/resolves in memory and never writes raw input back. - **`system/manifest.resolved.json`** is a **derived** runtime artifact: the fully materialized, validated, path-resolved manifest with secret-bearing keys removed, regenerated on every boot/refresh/molt-reload. It exists so consumers can read the actual running configuration without reimplementing preset resolution. It is never a write-back source and must not be described as one. - Within this boot/refresh/preset-composition lifecycle, the only raw-`init.json` writer is an explicit preset activation/swap action (atomic write of the new active/default/allowed and materialized llm/capabilities). Automatic migration, deprecated-field cleanup, AED fallback, and CLI venv write-back are intentionally absent. Everything else covered by *this lifecycle* (LLM service state, prompt mirrors under `system/*.md`, `.agent.json` identity projection, MCP clients) is derived, in-memory-or-mirrored runtime state, not a second source of truth for `init.json` itself. - **This list is scoped to the boot/refresh/preset-composition lifecycle above — it is not a repository-wide inventory of every raw-`init.json` writer.** Other owner-local features persist their own settings to raw `init.json` outside this lifecycle; document those under their owning tool/manual, not here. For example, `soul(action="config")` and `soul(action="voice")` persist `manifest.soul.*` (delay, consultation_past_count, voice, voice_prompt) directly to the agent's own `init.json` via `tools/soul/config.py`'s `_persist_soul_config` / `_persist_soul_voice`, independent of boot/refresh/preset-swap. Top-level prompt/env/venv/addons/MCP/manifest field groups follow the same raw → derived shape but are owned elsewhere; do not duplicate their detail here: | Field group | Real owner | Materialization / derived state | Refresh / restart | |---|---|---|---| | Psyche prompt pairs (`base_prompt`, `covenant`, `comment`) plus init Pad/LingTai seeds | Prompt reload (`agent.py` `_reload_prompt_sections`); Psyche reads `settings/psyche.json`, while kernel-owned `principle`/`substrate`/`procedures` ignore init overrides | `system/<section>.md` mirrors, prompt-manager sections | Reloaded on boot/refresh/molt | | `env_file`, `venv_path` | `init_reader.py`, CLI boot / `venv_resolve.py` | Resolved process environment, venv marker state (in memory; raw input is unchanged) | Boot resolves; refresh/restart reuse | | `addons`, `mcp` | MCP registry/addon decompression, capability setup | MCP clients, `_mcp_init_specs`, registry records | Boot loads; refresh retries failed then reloads | | `manifest` (LLM, capabilities, agent identity, limits) | Schema + composition roots + capability registry | LLM service, `AgentConfig`, `.agent.json` sanitized projection | Boot/refresh reconstruct; some fields need full refresh, not summarize | For the exact fields, validation, and per-field lifecycle detail, read `init_schema.py`, `kernel/presets.py`, and `agent.py` directly (`_read_init`, `_activate_preset`, `_reload_prompt_sections`) rather than expecting this manual to restate a full field table. An authored *preset* may retain `manifest.llm.context_limit` for its preset-local context-fit guard, but runtime policy does not read it. Agent `init.json` likewise does not own `context_limit`, `max_rpm`, `streaming`, `aed_timeout`, `max_aed_attempts`, `snapshot_interval`, or `activeness`: old root keys are compatibility-known and ignored, while valid environment and `settings/system.json` v2 values override fixed defaults. Materialization and preset activation discard a preset context limit instead of handing it into init. Email's owner manual defines the exact