memwal

# Walrus Memory — Portable Agent Memory Walrus Memory enables AI agents to operate reliably across apps and sessions, without losing context. It stores memories on Walrus (decentralized storage), encrypts them with SEAL, enforces ownership onchain via Sui smart contracts, and retrieves them with semantic (vector) search. Memory is portable by design — not tied to a single runtime or provider — and scoped by `owner + namespace` for isolation and coordination. --- ## When to Use Use Walrus Memory when your app or agent needs: - **Portable memory** — persists outside prompts and context windows, moves across agents, apps, and workflows - **Full owner control** — programmable permissions and explicit ownership define how memory is shared and accessed - **Agent coordination** — shared memory spaces help agents coordinate across long-running and multi-step workflows - **Semantic recall** — retrieve memories by meaning, not just keywords - **Verifiable integrity** — memory integrity can be independently verified without centralized trust - **Cross-app memory** — not tied to a single runtime or provider, share memory between apps via delegate keys --- ## When NOT to Use - Temporary conversation context that only matters in the current session - Large file storage (Walrus Memory is optimized for text memories) - Use cases that don't need encryption or decentralization --- ## Installation ```bash # Install the SDK pnpm add @mysten-incubation/memwal # Optional: for Vercel AI SDK integration pnpm add ai zod # Optional: for manual client (client-side SEAL encryption) pnpm add @mysten/sui @mysten/seal @mysten/walrus ``` --- ## Quick Start ### 1. Get Your Credentials You need a **delegate key** (Ed25519 private key) and **account ID** (Walrus Memory account object ID on Sui). Generate them at: - Production: https://memory.walrus.xyz - Staging: https://staging.memory.walrus.xyz ### 2. Initialize the SDK ```ts import { MemWal } from "@mysten-incubation/memwal"; const memwal = MemWal.create({ key: process.env.MEMWAL_PRIVATE_KEY!, accountId: process.env.MEMWAL_ACCOUNT_ID!, serverUrl: process.env.MEMWAL_SERVER_URL ?? "https://relayer.memory.walrus.xyz", namespace: "my-app", }); ``` ### 3. Store and Recall Memories ```ts // Store one already-distilled fact and wait until it is indexed. await memwal.rememberAndWait( "User prefers dark mode and works in TypeScript.", undefined, { timeoutMs: 30_000 }, ); // Recall by meaning const result = await memwal.recall({ query: "What are the user's preferences?" }); console.log(result.results); // Extract facts from free-form text and wait until all accepted facts are indexed. const analyzed = await memwal.analyzeAndWait( "I live in Hanoi and prefer dark mode.", undefined, { timeoutMs: 30_000 }, ); console.log(analyzed.facts.map((fact) => fact.text)); // Check relayer health await memwal.health(); ``` Use `*AndWait` when a workshop UI saves and then immediately recalls in the same flow. Indexing can lag by a few seconds, so `remember()` / `analyze()` may return before recall can find the new memory. Manual polling is still available for advanced async UIs: ```ts const accepted = await memwal.remember("User likes Sui."); const stored = await memwal.waitForRememberJob(accepted.job_id, { pollIntervalMs: 750, timeoutMs: 30_000, }); ``` --- ## SDK Entry Points | Entry Point | Import | Description | |---|---|---| | `MemWal` | `@mysten-incubation/memwal` | **Default.** Relayer handles embedding, SEAL encryption, Walrus upload, vector search | | `MemWalManual` | `@mysten-incubation/memwal/manual` | Manual flow — client handles embedding and SEAL encryption | | `withMemWal` | `@mysten-incubation/memwal/ai` | Vercel AI SDK middleware — auto recall + save around AI conversations | | Account utils | `@mysten-incubation/memwal/account` | Account creation, delegate key management | --- ## API Surface ### Walrus Memory Methods | Method | Description | Returns | |---|---|---| | `remember(text, namespace?)` | Accept one memory job immediately | `{ job_id, status }` | | `rememberAndWait(text, namespace?, opts?)` | Store one memory and wait for completion | `{ id, job_id, blob_id, owner, namespace }` | | `recall({ query, limit?, topK?, namespace?, maxDistance? })` *(preferred)* or `recall(query, limit?, namespace?)` | Semantic search for memories | `{ results: [{ blob_id, text, distance }], total }` | | `analyze(text, namespace?)` | Extract facts and accept one memory job per fact | `{ job_ids, facts, fact_count, status, owner }` | | `analyzeAndWait(text, namespace?, opts?)` | Extract facts and wait for all fact jobs to complete | `{ results, facts, total, succeeded, failed, owner }` | | `restore(namespace, limit?)` | Rebuild missing index entries from Walrus | `{ restored, skipped, failed, total, namespace, owner, truncated }` | | `health()` | Check relayer health | `{ status, version }` | | `getPublicKeyHex()` | Get hex-encoded public key | `string` | ### Lower-Level Methods | Method | Description | |---|---| | `rememberManual({ encryptedData, vector, namespace? })` | Send SEAL-encrypted bytes + pre-computed vector; relayer uploads | | `recallManual({ vector, limit?, namespace? })` | Search with pre-computed vector (returns blob IDs only) | | `embed(text)` | Generate embedding vector (no storage) | ### All Response Shapes ```ts interface RememberAcceptedResult { job_id: string; status: string; } interface RememberJobStatus { job_id: string; status: "pending" | "running" | "uploaded" | "done" | "failed" | "not_found"; owner?: string; namespace?: string; blob_id?: string; error?: string; } interface RememberResult { id: string; job_id?: string; blob_id: string; owner: string; namespace: string; } interface RecallMemory { blob_id: string; text: string; distance: number; } interface RecallResult { results: RecallMemory[]; total: number; } interface RecallOptions { limit?: number; topK?: number; // alias of limit; if both are set, topK wins namespace?: string; maxDistance?: number; } // `topK` and `limit` are aliases for the same value; if both are provided, `topK` takes precedence. interface RememberBulkAcceptedResult { job_ids: string[]; total: number; status: string; } interface AnalyzedFact { text: string; id: string; job_id?: string; blob_id?: string; } interface AnalyzeResult { job_ids: string[]; facts: AnalyzedFact[]; fact_count: number; status: string; owner: string; } interface RememberBulkStatusItem { job_id: string; status: "pending" | "running" | "uploaded" | "done" | "failed" | "not_found"; blob_id?: string; error?: string; } interface RememberBulkStatusResult { results: RememberBulkStatusItem[]; } interface RememberBulkItemResult { id: string; blob_id: string; status: "done" | "failed" | "timeout"; namespace: string; error?: string; } interface RememberBulkResult { results: RememberBulkItemResult[]; total: number; succeeded: number; failed: number; } interface AnalyzeWaitResult extends RememberBulkResult { facts: AnalyzedFact[]; owner: string; } interface EmbedResult { vector: number[]; } interface RestoreResult { restored: number; skipped: number; failed: number; total: number; namespace: string; owner: string; truncated: boolean; } interface HealthResult { status: string; version: string; mode?: string; prompt_versions?: { extract: string; ask: string; }; relayerVersion?: string; apiVersion?: string; minSupportedSdk?: { typescript: string; python: string; mcp: string; }; featureFlags?: Record<string, boolean>; deprecations?: Array<{ surface: string; deprecatedSince: string; removalApiVersion: string; guidance: string; }>; build?: { commit?: string; buildTimestamp?: string; }; } ``` `facts[].text` is the extracted fact text to render in UIs. `job_ids[]` aligns with the accepted fact jobs; use `analyzeAndWait()` when the UI needs those facts indexed before continuing. ### Namespace Semantics A namespace is an **opaque, flat string label** scoped to a single owner. It is the unit of memory isolation: a recall in namespace `A` will never surface entries written to namespace `B`, even for the same owner, and never surfaces other owners' entries even in the same namespace. #### Validation Omit `namespace` and the server uses the literal string `"default"`. An explicit empty string is rejected with HTTP 400 (`namespace cannot be empty`). The server then accepts any non-empty UTF-8 string except: - more than **255 bytes** (UTF-8 byte length, not character count — Rust `str::len()`) → HTTP 400 `namespace exceeds maximum length of 255 bytes` - a NUL byte (`\0`) → HTTP 400 `namespace contains a NUL byte` (WALM-439). Tabs, newlines, and other control characters are still allowed so older namespaces stay readable. There is no character whitelist, no case folding, no trim, and no Unicode normalization. Whatever passes validation is stored verbatim and matched with exact equality. > **Implication:** `"my-app"`, `" my-app"` (leading space), `"My-App"`, and `"my-app/"` are four distinct namespaces. Pick a convention and stick to it. Multi-byte characters (CJK, emoji) consume more than one byte each, so they hit the 255-byte cap sooner than a character count would suggest. #### Flat, not hierarchical Slashes and dots have **no special meaning**. `"chat/user-42"` is a single opaque label, not a path. The server uses `WHERE namespace = $1` exact-equality for every read; there is no prefix matching, no parent/child traversal, and no wildcard query. If you need hierarchy, build it in the application layer (e.g. recall across known namespaces and merge client-side). #### Overwrite behavior — `remember()` is **always append, never upsert** Every accepted `remember()` call creates a **new memory entry** with a freshly generated UUID. Sending the same text to the same `(owner, namespace)` twice will produce **two separate entries** that both surface in future recalls. The namespace is metadata for filtering, not a key for deduplication. ```ts await memwal.remember("I prefer dark mode", "prefs"); await memwal.remember("I prefer dark mode", "prefs"); // recall("preferences", { namespace: "prefs" }) → 2 entries, both with the same text ``` If you need uniqueness, either dedupe before calling `remember()`, or delete the prior entry first. #### Isolation guarantees | Scenario | Visible to recall? | |---|---| | Same owner, same namespace | ✅ | | Same owner, different namespace | ❌ | | Different owner, same namespace | ❌ | | Different owner, different namespace | ❌ | Cross-namespace and cross-owner reads are not just filtered out of results — the server's SQL `WHERE` clause excludes them entirely, so they are never decrypted or transferred. ### Restore Semantics `restore(namespace, limit?)` rebuilds **missing** local index entries for a namespace from Walrus. It is a recovery operation, not a sync — already-indexed blobs are left alone. #### Response fields | Field | Counts | Notes | |---|---|---| | `restored` | Blobs the relayer just rebuilt this call | Pulled from Walrus → SEAL decrypted → re-embedded → inserted as a new row | | `skipped` | On-chain blobs already in the local **success** index | No work needed; relayer left them as-is. Does not include decrypt/UTF-8 failures. | | `failed` | Permanent decrypt/UTF-8 failures | On-chain blobs in this page that are negative-cached, plus new permanent failures this call. Older relayers omit the field; SDKs default it to `0`. | | `total` | All on-chain blobs the relayer saw for `(owner, namespace)` | Before the limit was applied | | `namespace` | Echo of the request | | | `owner` | Resolved owner address | | | `truncated` | Known-retryable-incomplete | `true` is not a hard failure; `false` is not completeness | `truncated=true` means this restore is **known-retryable-incomplete**: more missing blobs than `limit` allowed this call to restore, **or** the sidecar's owner-wide candidate fetch hit its cap **and** raising `limit` can still expand that fetch (`limit < 20`). Once the sidecar cap is saturated (`limit >= 20`, cap pinned at 100), truncation follows this call's missing-blob page length, not onchain `total`. A fully restored namespace does not loop. `truncated=false` is **not** proof the sidecar saw every onchain blob; blobs beyond the owner-wide sidecar candidate cap can still be missing. WALM-451 tracks a `sourceCapped` field for that case. Relayers older than WALM-319 omit `truncated`; SDKs default it to `false`. Permanent decrypt or invalid-UTF-8 failures count in `failed`, not `skipped`. Transient download/decrypt/embed errors are still not counted in `restored`, `skipped`, or `failed` and may be retried (`truncated=true` when a page yields only those). `restored + skipped + failed` therefore never exceeds `total`, and falls short of it whenever transient errors leave blobs uncounted. #### Default and limit * `limit` defaults to `10` in both TypeScript and Python SDKs and matches the server-side default. The Python SDK historically defaulted to `50`; it is now realigned with the server. * `limit` caps the **inspected** blob set, newest-first. It does not cap `restored` independently — if all 10 inspected blobs are already indexed, `restored = 0` and `skipped = 10`. * The relayer clamps `limit` to 1–100 (values outside that range are clamped, not rejected). #### Pagination **Restore is single-shot — there is no cursor.** To rebuild a namespace larger than your chosen `limit`, call again with a larger `limit`, or delete local rows you want re-imported first. Pagination is on the roadmap; until it lands, treat `restore()` as a "top up to N most recent" operation. #### Performance Latency scales linearly in `limit`: * Up to **10 concurrent** Walrus aggregator downloads * Up to **3 concurrent** SEAL decrypts (CPU-bound, capped intentionally) * Embedding requests in parallel (bounded by the relayer's embedding pool) Expect **seconds per blob** on a cold cache. Use small limits (≤ 50) for interactive flows and run larger restores out-of-band. ### Recall Distance and Filtering `recall()` returns the closest K memories by vector distance. There is no default relevance threshold, so small namespaces may return weak filler results because they are still the closest available matches. Lower distance means more similar: | Distance | Rough meaning | |---|---| | `< 0.25` | Duplicate or very close | | `0.25 - 0.55` | Related |