deep-sleep

name	deep-sleep
description	Extract learnings from past Codex session transcripts and create appropriately-typed skills. Drains the Stop-hook learnings queue and scans rollout JSONL for recurring patterns.
queries	["learn from past sessions","extract patterns from conversation history","create memories from session transcripts","what did I learn in past sessions","analyze session history for patterns","process pending learnings queue"]

/deep-sleep — Extract Learnings from Session Transcripts

Analyze past Codex session transcripts to extract user preferences, recurring patterns, and troublesome workflows, then create appropriately-typed skills for future semantic injection.

When to Use

• Daily to consolidate learnings from recent sessions
• After a productive session where many patterns were established
• When you notice the same corrections being made repeatedly
• When ~/.codex/cache/memex-learnings-queue.json has pending sessions (Stop hook extractLearnings)

Context: Codex session storage

Codex stores the canonical turn log in rollout JSONL, not Claude-style project jsonl:

~/.codex/sessions/YYYY/MM/DD/rollout-<timestamp>-<uuid>.jsonl

Each line is a JSON object. User messages appear as:

{"type":"event_msg","payload":{"type":"user_message","message":"..."}}

The Stop hook (when hooks.Stop.extractLearnings: true) enqueues sessions mechanically — no LLM in the hook. This skill is the LLM consumer.

Process

Perform the following steps directly — no external scripts or API keys needed.

1. Drain the learnings queue (primary)

Pending sessions live at:

~/.codex/cache/memex-learnings-queue.json

Queue shape (version: 1):

{
  "version": 1,
  "sessions": [
    {
      "session_id": "019f…",
      "cwd": "/path/to/worktree",
      "transcript_path": "${CODEX_HOME:-~/.codex}/sessions/…/rollout-….jsonl",
      "captured_at": "2026-07-03T06:00:00.000Z",
      "user_message_count": 12
    }
  ]
}

All queue mutations must run under withFileLock on the queue file (same semantics as enqueuePendingSession in src/core/learnings-queue.ts):

1. Acquire lock on memex-learnings-queue.json
2. Load and validate the queue (version: 1, filter malformed entries)
3. For each pending session (process oldest-first or as listed):
   • Read user messages via listUserMessages(transcript_path) — codexstore scans event_msg / user_message lines, min 10 chars (do not reimplement the parser)
   • Analyze messages for reusable learnings (step 3 below)
   • Write session-learning entries to ~/.codex/memex/projects/<encoded-cwd>/memory/ where <encoded-cwd> is the session's cwd with / → -, . → -, _ → -
   • Remove the processed entry from sessions (dequeue under the same lock)
4. Persist atomically: write path.<random>.tmp then rename(tmp, path)

If the queue is empty, proceed to step 2 (supplemental transcript scan).

Dedup: enqueuePendingSession skips duplicate session_id values. When dequeuing, only remove sessions you successfully processed.

2. Supplemental transcript scan (optional)

When the queue is empty or the user requests --since, scan rollout files directly.

Resolve transcripts from the current working directory:

~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl

Match rollouts whose first session_meta line has payload.cwd equal to the current cwd (resolved path). Prefer the newest rollout filename when multiple match.

Check the watermark to skip already-processed rollouts:

~/.codex/cache/deep-sleep-watermark

If the watermark exists, only process rollout files modified after that timestamp. If no watermark exists, process files from the last 7 days. The user may also specify --since YYYY-MM-DD.

For each rollout, extract user messages the same way: conceptually listUserMessages(path) — event_msg lines where payload.type is user_message, content longer than 10 characters.

3. Analyze for learnings

Review the collected user messages and identify reusable patterns. Look for:

• Preferences: "always use X", "prefer Y over Z", "don't use W"
• Recurring corrections: User repeatedly fixing the same kind of mistake
• Workflow patterns: Multi-step processes the user follows
• Tool usage tips: Guidance about how to use specific tools (Bash, Edit, etc.)
• Stop rules: Patterns in assistant responses that should trigger continuation

Skip one-off requests. Only extract clear, reusable patterns.

For queue-drained sessions, prefer type: session-learning for ephemeral session-scoped facts. Promote to memory or rule only when the pattern is clearly durable across sessions.

4. Diagnose match quality (ASI extraction)

Scan each processed transcript for memex injection markers — lines containing "The following was automatically loaded based on semantic relevance".

For each injection found:

1. Identify which skill/memory/rule was injected (from the section headings that follow the marker)
2. Classify the outcome:
   • used: The assistant clearly used the injected knowledge in its response
   • ignored: The injected knowledge was not referenced or acted upon
   • corrected: The user corrected the assistant's response in a way that contradicts the injected knowledge
3. Write a one-sentence diagnosis explaining why the match was helpful or unhelpful

Also scan for missed matches — cases where the user provided information that an existing skill already contains, but that skill was not injected. For missed matches, use score: 0 and queryIndex: -1.

Record each observation to telemetry:

cat ~/.codex/cache/memex-telemetry.json

For each observation, add it to the entry's observations array in the telemetry file. The observation format:

{
  "sessionId": "<session-id>",
  "prompt": "<the user prompt that triggered the injection>",
  "score": <similarity score or 0 for missed>,
  "queryIndex": <index of the matching query or -1 for missed>,
  "outcome": "used|ignored|corrected|missed",
  "diagnosis": "<one-sentence explanation>",
  "timestamp": "<ISO timestamp>"
}

Cap observations at 100 per entry (keep newest).

5. Deduplicate against existing knowledge

For each candidate learning, use memex's own semantic search to check for overlapping entries. Pipe the learning text as a UserPromptSubmit query:

echo '{"hook_event_name":"UserPromptSubmit","session_id":"deep-sleep-dedup","cwd":"<cwd>"}' \
  | jq --arg prompt "$CANDIDATE_TEXT" '. + {prompt: $prompt}' \
  | $PLUGIN_ROOT/bin/memex

If the output contains additionalContext with a match at relevance >= 80%, the learning is already covered. Read the matched entry to confirm — if the existing entry says the same thing, skip the candidate. If the existing entry is related but incomplete, update it instead of creating a duplicate.

This uses the same embedding-based similarity that memex uses at runtime, so dedup quality matches injection quality.

6. Classify and create entries

For each novel learning, determine the right type based on how critical and universal it is:

Pattern observed	Type	Destination
Corrected 3+ times across sessions	rule	~/.codex/skills/<name>/SKILL.md or .agents/skills/<name>/SKILL.md with type: rule
Preference or fact stated once	memory	~/.codex/skills/<name>/SKILL.md or .agents/skills/<name>/SKILL.md
Session-scoped ephemeral fact	session-learning	~/.codex/memex/projects/<encoded-cwd>/memory/<name>.md
Multi-step procedure	skill	~/.codex/skills/<name>/SKILL.md or .agents/skills/<name>/SKILL.md
Ordered multi-step process	workflow	~/.codex/skills/<name>/SKILL.md or .agents/skills/<name>/SKILL.md
Tool-specific guidance	tool-guidance	~/.codex/skills/<name>/SKILL.md or .agents/skills/<name>/SKILL.md
Stop condition pattern	stop-rule	~/.codex/skills/<name>/SKILL.md or .agents/skills/<name>/SKILL.md

For entries classified as rules (corrections made 3+ times), create a SKILL.md with full frontmatter:

---
name: <kebab-case-name>
description: "<one sentence: what this rule prevents>"
type: rule
queries:
  - "<query 1>"
  - "<query 2>"
  - "<query 3>"
one-liner: "<short reminder version>"
---
<the full rule explanation>

For session-learning and other types, create a SKILL.md or memory markdown:

---
name: <kebab-case-name>
description: "<one sentence: when is this useful>"
type: <session-learning|memory|skill|workflow|tool-guidance|stop-rule>
queries:
  - "<natural query 1>"
  - "<natural query 2>"
  - "<natural query 3>"
  - "<natural query 4>"
  - "<natural query 5>"
---
<the actual instruction or knowledge, 1-5 lines>

7. Update watermark

After supplemental rollout scans (step 2), write the current ISO timestamp:

mkdir -p ~/.codex/cache
date -u +%Y-%m-%dT%H:%M:%SZ > ~/.codex/cache/deep-sleep-watermark

Skip this when you only drained the learnings queue.

8. Report results

Summarize what was created:

• Queue sessions drained (count)
• Rollout files scanned (if any)
• Learnings found (by type)
• Rules created (for repeatedly-corrected patterns)
• Session-learning files written under ~/.codex/memex/projects/<encoded>/memory/
• Skills/memories created (for other learnings)
• Duplicates skipped

Options

The user may specify:

• --dry-run: Show extracted learnings without creating files
• --since <date>: Process rollout transcripts from this date (ISO format)
• --global-scope: Write skills to ~/.codex/skills/ instead of .agents/skills/
• --queue-only: Drain memex-learnings-queue.json only; skip supplemental rollout scan

$ARGUMENTS