When auditing agent state portability, setting up a new machine, or investigating why gitignore exceptions aren't working
effort
medium
Agent Learnings Portability
Problem
AI agents (Codex, Hermes, Codex, Gemini) accumulate valuable learnings in machine-local directories (~/.Codex/, ~/.hermes/, ~/.codex/, ~/.gemini/). Without explicit git tracking, these are lost on machine failure.
Architecture: Three Tiers
Git-committed in .Codex/memory/ (survives git clone, PRIMARY): Memory facts, conventions, user context — travels with the repo
Rsync backup (survives single-machine failure): Large files — session transcripts, SQLite databases, full Codex project memory dirs
Regenerable (no backup needed): Caches, plugins, debug logs
The .Codex/memory/ Pattern (issue #1886)
The primary mechanism for cross-machine context parity is writing canonical memory facts directly into .Codex/memory/ inside the repo. This directory is already git-tracked and read by Codex at session start on every machine.
How it works:
Hermes memory (~/.hermes/memories/MEMORY.md + USER.md) is the authoritative source
.Codex/memory/agents.md — User profile, AI subscriptions, workflow rules, GSD facts
.Codex/memory/Codex-auto-memory.md — Snapshot of Codex's auto-generated MEMORY.md
scripts/memory/bridge-hermes-Codex.sh — The bridge script, run with --commit flag
Why not tarballs or export/import scripts? One-time snapshots go stale immediately. Git gives version history, diffs, rollback, and automatic updates on every git pull. The bridge should run daily via cron, not manually.
Return enrichment flow: Non-Hermes machines (Windows) enrich KNOWLEDGE.md or topic files → git commit/push → Linux pulls → bridge script picks up on next run via Codex auto-memory snapshot capture.
This pattern REPLACES the older config/agents/ snapshot approach for memory (keep config/agents/ for large state files like session exports, but use .Codex/memory/ for human-readable context).
What to Git-Track (by agent)
Hermes (~/.hermes/)
memories/MEMORY.md + USER.md → snapshot to config/agents/hermes/memories/
Codex (~/.Codex/)
projects/<encoded-path>/memory/*.md → snapshot to config/agents/Codex/memory-snapshots/
Especially feedback_*.md (user corrections) and project_*.md (context)
history.jsonl, session_index.jsonl
→ snapshot to config/agents/codex/state-snapshots/
Gemini (~/.gemini/)
state.json, projects.json
→ snapshot to config/agents/gemini/state-snapshots/
CRITICAL PITFALL: Gitignore Directory vs Glob
This is the #1 gotcha that makes gitignore exceptions silently fail.
The Bug Pattern
# THIS IS BROKEN — exceptions below are DEAD CODE
.Codex/state/ # <-- trailing slash = ignores the DIRECTORY
!.Codex/state/corrections/ # <-- NEVER WORKS
When git ignores a directory (trailing /), it never looks inside it, so child exceptions cannot un-ignore anything.
The Fix (three layers required)
# Layer 1: Top-level ignore with glob (not directory)
.Codex/*
# Layer 2: Un-ignore the parent directory itself
!.Codex/state/
# Layer 3: Re-ignore contents, THEN add specific exceptions
.Codex/state/*
!.Codex/state/corrections/
!.Codex/state/patterns/
!.Codex/state/learned-patterns.json
Verification
# Check if a file is ignored (should show NO output for tracked files)
git check-ignore -v .Codex/state/corrections/foo.jsonl
# If it shows a rule, the exception is NOT working# The -v flag shows WHICH rule is blocking
Debugging Workflow (when exceptions don't work)
# Step 1: Check which rule is blocking
git check-ignore -v .Codex/state/corrections/foo.jsonl
# Output shows the EXACT line: ".gitignore:127:.Codex/*"# Step 2: Check if there are MULTIPLE blocking rules
git check-ignore -v .Codex/state/corrections/ .Codex/state/learned-patterns.json
# Step 3: After fixing, verify files CAN be staged
git add .Codex/state/corrections/ 2>&1
# If you see "ignored by .gitignore" there's still a blocking rule
Common Failure Modes
Parent dir ignored with / instead of /* — children can never be excepted
Multiple ignore rules at different levels — e.g., .Codex/* at line 127 AND .Codex/state/ at line 157. You fix one but the other still blocks. Always run git check-ignore -v AFTER each fix to verify
Exception added but files never git add-ed — the gitignore exception alone doesn't track files
Cron uses inline command instead of script — after updating a backup script, verify the crontab actually calls the script (not an old inline rsync)
Nightly Pipeline Integration
Script: scripts/cron/commit-learning-artifacts.sh
Steps:
Snapshot agent memories from home dirs to config/agents/*/
git add all state directories and snapshot directories
Run legal-sanity-scan.sh --diff-only (MANDATORY — corrections and session data can contain client names)
Commit and push if changed
Wire into: comprehensive-learning-nightly.sh as final step.
Session-Signal Redaction
session-signals/*.jsonl files contain a last_assistant_message field with full LLM response text. This text often references client project names (from doc-intelligence classification outputs). The field must be redacted before git-add.
Script: scripts/cron/redact-session-signals.sh
Scans all .Codex/state/session-signals/*.jsonl
Replaces last_assistant_message with [REDACTED] using Python JSON parsing
Preserves all metadata fields (session_id, hook_event_name, permission_mode, etc.)
Must run BEFORE git add in the nightly pipeline
Codex Session Export
Codex sessions live at ~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl in a different format than Codex/Hermes.
Key Codex JSONL types:
session_meta — session ID, cwd, model provider
response_item with payload.type == "function_call" — tool calls
response_item with payload.type == "function_call_output" — results
Exports to logs/orchestrator/codex/session_YYYYMMDD.jsonl
Maps Codex tool names to orchestrator conventions (exec_command→Bash, etc.)
Incremental via .last-export-ts timestamp file
Legal Risk
Session signals, corrections, and memory files can contain client project names. The legal scan MUST gate all commits. If it finds violations, skip the commit rather than pushing sensitive data.
Key mitigation: run redact-session-signals.sh before staging. The session-signals directory is the #1 source of legal violations (522 hits in first attempt without redaction).
Backup Coverage (Tier 2)
scripts/cron/memory-backup.sh rsyncs to ace-linux-2 daily: