# output-compress
Tiered, opt-in output compression with a deterministic fidelity gate. See `SKILL.md`
for the full rules (Claude Code / Agent Skills format) or `AGENTS.md` for the same
rules in Codex's format. `USAGE.md` covers day-to-day workflows, log analysis, and
auto-activation wiring (§7).
## What's in the box (v1.5.0)
- **Three compression levels** — `lite` / `full` / `ultra` (caveman-derived syntax);
deletion only, never rewriting, with negation-first ordering to avoid the most
common gate failure (`SKILL.md` §1, §4).
- **Model-tier compression caps** — the level is capped by who *reads* the text
(small models `lite`, mid-tier `full`, frontier `ultra`; uncalibrated models start
at `lite`) (`SKILL.md` §2).
- **Never-compress whitelist** (7 categories) — code/paths/URLs/env vars, all numbers
and dates, negations + quantifier bounds with their clause, project structured tags
(`CUSTOM_TAG_PATTERNS`), safety-critical statements, contract fields as whole
blocks, and hedges/qualifiers (the "decontextualization" axis) (`SKILL.md` §3).
- **Deterministic fidelity gate** — `scripts/fidelity-check.py` regex/set-diffs the
whitelist between original and compressed text; no LLM self-judgment. `--coverage`
pre-check tells you when compressing isn't worth it; `--log` appends JSONL records
with `saving_pct` and the `grounded_pct` rewriting-detector axis (`SKILL.md` §4).
- **Delegated compression** (optional) — hand the deletion judgment to a cheap worker
model while the orchestrator keeps the gate (scaling-paradox grounding: bigger
compressor models are *less* faithful) (`SKILL.md` §4b).
- **Pace/usage coupling** (optional) — `scripts/usage-pacer.py` emits
AHEAD/ON_PACE/BEHIND burn-rate verdicts plus absolute-threshold escalation (80%/95%
used) and handoff states (`HANDOFF_PREP`/`HANDOFF_HALT`); provider feeders
`claude-usage-fetch.py` / `codex-usage-fetch.py` supply real usage numbers, and
`codex-handoff.py` offers opt-in packet-backed handoff wiring for Codex
(`SKILL.md` pace sections, `USAGE.md` §7).
## Install
**Fastest path:** paste `INSTALL-PROMPT.md` (below the `---`) into any AI assistant
with filesystem access — it detects your environment, installs, calibrates the
negation list for your language, and runs the mechanical verification for you.
Manual paths:
**Claude Code (Agent Skills):**
```bash
cp -r output-compress ~/.claude/skills/
```
Claude Code will pick up `SKILL.md`'s frontmatter automatically on next session start.
**Codex Skill:**
Install the repo root as a Codex skill into `$CODEX_HOME/skills`:
```bash
python3 "${CODEX_HOME:-$HOME/.codex}/skills/.system/skill-installer/scripts/install-skill-from-github.py" \
--repo zeuikli/output-compress --path . --name output-compress
```
Codex will make the `SKILL.md` frontmatter available on the next turn or new session.
If the destination already exists, remove or rename the old
`$CODEX_HOME/skills/output-compress` directory only after confirming you do not need
that copy.
**Codex AGENTS.md fallback / always-loaded advisory:**
Append the section from `AGENTS.md` (everything after the `---`) into your project's
own `AGENTS.md` file, or copy this whole directory into your project and add a pointer
line to your `AGENTS.md`, e.g. `See output-compress/AGENTS.md for the compression
advisory.`
For the AGENTS.md fallback, also copy `scripts/fidelity-check.py` into your project so
the gate command in the docs resolves to a real path. With a Codex skill install, use
the installed skill path, e.g.
`$CODEX_HOME/skills/output-compress/scripts/fidelity-check.py`. `scripts/usage-pacer.py`
is optional — only needed for the pace-coupling described in `SKILL.md` / `USAGE.md`
§7. Claude and Codex users can also copy the optional provider feeders
(`scripts/claude-usage-fetch.py` or `scripts/codex-usage-fetch.py`), which feed the
pacer real usage numbers from the provider's usage endpoint instead of a
bring-your-own JSON feed. Codex users who want packet-backed, memory-assisted handoff
wiring can opt in with `OC_CODEX_HANDOFF_PACKET=1` and wire
`python3 "${CODEX_HOME:-$HOME/.codex}/skills/output-compress/scripts/codex-handoff.py" --refresh`
as a `UserPromptSubmit` hook. It writes an
authoritative pending JSON packet plus a derived Markdown view when the verdict is
fresh in a Git working tree. `--write-packet` validates nonblank checkpoint fields and a
complete repo guard before PREP becomes ready; the host then receives `resume_at`, a stable
heartbeat name, and an exact-ID/path prompt. After a host receipt, `--resume-context`
rejects repository drift. The helper does not write Codex
memory or create automation. Register both `UserPromptSubmit` and
`SessionStart(compact|resume)`, then review/trust them with `/hooks`. The default
`.codex/handoffs` directory may produce untracked repo metadata; configure
`OC_CODEX_HANDOFF_DIR` if that is not desired.
## Quickstart: fidelity gate demo
The gate is the deterministic script in `scripts/fidelity-check.py`. It diffs
whitelisted elements (numbers, negations, quantifier bounds, hedges, code, paths,
URLs, tags) between an original and a compressed file. Both examples below use `--log` so you can see the JSONL trail
it leaves — a failed (exit 1) attempt is logged too, not just discarded, because the
failure pattern itself is useful signal (see `SKILL.md` §5).
**GOOD — compression that preserves every whitelisted element (exit 0):**
```bash
cat > /tmp/orig-good.txt <<'EOF'
The deploy failed 3 times. Do not retry without checking `deploy.log` first.
See https://example.com/runbook for the rollback steps.
EOF
cat > /tmp/comp-good.txt <<'EOF'
Deploy failed 3 times. Do not retry without checking `deploy.log`.
Rollback: https://example.com/runbook
EOF
rm -f ./compress-log.jsonl
python3 scripts/fidelity-check.py \
--original /tmp/orig-good.txt --compressed /tmp/comp-good.txt \
--log --level full --context report
echo "exit=$?"
wc -l ./compress-log.jsonl
```
Expect `exit=0` and `./compress-log.jsonl` to contain exactly 1 line with `"pass": true`.
**BAD — compression that drops the retry count and the negation (exit 1):**
```bash
cat > /tmp/orig-bad.txt <<'EOF'
The deploy failed 3 times. Do not retry without checking `deploy.log` first.
EOF
cat > /tmp/comp-bad.txt <<'EOF'
Deploy failed. Check deploy.log then retry.
EOF
python3 scripts/fidelity-check.py \
--original /tmp/orig-bad.txt --compressed /tmp/comp-bad.txt \
--log --level full --context report
echo "exit=$?"
wc -l ./compress-log.jsonl
```
The BAD example strips both the number (`3`) and the negation (`Do not retry`), and
also drops the inline-code backticks around `deploy.log` — the script reports all three
categories as missing and exits 1, which per the skill's rule means: drop one
compression level and retry, or fall back to the original text. `./compress-log.jsonl`
now has 2 lines — the first `"pass": true` entry from the GOOD run above, and a second
`"pass": false` entry recording exactly what this failed run lost.
## Customizing the whitelist
If your project uses its own structured tags (ticket IDs, custom status markers,
evidence-level flags), add regex patterns to `CUSTOM_TAG_PATTERNS` at the top of
`scripts/fidelity-check.py` rather than editing the built-in categories. Non-English
projects should also extend the `NEGATIONS` and `HEDGES` word lists with their
language's negation and qualifier words before first use — the gate can only protect
what it knows to look for.
## Attribution
Compression syntax and the three-tier (`lite`/`full`/`ultra`) idea are derived from
[`github.com/juliusbrussee/caveman`](https://github.com/juliusbrussee/caveman). This
version differs from upstream in two ways: (1) a deterministic fidelity gate replaces
upstream's reliance on model self-judgment of "meaning stays clear", and (2)
model-tier compression caps replace a single fixed compression level for all readers.
## License
MIT — see [LICENSE](LICENSE).