pdf-lab

# pdf-lab `/pdf-lab` is a convergence loop that diagnoses PDF extraction failures, reproduces them on synthetic PDFs, discovers optimal parameters, and **writes those fixes back to the extractor pipeline code** permanently. ## Why This Exists There is no absolute ground truth for real PDFs. The system works with **deltas** between S00's estimate and actual extraction results. When S00 predicts 40 sections but extraction finds 15, something is wrong. `/pdf-lab` figures out what, fixes it, and makes the fix permanent. ## Quick Start ```bash cd ${HOME}/workspace/experiments/pi-mono/.pi/skills/pdf-lab # Main: diagnose, reproduce, converge, and write fix back ./run.sh tune /path/to/real.pdf \ --review-json /path/to/review_result.json \ --debug-json /path/to/debug_patterns.json \ --converge --write-back --json # Dry run: find the fix but don't write it ./run.sh tune /path/to/real.pdf \ --review-json ... --debug-json ... \ --converge --dry-run --json # Quick diagnosis only (compute delta, no tuning) ./run.sh diagnose /path/to/real.pdf \ --profile-json /path/to/profile.json \ --structural-json /path/to/structural.json # Generate synthetic reproduction PDF only ./run.sh synthetic \ --patterns '["multi_column","split_tables"]' \ --output /tmp/repro.pdf # Show recent tuning results ./run.sh status # List all pdf-lab code changes ./run.sh history # Rollback a specific fix ./run.sh rollback --sha abc123 # Deterministic post-run verification for substantial-skill jobs ./run.sh verify --job-dir /tmp/pdf-lab-job # Build a maintainer escalation packet when verify fails ./run.sh file-maintainer-ticket --job-dir /tmp/pdf-lab-job # Standalone PDF Lab UX cd ${HOME}/workspace/experiments/agent-skills/skills/pdf-lab/ui npm install npm run dev:all ``` If the workstation has exhausted file watchers, use the no-watch preview path: ```bash npm run build npm run preview:all ``` The standalone UI runs at `http://127.0.0.1:3012/#pdf-lab`. In `dev:all`, the Vite app runs on port `3012` and the local API bridge runs on port `3013`. In `preview:all`, the API bridge serves the built UI and API together on port `3012`. The bridge serves real artifacts from `PDF_LAB_PUBLIC_ROOT` and `PDF_LAB_ARTIFACTS_ROOT`; if an artifact or runtime bridge is missing, endpoints fail closed with an explicit JSON error instead of returning mock operational state. ## How Fixes Get Written Back The pipeline has a tiered configuration system. `/pdf-lab` writes to the appropriate tier: | Tier | Target | Example | |------|--------|---------| | 1 | Env var defaults in step files | `CAMELOT_LINE_SCALE_DEFAULT` 15 -> 40 | | 2 | Heuristic thresholds (code constants) | `LARGE_FONT_THRESHOLD` 11.0 -> 9.5 | | 3 | Pattern rules (regex, filters) | New citation pattern in S04 | | 4 | Preset YAML | `line_scale: 80` in arxiv twin_config.yml | | 5 | Calibration records (ArangoDB) | Learned pattern in `learned_patterns` | | 6 | /memory (runtime recall) | Winning params stored for instant recall | ## Persona Attribution Every code change is traceable to the persona who flagged the issue via git commit trailers (`Reviewed-By`, `Persona-Role`, `Issue-Codes`). ## Integration Called by `inline_review_loop.py` when a persona review score is below threshold. Falls back to heuristic adaptive params if convergence fails. ## Memory + Taxonomy Integration The skill integrates with `/memory` through `memory/run.sh` subcommands in `memory_integration.py`; it does not import ArangoDB clients or the memory Python package directly. - **Pre-hook (`recall_prior_convergence`)**: Before tuning, recalls prior convergence results for the same PDF type or URL. Enables the tuner to skip failed strategies and start from previously winning parameters. - **Post-hook (`learn_convergence`)**: After tuning completes, stores the convergence outcome (strategy, iterations, final score, improvements, write-back results) to memory with taxonomy bridge tags for cross-skill recall. - **Bridge keywords**: Precision, Resilience, Fragility, Corruption, Loyalty, Stealth (tuned to PDF extraction domain). - **Tags**: `["pdf_lab", "convergence"] + bridges` Gracefully degrades if `/memory` is unavailable. ## Runtime Verification Post-run verification is mandatory for non-trivial pdf-lab jobs. `pdf-lab` is a substantial runtime self-improvement skill. After any job that creates extraction, convergence, or write-back artifacts, run: ```bash ./run.sh verify --job-dir <job-dir> ``` The verifier writes `<job-dir>/verify-receipt.json` and exits non-zero on missing or inconsistent artifacts. When verification fails, create a maintainer packet with: ```bash ./run.sh file-maintainer-ticket --job-dir <job-dir> ``` Runtime workers must not patch or commit `agent-skills` from inside a failed pdf-lab job; maintainer escalation is documented in `references/maintainer-escalation.md`. ## File Structure ``` pdf-lab/ SKILL.md # This file run.sh # Shell entry point pdf_lab.py # Typer CLI entry point memory_integration.py # Memory + Taxonomy hooks pyproject.toml # Dependencies sanity.sh # Local behavioral sanity gate ui/ # Standalone Vite PDF Lab UX and local API bridge scripts/ # Runtime verification and compliance helpers references/ # Maintainer escalation and long-form contracts agents/pdf-lab/AGENTS.md # Worker post-run rules lib/ # Core libraries (delta, tuner, writer, etc.) docs/ # Additional documentation ``` ## Outputs - Code changes written to `src/extractor/pipeline/steps/` - `/memory` entries for future recall (synthetic creation, convergence, reverts) - Git commits with persona attribution trailers - JSON report of convergence results - `verify-receipt.json` for deterministic runtime verification - `maintainer-ticket.json` when a failed job needs skill-maintainer repair