pdf-lab

name	pdf-lab
description	Self-improving PDF extraction convergence loop. Diagnoses extraction failures by computing the delta between S00 estimates and actual extraction, reproduces issues on synthetic PDFs, discovers optimal parameters, and writes fixes back to the extractor pipeline code permanently.
allowed-tools	["Bash","Read","Write","Edit","Glob","Grep"]
triggers	["pdf lab","tune pdf extraction","improve pdf extraction","converge extraction parameters","fix extraction delta","self improve extractor","write back pipeline fix"]
metadata	{"short-description":"Self-improving PDF extraction with convergence + write-back","version":"1.0.0"}
runtime_self_improvement	substantial
provides	["pdf-extraction"]
composes	["memory","scillm","task-monitor"]
complies	["best-practices-skills","best-practices-python","best-practices-arangodb"]
taxonomy	["precision","validation","resilience"]

pdf-lab

/pdf-lab is a convergence loop that diagnoses PDF extraction failures, reproduces them on synthetic PDFs, discovers optimal parameters, and writes those fixes back to the extractor pipeline code permanently.

Why This Exists

There is no absolute ground truth for real PDFs. The system works with deltas between S00's estimate and actual extraction results. When S00 predicts 40 sections but extraction finds 15, something is wrong. /pdf-lab figures out what, fixes it, and makes the fix permanent.

Quick Start

cd ${HOME}/workspace/experiments/pi-mono/.pi/skills/pdf-lab

# Main: diagnose, reproduce, converge, and write fix back
./run.sh tune /path/to/real.pdf \
  --review-json /path/to/review_result.json \
  --debug-json /path/to/debug_patterns.json \
  --converge --write-back --json

# Dry run: find the fix but don't write it
./run.sh tune /path/to/real.pdf \
  --review-json ... --debug-json ... \
  --converge --dry-run --json

# Quick diagnosis only (compute delta, no tuning)
./run.sh diagnose /path/to/real.pdf \
  --profile-json /path/to/profile.json \
  --structural-json /path/to/structural.json

# Generate synthetic reproduction PDF only
./run.sh synthetic \
  --patterns '["multi_column","split_tables"]' \
  --output /tmp/repro.pdf

# Show recent tuning results
./run.sh status

# List all pdf-lab code changes
./run.sh history

# Rollback a specific fix
./run.sh rollback --sha abc123

# Deterministic post-run verification for substantial-skill jobs
./run.sh verify --job-dir /tmp/pdf-lab-job

# Build a maintainer escalation packet when verify fails
./run.sh file-maintainer-ticket --job-dir /tmp/pdf-lab-job

# Standalone PDF Lab UX
cd ${HOME}/workspace/experiments/agent-skills/skills/pdf-lab/ui
npm install
npm run dev:all

If the workstation has exhausted file watchers, use the no-watch preview path:

npm run build
npm run preview:all

The standalone UI runs at http://127.0.0.1:3012/#pdf-lab. In dev:all, the Vite app runs on port 3012 and the local API bridge runs on port 3013. In preview:all, the API bridge serves the built UI and API together on port 3012. The bridge serves real artifacts from PDF_LAB_PUBLIC_ROOT and PDF_LAB_ARTIFACTS_ROOT; if an artifact or runtime bridge is missing, endpoints fail closed with an explicit JSON error instead of returning mock operational state.

How Fixes Get Written Back

The pipeline has a tiered configuration system. /pdf-lab writes to the appropriate tier:

Tier	Target	Example
1	Env var defaults in step files	CAMELOT_LINE_SCALE_DEFAULT 15 -> 40
2	Heuristic thresholds (code constants)	LARGE_FONT_THRESHOLD 11.0 -> 9.5
3	Pattern rules (regex, filters)	New citation pattern in S04
4	Preset YAML	line_scale: 80 in arxiv twin_config.yml
5	Calibration records (ArangoDB)	Learned pattern in learned_patterns
6	/memory (runtime recall)	Winning params stored for instant recall

Persona Attribution

Every code change is traceable to the persona who flagged the issue via git commit trailers (Reviewed-By, Persona-Role, Issue-Codes).

Integration

Called by inline_review_loop.py when a persona review score is below threshold. Falls back to heuristic adaptive params if convergence fails.

Memory + Taxonomy Integration

The skill integrates with /memory through memory/run.sh subcommands in memory_integration.py; it does not import ArangoDB clients or the memory Python package directly.

• Pre-hook (recall_prior_convergence): Before tuning, recalls prior convergence results for the same PDF type or URL. Enables the tuner to skip failed strategies and start from previously winning parameters.
• Post-hook (learn_convergence): After tuning completes, stores the convergence outcome (strategy, iterations, final score, improvements, write-back results) to memory with taxonomy bridge tags for cross-skill recall.
• Bridge keywords: Precision, Resilience, Fragility, Corruption, Loyalty, Stealth (tuned to PDF extraction domain).
• Tags: ["pdf_lab", "convergence"] + bridges

Gracefully degrades if /memory is unavailable.

Runtime Verification

Post-run verification is mandatory for non-trivial pdf-lab jobs.

pdf-lab is a substantial runtime self-improvement skill. After any job that creates extraction, convergence, or write-back artifacts, run:

./run.sh verify --job-dir <job-dir>

The verifier writes <job-dir>/verify-receipt.json and exits non-zero on missing or inconsistent artifacts. When verification fails, create a maintainer packet with:

./run.sh file-maintainer-ticket --job-dir <job-dir>

Runtime workers must not patch or commit agent-skills from inside a failed pdf-lab job; maintainer escalation is documented in references/maintainer-escalation.md.

File Structure

pdf-lab/
  SKILL.md                   # This file
  run.sh                     # Shell entry point
  pdf_lab.py                 # Typer CLI entry point
  memory_integration.py      # Memory + Taxonomy hooks
  pyproject.toml             # Dependencies
  sanity.sh                  # Local behavioral sanity gate
  ui/                        # Standalone Vite PDF Lab UX and local API bridge
  scripts/                   # Runtime verification and compliance helpers
  references/                # Maintainer escalation and long-form contracts
  agents/pdf-lab/AGENTS.md   # Worker post-run rules
  lib/                       # Core libraries (delta, tuner, writer, etc.)
  docs/                      # Additional documentation

Outputs

• Code changes written to src/extractor/pipeline/steps/
• /memory entries for future recall (synthetic creation, convergence, reverts)
• Git commits with persona attribution trailers
• JSON report of convergence results
• verify-receipt.json for deterministic runtime verification
• maintainer-ticket.json when a failed job needs skill-maintainer repair