fix-parser-gap

name

fix-parser-gap

description

Fix a parser gap identified in the spec coverage audit. Use when user says 'fix a gap', 'implement X parsing', 'improve parser coverage', 'close a gap', 'next P1', or references a specific ECMA-376 spec section like '§17.9'. Also use when the user asks to improve DOCX/XLSX/PPTX parsing quality, add support for a missing Office feature, or mentions numbering, styles, hyperlinks, formatting, fields, merged cells, equations, or speaker notes in the context of parser improvement.

Fix Parser Gap

Implements missing ECMA-376 Office parsing features using a TDD red-to-green workflow: write a failing test first, read the spec, implement in AILANG, verify the test goes green, then re-run benchmarks.

Workflow: Red -> Green -> Benchmark

Step 1: Select the Gap

If the user specifies a feature, use that. Otherwise pick the highest-priority unresolved gap from the audit:

# Show current gap scores (0% = complete gap, 100% = fully handled)
uv run benchmarks/office/eval_gaps.py --verbose

Consult resources/reference.md for the priority table and spec page numbers.

Step 2: RED — Write or Update the Failing Test

This step is critical. No implementation without a failing test first.

Two places tests live:

1. benchmarks/office/eval_gaps.py — Gap analysis checks. Each check is a function like check_custom_heading_styles(output) that returns a score from 0.0-1.0. If a check doesn't exist for the gap, add one. If it exists but is too lenient, tighten it.

2. benchmarks/officedocbench/ground_truth/challenge_*.json — Hand-verified ground truth for challenge files. These files have "verified": true so annotate.py won't overwrite them. If the ground truth doesn't reflect the expected correct behavior, fix it.

Also create or verify the challenge test file in data/test_files/challenge/ — these are generated by benchmarks/create_challenge_files.py using python-docx/openpyxl. If the existing challenge file doesn't test the feature well enough, update the generator.

Run the test to confirm it fails (RED):

uv run benchmarks/office/eval_gaps.py --verbose  # Check the specific gap shows 0%

Step 3: Read the Spec

Read the relevant ECMA-376 spec section for the feature being implemented:

# Spec PDF is at:
specs/ecma-376/part1/ECMA-376-1-5th-edition-december-2016-Part1.pdf

Use the page numbers from resources/reference.md. Focus on:

• The XML element structure (tag names, attributes, child elements)
• Where the element appears in the ZIP (which XML file)
• How relationships work (if the feature uses r:id references)

Also examine the actual challenge file XML to understand the real structure:

# Extract and inspect the XML from a test file
cd /tmp && mkdir -p inspect && cd inspect
unzip -o /path/to/data/test_files/challenge/challenge_*.docx
cat word/document.xml | xmllint --format - | head -100

Step 4: GREEN — Implement in AILANG

Modify the parser file (see reference.md for which file). Follow these principles:

• All changes must be AILANG code — no Python wrappers, no Go workarounds
• Prefix internal helpers with the module name (e.g., docxParserExtractNumDef) to avoid name collisions
• Use pure func for all deterministic parsing functions — this enables Z3 static verification
• Add ensures contracts where the function has provable bounds (e.g., ensures { result >= 0 && result <= 6 } for heading levels, ensures { listLength(result) == listLength(input) } for mappers). Z3 verifies these hold for ALL inputs.
• Add tests [...] inline test cases on pure functions (input/output pairs the compiler checks)
• Keep effects at the boundary — parsing XML is pure; only ZIP/FS reads need ! {FS}. The pure func core + effectful shell pattern lets Z3 verify the parsing logic statically.
• Read new ZIP entries using extractFileFromZip(zipData, "word/numbering.xml")
• Type-check after changes: ailang check docparse/
• Run contract verification: ./bin/docparse --prove (Z3 static verification)
• Run inline tests: ailang run --entry main --caps IO,FS,Env docparse/main.ail --test

After implementing, run the test to confirm it passes (GREEN):

# Parse the challenge file
./bin/docparse data/test_files/challenge/challenge_*.docx

# Check the output
cat docparse/data/challenge_*.docx.json | python3 -m json.tool | head -50

# Run gap analysis — the specific check should now score > 0%
uv run benchmarks/office/eval_gaps.py --verbose

Step 5: Update Golden Outputs

After the parser change, regenerate golden outputs for affected files:

# Regenerate golden outputs for the changed format
bash benchmarks/generate_golden.sh

# Re-run the full benchmark to see the composite score improve
uv run benchmarks/officedocbench/eval_officedocbench.py

Step 6: Update the Audit

Update design_docs/planned/v0_5_0/spec_coverage_audit.md:

• Move the feature from "NOT Handled" to "Handled Features"
• Update the coverage percentage
• Note the parser lines where it was implemented

Step 7: Run Full Verification

# Type-check all modules
ailang check docparse/

# Quick smoke test — must still pass
bash benchmarks/quick_check.sh

# Full gap analysis
uv run benchmarks/office/eval_gaps.py --verbose

# Full benchmark
uv run benchmarks/officedocbench/eval_officedocbench.py

# API tests (if touching shared code)
bash tests/test_serve_api.sh

Step 8: Commit

Commit message should reference the spec section and show the score improvement:

Implement §17.9 numbering definitions in DOCX parser

Gap score: 0% -> 100% (list detection)
Benchmark: 97.8% -> 98.X% composite

Scripts

scripts/run_gap_check.sh

Runs both gap analysis and OfficeDocBench benchmark in sequence.

.claude/skills/fix-parser-gap/scripts/run_gap_check.sh              # both
.claude/skills/fix-parser-gap/scripts/run_gap_check.sh --gaps-only   # gap analysis only
.claude/skills/fix-parser-gap/scripts/run_gap_check.sh --bench-only  # benchmark only

Resources

See resources/reference.md for:

• Gap priority table with spec page numbers
• Key file locations
• Challenge file expected behaviors
• AILANG parser patterns and conventions