with one click
yaml-shadow-expand
// Incrementally expand the Panache YAML shadow parser by triaging yaml-test-suite fixtures one (or a few) at a time and allowlisting cases as parser/projection support grows.
// Incrementally expand the Panache YAML shadow parser by triaging yaml-test-suite fixtures one (or a few) at a time and allowlisting cases as parser/projection support grows.
[HINT] Download the complete skill directory including SKILL.md and all related files
| name | yaml-shadow-expand |
| description | Incrementally expand the Panache YAML shadow parser by triaging yaml-test-suite fixtures one (or a few) at a time and allowlisting cases as parser/projection support grows. |
Use this skill when asked to extend YAML shadow parser coverage, add a new yaml-test-suite case to the allowlist, or pick "the next best case" to work on.
crates/panache-parser/src/parser/yaml/ and the event-parity harness in
crates/panache-parser/tests/yaml.rs.yaml_parser dependency, not a forever-shadow. Each session grows
spec coverage toward that cutover. Don't promise near-term replacement
and don't block incremental wins on the eventual rewrite — but don't
read this as "we're keeping the current lexer indefinitely" either.The streaming scanner rewrite has landed and the legacy line-based lexer is gone. The live tree-building path is now:
parser.rs::parse_yaml_report — slim orchestrator. Calls the
validator, then builds the tree from parser_v2.validator.rs::validate_yaml — v2-aware structural validator.
Each check_* function is one cluster of error contracts
(directive ordering, trailing content, unterminated flow, flow
comma anomalies, multi-line quoted indent, block indent anomalies,
block-scalar header, doc-level/value-level mixed scalar+map, flow
continuation indent, invalid double-quoted escapes, etc.). Runs
the scanner internally for token-level checks.parser_v2.rs::parse_v2 — consumes the streaming scanner.rs
and emits the rowan green tree.scanner.rs is the streaming, char-by-char scanner modeled on
libyaml / PyYAML / snakeyaml: position-tracked, indent-stack driven,
simple-key-table based, with a token queue and lookahead. Trivia
(whitespace, comments, newlines) is interleaved in the queue rather
than dropped, so the CST stays lossless. Key/value pairing,
multi-line scalars, and explicit-key (? / :) entries unify under
one mechanism.
Residual cutover work (deferred):
events.rs::collect_doc_scalar_text_with_newlines,
collect_value_scalar_text_with_newlines,
quoted_val_event_multi_line — projection still re-stitches
multi-line scalars.!, &, *) is not yet implemented in
the scanner; these characters fall through to plain scalar at the
start of a token. The validator-driven directive-ordering pass
inherits the scanner's view, so cases like
!foo "bar"\n%TAG ...\n---\n are currently parsed as one big plain
scalar followed by a doc-start marker. Adding tag dispatch is the
next large scanner feature; once it lands the
parse_yaml_report_detects_directive_after_content test should be
switched back to the tag-shaped input.The concrete plan and design decisions for the rewrite — including
trivia model, token enum lifetime, scalar cooking, diagnostic channel,
and the step-by-step migration sequence — live in scanner-rewrite.md
alongside this file. Consult it for context on residual work and for
the rationale behind the validator-driven cutover.
crates/panache-parser/src/parser/yaml/scanner.rs — streaming
char-by-char scanner with simple-key table (~2,851 LOC). Emits the
token stream consumed by parser_v2.crates/panache-parser/src/parser/yaml/parser_v2.rs — consumes the
scanner and builds the rowan green tree (~1,134 LOC). parse_v2
is the entry point.crates/panache-parser/src/parser/yaml/validator.rs — v2-aware
structural-diagnostic validator. validate_yaml(input) composes
per-cluster check_* functions in priority order. Add new
diagnostic clusters here as check_* functions and wire them into
validate_yaml.crates/panache-parser/src/parser/yaml/parser.rs — slim
orchestrator. parse_yaml_report runs validate_yaml, then
parser_v2::parse_v2, and wraps the v2 stream in the
DOCUMENT > YAML_METADATA_CONTENT > YAML_STREAM envelope. No
emitter logic lives here — work in parser_v2.rs or scanner.rs
for tree-shape changes.crates/panache-parser/src/parser/yaml/events.rs — event projection
(project_events plus project_* helpers). Walks the CST and
produces a yaml-test-suite event stream. The *_with_newlines /
*_multi_line re-stitching helpers are technical debt awaiting
unification once the scanner emits styled scalars as single tokens.crates/panache-parser/src/parser/yaml/model.rs — YamlDiagnostic,
diagnostic_codes, YamlParseReport, shadow report shape.crates/panache-parser/tests/yaml.rs — fixture-driven tests, including:
yaml_allowlist_cases_snapshot — diagnostic/tree snapshot per caseyaml_allowlist_cases_cst_snapshot — full CST snapshot per caseyaml_allowlist_losslessness_raw_input — byte-exact round-tripyaml_allowlist_projected_event_parity — event stream vs
fixture test.eventyaml_suite_generate_triage_report (ignored) — regenerates
tests/yaml/triage.json bucketing every fixturecrates/panache-parser/tests/yaml/allowlist.txt — small, intentionally curated
list of case IDs. One case per addition, with a short # comment explaining
what the case exercises.crates/panache-parser/tests/yaml/triage.json — derived; do not hand-edit.crates/panache-parser/tests/fixtures/yaml-test-suite/ — vendored fixtures,
refreshed via scripts/update-yaml-test-suite-fixtures.sh.triage.json splits every fixture into four buckets. Understand which bucket a
case is in before touching it:
passes_now — tree parses AND projected events match test.event. Safe to
allowlist if not already listed.error_contract_ok — case has an error file and we correctly reject it
with at least one diagnostic. Do not allowlist unless the test harness
explicitly models the expected error contract.fails_needs_error_path — case has an error file but we currently parse
it successfully (no diagnostic). Needs parser work to detect the error.fails_needs_feature — no error file. Two sub-patterns:
tree: true, event_parity: false — parses OK, projection fails. Usually
low-effort: fix cst_yaml_projected_events / helpers in tests/yaml.rs.tree: false — parser rejects. Usually needs lexer/parser work.Regenerate triage if stale:
cargo test -p panache-parser --test yaml yaml_suite_generate_triage_report -- --ignored
Then inspect counts:
grep -E '"passes_now_count"|"fails_needs_feature_count"|"error_contract_ok_count"|"fails_needs_error_path_count"' \
crates/panache-parser/tests/yaml/triage.json
Pick a case — prefer highest-leverage, lowest-risk:
fails_needs_feature entries where tree: true — these only
need projection fixes.in.yaml and test.event for a few candidates. Group cases that
share a root cause so one fix unlocks several.error file without modeling the
error contract explicitly.Probe the gap if not obvious. A throwaway #[ignore] test in
tests/yaml.rs printing parse_yaml_tree(input) and
project_events(input) is cheap and informative. Remove the probe before
finishing.
Classify the fix before coding:
parser/yaml/events.rs helpers
(project_document, project_block_map_entries,
project_block_sequence_items, project_flow_map_entries,
scalar_document_value).parser/yaml/parser_v2.rs. The v2 emitter is keyed on the
scanner's token kinds (BlockMappingStart / Key / Value /
BlockEntry / BlockEnd / flow indicators); trivia is consumed
inline. Do not edit parser.rs — it's a slim orchestrator
and contains no emitter logic.parser/yaml/scanner.rs. Consider indent/flow/block-scalar/
simple-key-table state interactions.check_* function in parser/yaml/validator.rs and wire it
into validate_yaml. Each check is one cluster of error
contracts. New diagnostic codes go in
model.rs::diagnostic_codes first.Scanner::diagnostics
from parser/yaml/scanner.rs (use push_diagnostic), or, if
it requires CST inspection, add a check_* cluster in
validator.rs.Apply the smallest focused change. Keep changes parser-crate scoped, CST-lossless, and don't regress already-allowlisted cases.
Add the case(s) to allowlist.txt with a one-line # comment capturing
the pattern (not the case ID — the shape, e.g. "Block map with inline
flow-map values"). One commit/session can add several if they share a root
cause, but annotate each.
Run the parity tests:
cargo test -p panache-parser --test yaml
Expect snapshot tests to fail the first time with .snap.new files. Review
each new snapshot before accepting:
tests/snapshots/yaml__yaml_suite_<ID>.snap.new — summarytests/snapshots/yaml__yaml_cst_suite_<ID>.snap.new — CST tree
Accept by renaming (mv ...snap.new ...snap) only after confirming the CST
shape matches the fixture semantics. Note: insta stops on the first
snapshot failure, so you may need to iterate (accept, re-run, accept…).Check for unlocked cases. A single projection or parser fix can flip
several cases to passing. After regenerating triage, diff passes_now vs
the allowlist and allowlist the cleanly-unlocked ones with their own
rationale comments.
Validate:
cargo test -p panache-parser --test yamlcargo clippy -p panache-parser --all-targets -- -D warningscargo fmt -p panache-parser -- --checktriage.json a final time so it reflects the new state.allowlist.txt intentionally small. One case per addition, with
an explanatory comment.triage.json — it is derived output.When done, report:
Triage counts before and after (passes_now, fails_needs_feature,
error_contract_ok, fails_needs_error_path).
Cases allowlisted this session and the shared pattern behind them.
Files changed and the root cause addressed.
Any cases unlocked but not yet allowlisted (candidates for follow-up).
Suggested next targets grouped by shared root cause.
Session continuation recommendation — close with one of:
Don't default to one answer; pick based on what the next target needs.