// Use when validating ragnarok generate outputs — checks TREC format compliance, citation integrity (valid indices, non-empty references), response length bounds, and JSONL structure. Wraps `ragnarok validate` plus custom assertions. Use after running generate or convert to verify output correctness.
Set up a ragnarok development environment — checks Python 3.11+, installs via uv or pip with cloud extras, and verifies with doctor. Use when someone is onboarding, setting up a fresh clone, or troubleshooting their environment.
Use when working with ragnarok dataset-backed generation — retrieval configuration, pyserini dependency setup, topk multi-stage pipelines, dataset naming conventions, and retrieval method selection. Use when setting up or debugging dataset-backed workflows.
Use when working with ragnarok CLI commands (generate, validate, convert), prompt modes (chatqa, ragnarok_v4, etc.), dataset-backed or file-backed generation, TREC RAG output formats, or introspection (doctor, describe, schema). Covers all entry points, flags, and generation workflows.
| name | ragnarok-verify |
| description | Use when validating ragnarok generate outputs — checks TREC format compliance, citation integrity (valid indices, non-empty references), response length bounds, and JSONL structure. Wraps `ragnarok validate` plus custom assertions. Use after running generate or convert to verify output correctness. |
Validates ragnarok generation outputs for TREC compliance, citation integrity, and structural correctness.
ragnarok generate — verify answer output integrityragnarok convert trec25-format — verify TREC submission formattopic_id, topic, references, response_length, and answer arrayanswer is an array of objects with text and citationsreferences array bounds)answer arraystopic_id valuesresponse_length matches actual word count (within tolerance)referencesteam_id, run_id, narrative_id, type)Run the verification script:
bash .claude/skills/ragnarok-verify/scripts/verify.sh <artifact-path> [--trec25]
Or use the built-in validators:
ragnarok validate generate --input-file requests.jsonl
ragnarok validate rag25-output --input answers.jsonl --topics topics.jsonl
See scripts/verify.sh for the runnable verification wrapper.
ragnarok validate generate checks input requests. The verify script checks output answers.ragnarok validate rag25-output is the built-in TREC 2025 output validator with --fix-length, --fix-citations, --apply-fixes for automatic repair.references array.response_length is word count, not character count.--prompt-mode ragnarok_v4_no_cite and chatqa produce answers without citations — citation checks should be skipped for these modes.