// Use when validating rank_llm artifacts after rerank, retrieve-cache, or related CLI workflows. Checks JSONL integrity, TREC formatting, candidate shape, invocation-history structure, and duplicate query identifiers. Wraps rank-llm view plus custom assertions.
Set up a rank_llm development environment. Use when someone is onboarding, setting up a fresh clone, choosing extras such as cloud, api, local, or pyserini, or troubleshooting whether the packaged rank-llm CLI is ready.
Use when working with the rank-llm CLI: rerank, evaluate, analyze, retrieve-cache, serve, validate, prompt, view, describe, schema, or doctor. Covers entry points, common flags, JSONL and TREC artifacts, and end-to-end retrieval plus reranking workflows.
Use when analyzing rank_llm evaluation outputs across runs or models. Covers aggregated trec_eval JSONL files, response-analysis metrics, retrieval-cache handoff files, and side-by-side comparison of stored evaluation artifacts.
| name | rank-llm-verify |
| description | Use when validating rank_llm artifacts after rerank, retrieve-cache, or related CLI workflows. Checks JSONL integrity, TREC formatting, candidate shape, invocation-history structure, and duplicate query identifiers. Wraps rank-llm view plus custom assertions. |
Validates stored RankLLM artifacts for correctness and structural consistency.
rank-llm rerankrank-llm retrieve-cacheevaluate or analyzequery and candidatesdoc or textquery and candidatesdocid, score, and docqid values are presentinvocations_history listRun the verification script:
bash .claude/skills/rank-llm-verify/scripts/verify.sh <artifact-path> [artifact-type]
Supported artifact types:
request-inputrerank-outputinvocations-historytrec-outputIf no artifact type is provided, the script attempts to auto-detect it with rank-llm view.
See scripts/verify.sh for the runnable verification wrapper.
rank-llm validate rerank checks input contracts before execution. The verify script checks stored output artifacts after execution.rank-llm view only detects supported artifact families. If a file is not one of those shapes, pass the artifact type manually or inspect the file directly.evaluate and analyze for score and response diagnostics.