castorini

Install and run Anserini quickly by downloading the latest fatjar from the official Maven Central repo instead of building from source. Use when users want fast setup, smoke tests, or command execution without Maven project compilation.

Use when onboarding to nuggetizer, ragnarok, rank_llm, or umbrela and you need development environment setup for one repo or several repos at once, including clone-if-needed, uv or pip installation paths, shared virtualenv reuse, and smoke tests.

Use when serving Anserini retrieval together with any subset of rank_llm, ragnarok, nuggetizer, or umbrela over HTTP, especially for local port planning, direct request payload compatibility, curl or jq pipelines, or sequencing retrieval, reranking, generation, nugget creation, nugget assignment, and passage judging from an Anserini fatjar RestServer.

Use when building, debugging, or reviewing CLI commands across nuggetizer, ragnarok, rank_llm, or umbrela and you need the shared castorini.cli.v1 envelope, common introspection commands, artifact shapes, or cross-repo CLI consistency rules.

Use when coordinating an end-to-end Castorini pipeline across rank_llm, ragnarok, nuggetizer, and umbrela, especially for stage handoffs, JSONL compatibility, retrieval-to-answer evaluation flow, or reproducing a multi-stage experiment.

Use when publishing nuggetizer, ragnarok, rank_llm, or umbrela to PyPI or TestPyPI and you need the release sequence for version bumps, build checks, twine validation, TestPyPI dry-runs, or final production publishing.

Install and verify Anserini quickly by downloading the published fatjar from Maven Central instead of cloning or building the source repository. Use when users want fast setup, smoke tests, or CLI examples from a released Anserini jar.

Run Anserini command-line and REST workflows from either a built fatjar or an Anserini source checkout. Use for PrebuiltIndexRegistry, TopicsRegistry, ad hoc search, interactive search, output formats, and RestServer examples.

Reproduce experimental results with Anserini. Use when Codex needs to run or explain Anserini reproduction workflows for published or reported results, including reproductions with prebuilt indexes, reproductions from raw document collections, reproduction YAMLs, run generation, evaluation, and metric verification.

Set up and verify Anserini source-development environments. Use for JDK 21, Maven 3.9+, submodules, Anserini build scripts, smoke tests, and Java/Maven troubleshooting in castorini/anserini.

Set up a rank_llm development environment. Use when someone is onboarding, setting up a fresh clone, choosing extras such as cloud, api, local, or pyserini, or troubleshooting whether the packaged rank-llm CLI is ready.

Use when working with the rank-llm CLI: rerank, evaluate, analyze, retrieve-cache, serve, validate, prompt, view, describe, schema, or doctor. Covers entry points, common flags, JSONL and TREC artifacts, and end-to-end retrieval plus reranking workflows.

Use when analyzing rank_llm evaluation outputs across runs or models. Covers aggregated trec_eval JSONL files, response-analysis metrics, retrieval-cache handoff files, and side-by-side comparison of stored evaluation artifacts.

Use when validating rank_llm artifacts after rerank, retrieve-cache, or related CLI workflows. Checks JSONL integrity, TREC formatting, candidate shape, invocation-history structure, and duplicate query identifiers. Wraps rank-llm view plus custom assertions.

Set up a ragnarok development environment — checks Python 3.11+, installs via uv or pip with cloud extras, and verifies with doctor. Use when someone is onboarding, setting up a fresh clone, or troubleshooting their environment.

Use when working with ragnarok dataset-backed generation — retrieval configuration, pyserini dependency setup, topk multi-stage pipelines, dataset naming conventions, and retrieval method selection. Use when setting up or debugging dataset-backed workflows.

Use when validating ragnarok generate outputs — checks TREC format compliance, citation integrity (valid indices, non-empty references), response length bounds, and JSONL structure. Wraps `ragnarok validate` plus custom assertions. Use after running generate or convert to verify output correctness.

Use when working with ragnarok CLI commands (generate, validate, convert), prompt modes (chatqa, ragnarok_v4, etc.), dataset-backed or file-backed generation, TREC RAG output formats, or introspection (doctor, describe, schema). Covers all entry points, flags, and generation workflows.

Use when analyzing umbrela evaluation results — comparing nDCG@10 scores across backends, interpreting confusion matrices, computing kappa agreement, and comparing modified qrels against human judgments. Use after running evaluate to interpret results.

Set up an umbrela development environment — checks Python 3.11+, installs via uv or pip with cloud extras, and verifies with doctor. Use when someone is onboarding, setting up a fresh clone, or troubleshooting their environment.

Use when validating umbrela judge outputs — checks label range (0–3), qid/docid completeness, result_status consistency, backend metadata, and JSONL integrity. Wraps `umbrela validate` plus custom assertions. Use after running judge or evaluate to verify output correctness.

Use when working with umbrela CLI commands (judge, evaluate), backend selection (gpt, gemini, hf, os, ensemble), qrel handling (dl19-passage, dl20-passage, etc.), relevance labels (0–3), or introspection (doctor, describe, schema, validate). Covers all entry points, flags, and evaluation workflows.

Use when analyzing nuggetizer metrics outputs — comparing runs, computing agreement between models, reporting token usage and latency from trace fields, and building side-by-side evaluation tables. Use after running metrics to interpret and compare results.

Set up a nuggetizer development environment — checks Python 3.11+, installs via uv or pip, and verifies with doctor. Use when someone is onboarding, setting up a fresh clone, or troubleshooting their environment.

Use when validating nuggetizer batch outputs — checks JSONL integrity, duplicate qids, missing nugget fields, assignment label validity, and metrics consistency. Wraps `nuggetizer validate` plus custom assertions. Use after running create, assign, or metrics to verify output correctness.

Use when working with nuggetizer CLI commands (create, assign, metrics), prompt templates, JSONL formats, or introspection (doctor, describe, schema, validate). Covers all entry points, flags, and the YAML prompt system.

Use this skill when the user wants to clone Pyserini from GitHub for local development, install Pyserini from source with pip editable mode using `pip install -e .`, create or reuse a development environment, install optional development extras, or debug source-checkout installation issues involving Python, Java, Git, pip, PyPI, or editable installs.

Use this skill when the user wants to install Pyserini with uv, create a uv-managed environment for Pyserini, add Pyserini to an existing uv project, install optional Pyserini extras, or debug uv/Pyserini installation issues involving Python, Java, PyPI, or dependency resolution.