PrimeIntellect-ai

Run and analyze evaluations for verifiers environments using prime eval. Use when asked to smoke-test environments, run benchmark sweeps, resume interrupted evaluations, compare models, inspect sample-level outputs, or produce evaluation summaries suitable for deciding next steps.

Review verifiers environments for correctness, robustness, and ecosystem compatibility. Use when asked for environment code review, quality audit, migration validation, or release readiness checks for local environments or environments pulled from the Hub.

Create or migrate verifiers environments for the Prime Lab ecosystem. Use when asked to build a new environment from scratch, port an eval or benchmark from papers or other libraries, start from an environment on the Hub, or convert existing tasks into a package that exposes load_environment and installs cleanly with prime env install.

Discover and inspect verifiers environments through the Prime ecosystem. Use when asked to find environments on the Hub, compare options, inspect metadata, check action status, pull local copies for inspection, or choose environment starting points before evaluation, training, or migration work.

Train models with verifiers environments using hosted RL or prime-rl. Use when asked to configure RL runs, tune key hyperparameters, diagnose instability, set up difficulty filtering, or create practical train and eval loops for new environments.

Optimize environment system prompts with GEPA through prime gepa run. Use when asked to improve prompt performance without gradient training, compare baseline versus optimized prompts, run GEPA from CLI or TOML configs, or interpret GEPA outputs before deployment.

Run interactive brainstorming across verifiers environments, evaluations, GEPA, and RL training. Use when the user wants ideation, literature scanning, concept teaching, roadmap planning, or research program design grounded in local CLI sources, verifiers, and RL trainer code.

Audit and optimize verifiers environments for async performance. Use when asked to profile, speed up, or review an environment for concurrency bottlenecks, event loop blocking, or scaling issues under high rollout counts.

Install and run a verifiers environment — smoke testing during development, full benchmark evals, and pushing results to the Prime platform. Covers the v1 `eval` CLI (the `*_v1` tasksets) and the legacy v0 `vf-eval` CLI. Independent sections; use whichever the request needs. Use while developing/iterating on an environment, when running/evaluating/benchmarking one, or when the user mentions eval, vf-eval, tasksets, eval runs, smoke tests, or pushing eval results.

Fetch a URL and return the full parsed text. Handles HTML and PDF. Returns the entire parsed page; no truncation. Errors come back as plain strings.

Search Google via the Serper API. Takes a single query. Returns titles, URLs, snippets, and knowledge-graph data.

Push all local environments to the Prime Intellect Environments Hub. Use when environments are out of sync and need to be published.

Diagnose and repair-or-remove broken tasks under environments/general_agent/tasks. Use after `general-agent validate` reports [FAIL] entries (post-synthesis, after a refactor, or on a periodic sweep).

Synthesize a new general-agent task family from a seed task, evolving through difficulty tiers with empirical pass-rate gating. Use when asked to create new tasks or grow the task set.

Replace a unique string in a file. old_str must appear exactly once in the file.

Compare local env versions against the Prime Intellect Environments Hub. Use when checking which environments need to be pushed, or to audit sync status.

Monitor an ongoing prime-rl training run — find the output directory, tail logs, check key metrics, inspect SLURM jobs, and restart safely. Use when asked to check on a run, debug training, or investigate performance.

Launch and monitor prime-rl training runs. Use when starting, supervising, or debugging an RL/SFT run. Routes to `start-run` (entrypoints + how to launch) and `monitor-run` (logs, metrics, check-ins).

How to launch prime-rl training runs — the `rl`, `sft`, and `inference` entrypoints, their config classes, and single-node/SLURM/dry-run modes. Use when starting a run or picking the right entrypoint.

How the prime-rl config system works — TOML files, CLI overrides, composition, and special patterns. Use when creating configs, debugging config errors, or overriding values via CLI.

How to install prime-rl and its optional dependencies. Use when setting up the project, installing extras like deep-gemm for FP8 models, or troubleshooting dependency issues.

How to prepare and publish GitHub releases for prime-rl. Use when drafting release notes, tagging versions, or publishing releases.