benchflow-ai

SkillsBench contribution workflow. Use when: (1) Creating benchmark tasks, (2) Understanding repo structure, (3) Preparing PRs for task submission.

SkillsBench task authoring — walk a contributor from idea to submission-ready task following CONTRIBUTING.md and the task-implementation rubric. Use when the user wants to create a new SkillsBench task, scaffold a task from an existing workflow (notebook, Excel workbook, document, dataset), convert a prompt or a benchmark item into a SkillsBench task, write skills for a task, or prepare a SkillsBench PR. Pairs with `task-review` (run that as a self-check before submitting).

SkillsBench task PR review — classifies the task track (standard / research / multimodal), runs static policy checks against the track-specific rubric, benchmarks the task across oracle plus Claude and Codex (with and without skills), audits trajectories for cheating and skill invocation, and produces a `pr-N-task-timestamp-run.txt` review report alongside a `prN.zip` bundle of trajectories. Use when reviewing a SkillsBench task PR (by number, branch, or local task path), when the user asks to review a task, run benchmarks on a PR, audit a submission, classify a task as research or multimodal track, or prepare a comment to post on a SkillsBench PR.

Methodology for clause-by-clause review of a contract against a structured deviation policy ("playbook"). Covers how to walk a playbook, locate the matching provision in the contract, apply rule types (max-value, must-be-present, must-be-absent, acceptable-set, must-have-feature), classify the result (ok / risk / reject), choose the prescribed action, and ground each finding in a verbatim excerpt. Use whenever reviewing any contract — NDA, MSA, vendor DD questionnaire, lease, DPA — against a structured rules-based playbook.

Reference for the standard clauses found in commercial non-disclosure agreements (mutual and one-way) — what each clause does, the surface forms it appears in, and how to recognise it in unfamiliar drafting. Use when reviewing, comparing, or extracting provisions from any confidentiality / NDA / mutual NDA / standstill-and-confidentiality agreement.

Read Microsoft Excel (.xlsx) files robustly with `openpyxl` (or `pandas`). Covers multi-sheet workbooks, header rows, empty cells, merged cells, comma-separated list cells, and converting a sheet to a list-of-dicts the rest of your code can consume. Use when a task input or reference document is an `.xlsx` file rather than JSON/CSV.

Build unified multi-level category taxonomy from hierarchical product category paths from any e-commerce companies using embedding-based recursive clustering with intelligent category naming via weighted word frequency analysis.

Build deterministic, verifiable data visualizations with D3.js (v6). Generate standalone HTML/SVG (and optional PNG) from local data files without external network dependencies. Use when tasks require charts, plots, axes/scales, legends, tooltips, or data-driven SVG output.

Review Benchflow or SkillsBench task-run trajectories and integration-test Benchflow code changes. Use this skill whenever the user asks to audit traj health, failed or timed-out runs, healthy pass/fail/timeout status, no-skill leakage, skill loading, reward hacking, verifier isolation, metadata completeness, token usage, timing, Daytona-vs-Docker parity, path/root handling, coverage gaps, Docker/Daytona failures, or release-readiness of benchmark data.

Run agent benchmarks, create tasks, analyze results, and manage agents using BenchFlow. Use when asked to benchmark an AI coding agent, run a benchmark suite, create tasks, view trajectories, or compare agent performance.

Run agent benchmarks, create tasks, analyze results, and manage agents using BenchFlow. Use when asked to benchmark an AI coding agent, run a benchmark suite, create tasks, view trajectories, or compare agent performance.

Adopt, convert, verify, and publish upstream benchmarks as BenchFlow benchmarks. Use when asked to port a benchmark, create a BenchFlow adapter, write or review benchmarks/<name>/benchflow.py, run parity, route a benchmark through L1/L2/L3, or decide whether a benchmark should run natively, be translated, or run as-is.

Review benchflow documentation for drift, staleness, duplication, and alignment

Manage a BenchFlow release end-to-end — choose the channel (public vs internal-preview vs release-candidate), stage and publish it safely, and run the pre-tag audit gates. Use when the user says "release", "cut a release", "release candidate", "rc", "internal tag", "publish to PyPI", "prerelease", "tag v0.x", or asks how releases/versions/channels work here. Complements `launch-prep` (which mechanically prepares a public cut); this skill owns the channel decision, the RC-without-merging-to-main flow, and the audit gates.

Periodic audit of src/benchflow/ shape (names, boundaries, file sizes, stability mix) — propose structural reorg opportunities, don't execute

Pre-push branch reviewer — runs lint+typecheck+tests, then fans /code-cleanup, /test-review, /docs-review at the branch diff, merges findings by file

Adapt an agent to run on BenchFlow (ACP) and verify it behaves identically inside the eval harness vs standalone — closing the eval↔prod gap. Use when adding a new agent to this repo or checking an existing one for eval/prod behavior parity.