list-bench

Sterne1

Forks0

Aktualisiert28. Juni 2026 um 21:16

Show the state of the benchmark — pinned projects, which of the six tasks each has, and how many evaluation reports exist per task. Use when the user says "list the bench", "what projects do we have", "show benchmark coverage", "what's been evaluated".

Installation

Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.

In Manus ausführen

Quelle

Agusx1211

Agusx1211/mi-primer-benchmarkito

GitHub-Repository öffnen Creator-Repositorys ansehen

Download

In Manus ausführen

Verwandte BerufeSOC

Basierend auf der SOC-Berufsklassifikation

SoftwareentwicklerInformatik- und Mathematikberufe·SOC 15-1252

SKILL.md

readonly

name	list-bench
description	Show the state of the benchmark — pinned projects, which of the six tasks each has, and how many evaluation reports exist per task. Use when the user says "list the bench", "what projects do we have", "show benchmark coverage", "what's been evaluated".
version	1.0.0

list-bench

Inventory the benchmark: projects, task coverage, and report counts.

Usage: /list-bench

Steps

Run the inventory helper:
```
lib/inventory.py
```
It prints each project (with its pinned short commit), the N/6 tasks authored, a per-task ✓/· grid, and how many reports exist per task.
Summarize for the user: which projects are fully authored (6/6), which tasks are missing, and which (project × task) cells still have no reports. Suggest the next action — /create-task for missing tasks or /evaluate-agent for un-evaluated ones.
If asked to aggregate scores (e.g. "average correctness per model"), read the relevant reports/*/metrics.json directly and roll up along run.model / run.project / run.difficulty / run.kind.

Mehr aus diesem Repository

gleiches Repository

create-task

Agusx1211/mi-primer-benchmarkito

Author a benchmark task (prompt + expected solution) for a pinned project. Use when the user says "create a task", "make a hard task on <project>", "add a scout/coder challenge", or asks to design an easy/medium/hard evaluation. Studies the repo, designs a difficulty-calibrated task, and writes task.md, solution.md, and task.json.

2026-06-281

evaluate-agent

Agusx1211/mi-primer-benchmarkito

Run a coding agent blind against a benchmark task, review its work, score it on every metric, and write a report. Also runs stateful, resumable multi-task campaigns ("fully eval <model>") that tick a durable checklist case-by-case in serial order. Use when the user says "evaluate <model> on <task>", "fully eval <model>", "run the benchmark", "score an agent", "test <model> against <project>", "resume the eval". Spins up a throwaway git worktree in /tmp/work, dispatches the agent (pi-as-mcp by default), waits serially, then produces reports/<timestamp>-<model>/ with report.md, metrics.json, frozen request+solution, and diff.zip.

2026-06-281

import-repo

Agusx1211/mi-primer-benchmarkito

Import and pin a codebase into the benchmark. Use when the user says "import repo", "add a project to the bench", "pin <repo>", or gives a git URL/path to benchmark against. Clones the repo at a frozen commit, writes its SUMMARY.md orientation doc and project.json, and registers it in manifest.json.

2026-06-281

name	list-bench
description	Show the state of the benchmark — pinned projects, which of the six tasks each has, and how many evaluation reports exist per task. Use when the user says "list the bench", "what projects do we have", "show benchmark coverage", "what's been evaluated".
version	1.0.0

list-bench

Inventory the benchmark: projects, task coverage, and report counts.

Usage: /list-bench

Steps

Run the inventory helper:
```
lib/inventory.py
```
It prints each project (with its pinned short commit), the N/6 tasks authored, a per-task ✓/· grid, and how many reports exist per task.
Summarize for the user: which projects are fully authored (6/6), which tasks are missing, and which (project × task) cells still have no reports. Suggest the next action — /create-task for missing tasks or /evaluate-agent for un-evaluated ones.
If asked to aggregate scores (e.g. "average correctness per model"), read the relevant reports/*/metrics.json directly and roll up along run.model / run.project / run.difficulty / run.kind.