| name | import-repo |
| description | Import and pin a codebase into the benchmark. Use when the user says "import repo", "add a project to the bench", "pin <repo>", or gives a git URL/path to benchmark against. Clones the repo at a frozen commit, writes its SUMMARY.md orientation doc and project.json, and registers it in manifest.json. |
| version | 1.0.0 |
import-repo
Pin a new project into the benchmark so tasks can be authored against a frozen,
non-drifting snapshot.
Usage: /import-repo <git-url-or-path> [ref]
(e.g. /import-repo https://github.com/foo/bar main)
What "importing" means
- Clone the source into
projects/<name>/repo (gitignored — not committed).
- Freeze it at a specific commit (detached HEAD) so it never mutates.
- Write
projects/<name>/SUMMARY.md — the "summary pole" that orients task
authors and evaluators (the agent under test never sees this).
- Write
projects/<name>/project.json — machine-readable pin + build commands.
- Register the project in
manifest.json.
Steps
-
Pick a slug. Derive a short kebab-case <name> from the repo (e.g.
github.com/foo/Bar-Baz → bar-baz). Confirm it isn't already in
manifest.json.
-
Clone & pin using the helper (resolves and prints the full SHA):
lib/clone_pin.sh <source> <name> [ref]
If no ref is given it pins the default branch's current HEAD. The repo ends
on a detached commit so nothing tracks a moving branch.
-
Understand the repo. Explore projects/<name>/repo enough to write a real
summary: read the README, top-level layout, build/test config, main modules.
For anything non-trivial, prefer launching an Explore agent to map it.
Identify the build/install/test/run commands.
-
Write SUMMARY.md from templates/SUMMARY.md. Fill every section
honestly: what it is, architecture/layout, how to build & test, key concepts,
good task surface (where easy→hard Scout & Coder tasks could live), and
gotchas. This is the single most valuable artifact for later task authoring —
make it genuinely useful.
-
Write project.json from templates/project.json: name, source, ref,
the full commit SHA, imported_at (use date -u +%Y-%m-%dT%H:%M:%SZ),
languages, and the build/test/run commands you found.
-
Register in manifest.json. Append an entry to projects[] with name,
source, ref, commit, imported_at, language, one-line summary, and an empty
tasks: []. Keep the JSON valid (the schema is docs/manifest.schema.json).
-
Report back: the slug, the pinned commit, languages, build/test commands,
and a one-line pitch of where good tasks could live. Suggest running
/create-task <name> easy scout next.
Notes
- Do not commit
projects/*/repo/ — it's gitignored by design and rebuildable
from source + commit.
- If the clone fails (auth, network), report it and stop; don't fabricate a pin.
- To re-pin an existing project, remove
projects/<name>/repo first, then
re-run, and warn that existing tasks/base_commits may need re-checking.