| name | prepare-submission-workflow |
| description | Prepare an evaluation for PR submission as an entry to the register. Use when user asks to prepare an eval for submission or finalize a PR. Trigger when the user asks you to run the "Prepare Evaluation For Submission" workflow. |
Prepare Eval For Submission
Since May 2026, new evaluations are submitted as entries to the register — the evaluation code lives in your own upstream repository, and you add a pointer to it here. Code is no longer added directly to src/inspect_evals/. If the user appears to be submitting evaluation code into the repo, direct them to register/README.md for the full process.
Workflow Steps
To prepare an evaluation for submission as a pull request:
1. Verify upstream repo requirements
The upstream repo must:
- Have a
pyproject.toml with a [project] table so it can be installed via uv sync
- Declare
inspect_ai as a dependency
- Define each task with the
@task decorator from inspect_ai
Ask the user whether their upstream repo meets these requirements. Offer to check for them — if they provide the GitHub repository URL, fetch the repo's pyproject.toml and task files (e.g. via WebFetch on the raw GitHub URLs) to verify the requirements are met. If any requirement is not met, tell the user what needs to be fixed upstream before they can register.
Important: The submitter must be the owner or a maintainer of the upstream repository. The submission workflow enforces this — if they are not, they will need to fork the repo so they can be listed as a maintainer.
2. Gather information and create register/<eval_name>/eval.yaml
