菜单
Prepare a new release of inspect_evals by creating a release branch, collecting changelog fragments, and opening a PR. Use when user asks to cut/prepare/create a new release or version bump.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
| name | prepare-release |
| description | Prepare a new release of inspect_evals by creating a release branch, collecting changelog fragments, and opening a PR. Use when user asks to cut/prepare/create a new release or version bump. |
This workflow prepares a new release of inspect_evals. It creates a release branch, collects changelog fragments, bumps the version tag, and opens a PR. After merge it tags the merge commit and creates a GitHub release.
Reference: PACKAGE_VERSIONING.md
gh CLI must be authenticatedchangelog.d/ (beyond .gitkeep and TEMPLATE.md)Ensure main is up to date:
git fetch origin
git checkout main
git merge --ff-only origin/main
If the merge fails, stop and inform the user that their local main has diverged from origin/main.
Check for changelog fragments:
List files in changelog.d/ excluding .gitkeep, TEMPLATE.md, and README.*. If there are no fragments, stop and tell the user there is nothing to release.
Determine the new version:
a. Find the current version from the latest v* git tag:
git tag --sort=-v:refname --list 'v*' | head -1
b. Ask the user what kind of bump this is, presenting the semver table from PACKAGE_VERSIONING.md:
| Component | When to bump | Examples |
|---|---|---|
| Major | Breaking changes | Removing an eval, API changes, scorer output format changes |
| Minor | New features | Adding new evals, new task parameters, new utilities |
| Patch | Bug fixes | Eval fixes, scorer fixes, dataset loading fixes |
c. Compute the new version string (e.g. 0.4.0, 0.3.107). Confirm with the user.
Create the release branch:
The branch name is release-YYYY-MM-DD using today's date.
git checkout -b release-YYYY-MM-DD
Run scriv collect:
uv run scriv collect --version <NEW_VERSION>
This removes the individual fragment files from changelog.d/ and prepends a new section to CHANGELOG.md.
Present the changelog diff to the user for review:
git diff CHANGELOG.md
Tell the user: "Please review the collected changelog above. You can edit CHANGELOG.md directly — let me know when you are satisfied, or tell me what changes to make."
Do not proceed until the user confirms the changelog is ready.
Stage and commit:
git add CHANGELOG.md changelog.d/
git commit -m "Prepare release v<NEW_VERSION>"
Push the branch:
git push -u origin release-YYYY-MM-DD
Open a draft PR:
Extract the new version's section from CHANGELOG.md (everything from the version heading up to but not including the next ## heading). Use this as the PR body:
gh pr create --draft \
--title "Release v<NEW_VERSION>" \
--body "<EXTRACTED_CHANGELOG_SECTION>"
Important: The PR title must start with Release v (e.g. Release v0.4.0) — this is how the release-on-merge.yml workflow identifies release PRs.
Tell the user the PR URL and that it is in draft. They should mark it ready for review when appropriate.
Once the release PR is merged into main, the .github/workflows/release-on-merge.yml GitHub Actions workflow automatically:
v<NEW_VERSION>No manual action is required. The package version is derived from the new tag by setuptools_scm.
setuptools_scm — there is no version file to edit.v prefix on tags is required (e.g. v0.3.107, not 0.3.107).weekly-release.yml workflow can also automate Phases 1–3 on a schedule or via manual dispatch, creating the release branch and draft PR for you.CI and GitHub Actions maintenance workflows — fix a failing test from a CI URL, fix a failing smoke test, add @pytest.mark.slow markers to slow tests, or review a PR against agent-checkable standards. Use when user asks to fix a failing test, fix a smoke test, mark slow tests, or review a PR. Trigger when the user asks you to run the "Write a PR For A Failing Test", "Fix A Failing Smoke Test", "Mark Slow Tests", or "Review PR According to Agent-Checkable Standards" workflow.
Prepare an evaluation for PR submission as an entry to the register. Use when user asks to prepare an eval for submission or finalize a PR. Trigger when the user asks you to run the "Prepare Evaluation For Submission" workflow.
Review a single evaluation's validity — whether its claims hold up, whether its name is accurate, whether samples can be both succeeded and failed at, and whether scoring measures ground truth. Use when user asks to check validity of an eval, or as part of the Master Checklist workflow. Do NOT use for code quality or test coverage (use eval-quality-workflow or ensure-test-coverage instead).
Fix code quality issues identified in a code quality review stored in agent_artefacts/code_quality/<topic>/. Systematically addresses issues found by the code-quality-review-all skill for ANY code quality topic, with validation and testing at each step. Use when user asks to fix issues from a code quality review, or asks to fix issues from agent_artefacts/code_quality/<topic>.
Create an evaluation report for a README by selecting models, estimating costs, running evaluations, and formatting results tables. Use when user asks to make/create/generate an evaluation report. Trigger when the user asks you to run the "Make An Evaluation Report" workflow.
Redirect to the inspect-evals-template for creating new evaluations. New evals are no longer created in this repository — they live in standalone repos. Use when user asks to create/implement/build a new evaluation.