一键在 Manus 中运行任何 Skill

prepare-submission-workflow

星标4

分支9

更新时间2026年4月29日 02:34

Prepare an evaluation for PR submission — add dependencies, tests, linting, eval.yaml, and README. Use when user asks to prepare an eval for submission or finalize a PR. Trigger when the user asks you to run the "Prepare Evaluation For Submission" workflow.

安装

用 Codex 或 Claude 帮你安装复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。

在 Manus 中运行

来源

Generality-Labs

Generality-Labs/inspect-evals-template

打开 GitHub 仓库查看创作者相关仓库

下载

在 Manus 中运行

Prepare Eval For Submission

Workflow Steps

To prepare an evaluation for submission as a pull request:

Add custom dependencies in the pyproject.toml file, if required. Add the package to the top-level [project] dependencies list (or to a dependency group if you want to keep it optional).
```
[project]
dependencies = [
    "inspect_ai>=0.3.158",
    "pyyaml>=5.1.0",
    "example-python-package",  # add your eval's runtime deps here
]
```
Pin the package version if your evaluation might be sensitive to version updates.

Make sure your module does not import your custom dependencies at module import time when those imports might fail. This can break the way inspect_ai discovers tasks via entry points.

To check for this, uninstall your optional dependencies and run another eval with --limit=0. If it fails due to one of your imports, you need to defer that import (move it inside the function that uses it).
Add pytest tests covering the custom functions included in your evaluation. See the Testing Standards section for more details.
Run pytest to ensure all new and existing tests pass. uv sync installs the dev dependency group (which includes pytest) by default, so a plain sync is enough:
```
uv sync
uv run pytest
```
Add an eval.yaml file to your evaluation directory (e.g., src/<eval_name>/eval.yaml) so the auto-generated README sections (Usage, Options, Parameters, Contributors) can be populated by tools/generate_readmes.py. See the eval.yaml Reference section in CONTRIBUTING.md for a complete example and field documentation.
Register the eval in pyproject.toml under [project.entry-points.inspect_ai] so uv run inspect eval <eval_name>/<task_name> resolves it:
```
[project.entry-points.inspect_ai]
<eval_name> = "<eval_name>"
```
Run linting, formatting, and type checking and address any issues:
```
make check
```
This runs the full check suite via tools/run_checks.sh, including autolint against the registry's recommended structural standards. Failures of advisory checks are reported but don't fail the build by default — see README.md "Checks and enforcement" for the toggle table. The same workflows run on incoming PRs, so address any enforced failures before opening a PR.
Fill in the missing sections of the generated README.md which are marked with 'TODO:'.

同仓库更多 Skills

同仓库

ensure-test-coverage

Generality-Labs/inspect-evals-template

Ensure test coverage for a single evaluation - both reviewing existing tests and creating missing ones. Analyzes testable components, checks tests against repository conventions, reports coverage gaps, and creates or improves tests. Use when user asks to check/review/create/add/ensure tests for an eval. Use whenever you are asked to review an evaluation that contains tests, or whenever you need to write a suite of tests.

2026-04-274

review-pr-workflow

Generality-Labs/inspect-evals-template

Automatically review a pull request against the agent-checkable standards in EVALUATION_CHECKLIST.md and BEST_PRACTICES.md. Produces a SUMMARY.md suitable for posting as a PR comment. Use when the user asks to run the "Review PR According to Agent-Checkable Standards" workflow, or to review a PR against template standards.

2026-04-274

check-trajectories-workflow

Generality-Labs/inspect-evals-template

Use Inspect Scout to analyze agent trajectories from evaluation log files. Runs default and custom scanners to detect external failures, formatting issues, reward hacking, and ethical refusals. Use when user asks to check/analyze agent trajectories. Trigger when the user asks you to run the "Check Agent Trajectories" workflow.

2026-04-274

create-eval

Generality-Labs/inspect-evals-template

Implement a new evaluation from an issue or paper. Guides through requirements gathering, architecture design, scaffolding, implementation, testing, and quality checks. Use when user asks to create/implement/build a new evaluation.

2026-04-274

eval-quality-workflow

Generality-Labs/inspect-evals-template

Fix or review a single evaluation against all EVALUATION_CHECKLIST.md standards. Use "fix" mode to refactor an eval into compliance, or "review" mode to assess compliance without making changes. Use when user asks to fix, review, or check an evaluation's quality. Trigger when the user asks you to run the "Fix An Evaluation" or "Review An Evaluation" workflow. Do NOT use for reviewing ALL evals against a single code quality standard (use code-quality-review-all instead).

2026-04-274

eval-report-workflow

Generality-Labs/inspect-evals-template

Create an evaluation report for a README by selecting models, estimating costs, running evaluations, and formatting results tables. Use when user asks to make/create/generate an evaluation report. Trigger when the user asks you to run the "Make An Evaluation Report" workflow.

2026-04-274

name	prepare-submission-workflow
description	Prepare an evaluation for PR submission — add dependencies, tests, linting, eval.yaml, and README. Use when user asks to prepare an eval for submission or finalize a PR. Trigger when the user asks you to run the "Prepare Evaluation For Submission" workflow.

Prepare Eval For Submission

Workflow Steps

To prepare an evaluation for submission as a pull request:

Add custom dependencies in the pyproject.toml file, if required. Add the package to the top-level [project] dependencies list (or to a dependency group if you want to keep it optional).
```
[project]
dependencies = [
    "inspect_ai>=0.3.158",
    "pyyaml>=5.1.0",
    "example-python-package",  # add your eval's runtime deps here
]
```
Pin the package version if your evaluation might be sensitive to version updates.

Make sure your module does not import your custom dependencies at module import time when those imports might fail. This can break the way inspect_ai discovers tasks via entry points.

To check for this, uninstall your optional dependencies and run another eval with --limit=0. If it fails due to one of your imports, you need to defer that import (move it inside the function that uses it).
Add pytest tests covering the custom functions included in your evaluation. See the Testing Standards section for more details.
Run pytest to ensure all new and existing tests pass. uv sync installs the dev dependency group (which includes pytest) by default, so a plain sync is enough:
```
uv sync
uv run pytest
```
Add an eval.yaml file to your evaluation directory (e.g., src/<eval_name>/eval.yaml) so the auto-generated README sections (Usage, Options, Parameters, Contributors) can be populated by tools/generate_readmes.py. See the eval.yaml Reference section in CONTRIBUTING.md for a complete example and field documentation.
Register the eval in pyproject.toml under [project.entry-points.inspect_ai] so uv run inspect eval <eval_name>/<task_name> resolves it:
```
[project.entry-points.inspect_ai]
<eval_name> = "<eval_name>"
```
Run linting, formatting, and type checking and address any issues:
```
make check
```
This runs the full check suite via tools/run_checks.sh, including autolint against the registry's recommended structural standards. Failures of advisory checks are reported but don't fail the build by default — see README.md "Checks and enforcement" for the toggle table. The same workflows run on incoming PRs, so address any enforced failures before opening a PR.
Fill in the missing sections of the generated README.md which are marked with 'TODO:'.