メニュー
Use when deploying code changes to dev environments (dev1-4), running terraform apply against dev, or verifying changes end-to-end. Triggers on "deploy to dev", "apply to dev2", "test in dev", "update dev environment".
デフォルトでは、最初にソースを確認する Prompt が選択されています。直接コマンドに切り替えるか、ローカルコピーをダウンロードすることもできます。
インストールを決める前に、SKILL.md と SkillsMP に表示されている付属ファイルをお読みください。
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
直接コマンドでは確認用 Prompt が省略されます。実行前にソースを確認してください。
npx skills add https://github.com/METR/inspect-action --skill deploy-devコマンドは1行のまま表示されます。コピー前に横へスクロールして全体を確認してください。
ローカルで確認しますか？SkillsMP が現在取得できるファイルをダウンロードできます。
SKILL.md を表示中
SOC 職業分類に基づく
| name | deploy-dev |
| description | Use when deploying code changes to dev environments (dev1-4), running terraform apply against dev, or verifying changes end-to-end. Triggers on "deploy to dev", "apply to dev2", "test in dev", "update dev environment". |
Dev environments (dev1-4) run in the staging AWS account.
Shared resources (when create_* flags are false in tfvars):
staging-metr-inspect-datastaging-inspect-ai-apistaging-eks-clusterDedicated per environment:
{env_name}-inspect-ai-warehouse{env_name}-inspect-ai-api{env_name}-inspect-ai-*{env_name}/inspect-ai/*Config lives in terraform/terraform.tfvars (gitignored — set to your target environment before deploying).
Note: Commands below may require AWS credentials for the staging account. Set AWS_PROFILE or configure credentials as appropriate for your environment.
When you only changed one component:
tofu -chdir=terraform apply \
-var-file="terraform.tfvars" -target=module.<target> -auto-approve
Common targets:
| Target | What It Deploys |
|---|
module.eval_log_importer | Batch job for importing eval logs to warehouse |
module.job_status_updated | Lambda processing S3 events |
module.api | API server (ECS Fargate) |
module.scan_importer | Lambda for scan imports |
module.eval_log_reader | Lambda for S3 Object Lambda access |
module.warehouse | PostgreSQL Aurora database |
# Preview changes
tofu -chdir=terraform plan -var-file="terraform.tfvars"
# Apply all changes
tofu -chdir=terraform apply -var-file="terraform.tfvars"
Terraform automatically tracks uv.lock, pyproject.toml, and Python source files to determine when the runner Docker image needs rebuilding (see terraform/modules/runner/ecr.tf). tofu apply is the correct way to rebuild runner images — it produces deterministic sha256.<hash> image tags.
Do NOT use scripts/dev/build-and-push-runner-image.sh for deploying to dev environments. That script is only for local hawk local testing.
Before tofu apply can build and push images, you must be logged into:
docker-credential-ecr-login helper and configure Docker to use itdocker login dhi.ioAfter changing Python dependencies (pyproject.toml or uv.lock), run ./scripts/dev/uv-lock-all.sh to update lock files in ALL terraform modules. Terraform modules have their own lock files that must stay in sync with the root.
See the db-migrations skill for full details.
Quick command:
DATABASE_URL=$(tofu -chdir=terraform output \
-var-file="terraform.tfvars" -raw warehouse_database_url_admin) \
alembic upgrade head
See the smoke-tests skill for full details.
Quick command:
unset HAWK_MODEL_ACCESS_TOKEN_ISSUER
scripts/dev/create-smoke-test-env.py env/smoke-dev2 --terraform-dir terraform
set -a && source env/smoke-dev2 && set +a && \
pytest tests/smoke -m smoke --smoke -vv -n 5
The typical deploy-and-verify loop:
pytest tests/{package} -n auto -vvruff check . && ruff format . --check && basedpyright .tofu apply -target=module.XDev environments default to reusing staging infra. To create dedicated resources, change these flags in your tfvars:
# Shared (default for dev environments)
create_s3_bucket = false
create_eventbridge_bus = false
create_eks_resources = false
# Dedicated (set to true + provide unique names)
create_s3_bucket = true
s3_bucket_name = "dev5-metr-inspect-data"
create_eventbridge_bus = true
eventbridge_bus_name = "dev5-inspect-ai-api"
create_eks_resources = true
Use shared (recommended): Cheaper, simpler, sufficient for most dev work.
Use dedicated: When testing infrastructure changes, or when you need full isolation from other devs.
When using shared infrastructure, dev environments reuse staging's k8s namespace, ClusterRole, and ClusterRoleBinding. This means:
create_eks_resources=true)| Component | Log Group |
|---|---|
| API server | {env_name}/inspect-ai/api |
| Batch importer | /{env_name}/inspect-ai/eval-log-importer/batch |
| Job status Lambda | /aws/lambda/{env_name}-inspect-ai-job-status-updated |
| Runner pods | Use hawk logs <eval-set-id> |
Tail logs: aws logs tail <log-group> --since 30m --format short
| Issue | Cause | Fix |
|---|---|---|
tofu apply fails with dependency cycle | Deposed resources from previous module refactors | tofu state rm on the orphaned resource, then re-apply |
exec format error during Docker build | Building wrong architecture (e.g., arm64 instead of amd64) | Terraform handles this correctly; if building manually, specify --platform linux/amd64 |
| Lambda can't find Docker image after apply | Docker build failed silently | Check ECR for the expected image tag; verify registry auth |
| Connection refused / timeout to dev resources | VPC routing issue — may be running in a different VPC | Check Tailscale is connected; if in a different AWS VPC, need user assistance for networking |
tofu apply succeeds but runner image is stale | Using manual build script instead of terraform | Use tofu apply — it tracks uv.lock and rebuilds automatically |
Debug stuck Hawk/Inspect AI evaluations. Use when user mentions "stuck eval", "eval not progressing", "eval hanging", "samples not completing", "eval set frozen", "runner stuck", "500 errors in eval", "retry loop", "eval timeout", or asks why an evaluation isn't finishing.
Use when creating alembic migrations, applying migrations to remote environments, or recovering from schema drift. Triggers on changes to models.py, "run migration", "schema drift", "alembic", "database error in batch jobs".
Use when developing the frontend and backend together, making UI changes, or setting up local dev with linked inspect_ai/scout libraries. Triggers on frontend changes, "yarn dev", "vite", "www/", or React component work.