메뉴
View and analyze Hawk evaluation results. Use when the user wants to see eval-set results, check evaluation status, list samples, view transcripts, or analyze agent behavior from a completed evaluation run.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
| name | view-results |
| description | View and analyze Hawk evaluation results. Use when the user wants to see eval-set results, check evaluation status, list samples, view transcripts, or analyze agent behavior from a completed evaluation run. |
When the user wants to analyze evaluation results, use these hawk CLI commands:
You can list all eval sets if the user do not know the eval set ID:
hawk list eval-sets
Shows: eval set ID, creation date, creator.
You can increase the limit of results returned by --limit N.
hawk list eval-sets --limit 50
Or you can search for a specific eval set by using --search QUERY.
hawk list eval-sets --search pico
With an eval set ID, you can list all evaluations in the eval-set:
hawk list evals [EVAL_SET_ID]
Shows: task name, model, status (success/error/cancelled), and sample counts.
Or you can list individual samples and their scores:
hawk list samples [EVAL_SET_ID] [--eval FILE] [--limit N]
To get the full conversation for a specific sample:
hawk transcript <UUID>
The transcript includes full conversation with tool calls, scores, and metadata.
To get even more details, you can get the raw data by using --raw:
hawk transcript <UUID> --raw
You can also download all transcripts for an entire eval set:
# Fetch all samples in an eval set
hawk transcripts <EVAL_SET_ID>
# Write to individual files in a directory
hawk transcripts <EVAL_SET_ID> --output-dir ./transcripts
# Limit number of samples
hawk transcripts <EVAL_SET_ID> --limit 10
# Raw JSON output (one JSON per line to stdout, or .json files with --output-dir)
hawk transcripts <EVAL_SET_ID> --raw
hawk list eval-sets to see available eval sets
2a. Run hawk list evals <EVAL_SET_ID> to see available evaluations
2b. or run hawk list samples <EVAL_SET_ID> to find samples of interest
3a. Run hawk transcript <uuid> to get full details on a single sample
3b. or run hawk transcripts <eval_set_id> --output-dir ./transcripts to download allProduction (https://api.inspect-ai.internal.metr.org) is used by default. Set HAWK_API_URL only when targeting non-production environments:
| Environment | URL |
|---|---|
| Staging | https://api.inspect-ai.staging.metr-dev.org |
| Dev1 | https://api.inspect-ai.dev1.staging.metr-dev.org |
| Dev2 | https://api.inspect-ai.dev2.staging.metr-dev.org |
| Dev3 | https://api.inspect-ai.dev3.staging.metr-dev.org |
| Dev4 | https://api.inspect-ai.dev4.staging.metr-dev.org |
Example:
HAWK_API_URL=https://api.inspect-ai.staging.metr-dev.org hawk list eval_sets
Debug stuck Hawk/Inspect AI evaluations. Use when user mentions "stuck eval", "eval not progressing", "eval hanging", "samples not completing", "eval set frozen", "runner stuck", "500 errors in eval", "retry loop", "eval timeout", or asks why an evaluation isn't finishing.
Use when creating alembic migrations, applying migrations to remote environments, or recovering from schema drift. Triggers on changes to models.py, "run migration", "schema drift", "alembic", "database error in batch jobs".
Use when deploying code changes to dev environments (dev1-4), running terraform apply against dev, or verifying changes end-to-end. Triggers on "deploy to dev", "apply to dev2", "test in dev", "update dev environment".
Use when developing the frontend and backend together, making UI changes, or setting up local dev with linked inspect_ai/scout libraries. Triggers on frontend changes, "yarn dev", "vite", "www/", or React component work.
Use when running smoke tests, debugging smoke test failures, or verifying a deployed environment works correctly. Triggers on "run smoke tests", "smoke tests failing", "test against dev", "verify deployment".
Monitor Hawk job status, view logs, and diagnose issues. Use when the user wants to check job progress, view error logs, debug a failing job, or generate a monitoring report for a Hawk evaluation run.