菜单
Use when discovering EvalHub providers, benchmarks, or collections — listing what evaluation options exist, filtering by capability (e.g. safety, reasoning), or reading agent metadata. Does NOT cover job submission or status monitoring.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
Scans Go repositories for CVEs with govulncheck and upgrades the Go toolchain only when a matching tag exists on registry.access.redhat.com/ubi9/go-toolset. Use when checking Go CVEs, remediating stdlib or module vulnerabilities, bumping go.mod or Dockerfile Go versions, or validating Red Hat UBI9 go-toolset availability. Make sure to use this skill whenever the user mentions Go CVEs, govulncheck, golang version bumps, or ubi9/go-toolset image availability.
Use when evaluating AI/ML models, running benchmarks, checking evaluation status, listing providers or collections, or performing model quality and safety assessments. Covers EvalHub provider discovery, benchmark execution, job lifecycle, and Evaluation-Driven Development (EDD) workflows.
Use when submitting an evaluation job to EvalHub — running a model against benchmarks or collections. Covers job creation only. For status monitoring, use evalhub-jobs. For provider/benchmark discovery, use evalhub-discovery.
Use when monitoring, waiting on, cancelling, or fetching logs for an EvalHub evaluation job. Covers the full job lifecycle after submission. For submitting new jobs, use evalhub-eval.
| name | evalhub-discovery |
| description | Use when discovering EvalHub providers, benchmarks, or collections — listing what evaluation options exist, filtering by capability (e.g. safety, reasoning), or reading agent metadata. Does NOT cover job submission or status monitoring. |
| user-invocable | false |
Discover providers, benchmarks, and collections from the EvalHub API. All knowledge comes from the API at runtime — never hardcode IDs or names.
| Variable | Purpose | Example |
|---|---|---|
EVALHUB_BASE_URL | EvalHub API base URL | https://evalhub.apps.cluster.example.com |
EVALHUB_TOKEN | Bearer token for auth | sha256~... (from oc whoami -t) |
EVALHUB_TENANT | Namespace / tenant | eval-test |
Default starting point — two parallel calls:
uv run ~/.claude/skills/evalhub/scripts/evalhub_providers.py --agent 2>/dev/null
uv run ~/.claude/skills/evalhub/scripts/evalhub_collections.py --agent 2>/dev/null
The --agent output contains the complete agent metadata block for every provider/collection. Do not fetch individual providers afterwards — everything is already here.
Filter when user intent is clear:
uv run ~/.claude/skills/evalhub/scripts/evalhub_providers.py --target-type agent 2>/dev/null
uv run ~/.claude/skills/evalhub/scripts/evalhub_providers.py --evaluates safety 2>/dev/null
uv run ~/.claude/skills/evalhub/scripts/evalhub_collections.py --evaluates safety 2>/dev/null
Single provider (only when user explicitly asks for benchmarks):
uv run ~/.claude/skills/evalhub/scripts/evalhub_providers.py PROVIDER_ID 2>/dev/null
When evalhub MCP server is connected, use resources instead of Python scripts:
| MCP Resource URI | Replaces |
|---|---|
evalhub://providers | evalhub_providers.py --agent |
evalhub://providers/{id} | evalhub_providers.py PROVIDER_ID |
evalhub://benchmarks | evalhub_providers.py --benchmarks |
evalhub://benchmarks?label=safety | evalhub_providers.py --evaluates safety |
evalhub://collections | evalhub_collections.py --agent |
evalhub://collections/{id} | evalhub_collections.py COLLECTION_ID |
| Field | How to use |
|---|---|
evaluates | Match against user intent — "safety" → --evaluates safety |
target_type | model, agent, or inference_server |
summary | Present to the user |
recommended_when | When to suggest this provider/collection |
hints | Constructing valid job requests — read before submitting |
result_interpretation | How to explain scores after a run |
complements | Suggest follow-up evaluations |
uv run ~/.claude/skills/evalhub/scripts/evalhub_providers.py # summary
uv run ~/.claude/skills/evalhub/scripts/evalhub_providers.py --agent # full metadata
uv run ~/.claude/skills/evalhub/scripts/evalhub_providers.py PROVIDER_ID # single provider
uv run ~/.claude/skills/evalhub/scripts/evalhub_providers.py --target-type model
uv run ~/.claude/skills/evalhub/scripts/evalhub_providers.py --evaluates safety
uv run ~/.claude/skills/evalhub/scripts/evalhub_providers.py --benchmarks
uv run ~/.claude/skills/evalhub/scripts/evalhub_providers.py --benchmarks --provider PROVIDER_ID
uv run ~/.claude/skills/evalhub/scripts/evalhub_providers.py --benchmarks --category CATEGORY
uv run ~/.claude/skills/evalhub/scripts/evalhub_collections.py # summary
uv run ~/.claude/skills/evalhub/scripts/evalhub_collections.py --agent # full metadata
uv run ~/.claude/skills/evalhub/scripts/evalhub_collections.py COLLECTION_ID # single
uv run ~/.claude/skills/evalhub/scripts/evalhub_collections.py --evaluates CAPABILITY