Skip to main content

kubernetes-sigs/devops-bench

最近记录的来源活动
SkillsMP 收录数据更新
已收集 skills
8
GitHub 星标
34
GitHub Forks
16

用 AI 助手安装

复制这段 Prompt,发给你正在使用的 AI 助手。

正在加载仓库 README…

已展示 8 / 8 个已收集 Skill。

职业分类
未分类
描述

Explain WHY a model scored low on an eval — read the judge's reasons and the agent's trajectory, compare against the task rubric, and produce a rationale (capability gap vs. task/rubric problem). Distinct from infrastructure failures. Invoke when someone asks…

原文语言:英语

更新
职业分类
未分类
描述

Use when the user adds or changes a benchmark TASK and asks to review it — e.g. "review this new task", "is this task parallel-safe", "review this task.yaml + stack", "will this task collide under the matrix", "check my new tf/prebuilt stack". Reviews a newly…

原文语言:英语

更新
职业分类
软件质量保证分析师与测试员
描述

Validate that a NEW eval is correct and runnable by running it and iterating in a loop until green — fixing infra/config/task bugs (not the model's score), with opt-in hands-off and unlimited/self-healing modes; invoke when authoring or vetting a task before…

原文语言:英语

更新
职业分类
软件开发工程师
描述

Keep the docs current after a code change — map changed code areas to the docs that describe them, update those docs in place, and resolve known-issues rows the code has verifiably fixed. Invoke after editing the pipeline, adding a model/agent/task/metric,…

原文语言:英语

更新
职业分类
软件开发工程师
描述

Run ONE evaluation — a single Task × Model × AgentConfig — end to end, local or on the bastion; invoke when the user wants to run, evaluate, or "kick off" a single eval combo (e.g. "run opa-remediation on gemini-3.1-pro-preview", "evaluate this task once and…

原文语言:英语

更新
职业分类
软件开发工程师
描述

Run a Task × Model × AgentConfig MATRIX of evals in parallel — local or on the bastion, with opt-in hands-off and unlimited/self-healing modes; invoke when the user wants a matrix, a comparison across models/configs, or to "run these evals in parallel" /…

原文语言:英语

更新
职业分类
软件质量保证分析师与测试员
描述

Use when the user asks for a CODE review of devops-bench changes — e.g. "review this PR", "review my changes", "review the working tree", "is this harness/deployer/metric change sound". Reviews a PR (number/URL) or the current working tree and returns ranked…

原文语言:英语

更新
职业分类
网络与计算机系统管理员
描述

Discover and remove cloud or local resources leaked by aborted or failed eval runs — stale per-run state, leftover kind clusters and their node containers, stuck harness processes, and orphaned GKE clusters, node service accounts, secrets, VPCs and their…

原文语言:英语

更新
已展示 8 / 8 个已收集 Skill。