fake-jira-skill

Query Jira for bugs in a project and summarize coverage gaps. Test fixture for e2e external-state field detection.

Repository: opendatahub-io/agent-eval-harness
Last source activity: April 30, 2026 at 12:42
Detected SKILL.md language: English
Stars: 40
Forks: 43

Install options

The review-first prompt is selected by default. You can switch to a direct command or download a local copy.

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

A direct command skips the review prompt. Inspect the source before running it.

npx skills add https://github.com/opendatahub-io/agent-eval-harness --skill fake-jira-skill

The command stays on one line. Scroll horizontally to inspect it before copying.

Prefer a local copy? Download the files currently available to SkillsMP.

Review the source files

Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.

SKILL.md

Source instructions · Read-only preview

Back to search Back to installation

eval-analyze

Generate eval.yaml for the agent eval harness. Two modes - (1) Skill-based - examines SKILL.md, sub-skills, scripts, test cases to verify implementation quality, OR (2) Prompt-based - tests agent capabilities using custom analysis prompts (documentation effectiveness, pattern understanding, API usage, constraint compliance). Produces complete config with execution mode, dataset schema, outputs, judges, models, thresholds. Use when setting up evaluation, testing skills/documentation, adding quality checks, or benchmarking. Auto-triggered by /eval-run when eval.yaml missing. Triggered by "how do I know if my skill/docs work?"

2026-08-27

eval-dataset

Generate evaluation test cases for an eval.yaml. Sources cases per generation.strategy - skill analysis (default), synthetic LLM generation from generation prompts (documentation and agent-capability evals), or MLflow production traces. Bootstraps a starter dataset or augments an existing one to improve coverage. Use when setting up evaluation, when the user needs test cases, when coverage is too thin, or after /eval-analyze when no dataset exists yet. Triggers on "create test cases", "generate test data", "need test inputs", "make a dataset", "add more cases", "improve coverage", "generate documentation eval cases". Also useful when /eval-run reports "no test cases found."

2026-08-27

eval-run

Execute an evaluation against test cases (skill or prompt mode), score with judges, and report results. Requires eval.yaml (generated by /eval-analyze). Use when the user wants to test a skill, run eval, benchmark, compare models, detect regressions, check skill quality, or verify changes didn't break anything. Triggers on "run eval", "test the skill", "evaluate", "benchmark", "check for regressions", "how does my skill perform", "score the skill", "run the tests", "run my evals", "compare against baseline", "did I break anything", "test my changes". Also called by /eval-optimize for automated iterations.

2026-08-27

View all from this repository

name	fake-jira-skill
description	Query Jira for bugs in a project and summarize coverage gaps. Test fixture for e2e external-state field detection.
user-invocable	true
allowed-tools	Read, Write, Bash, mcp__atlassian__searchJiraIssuesUsingJql, mcp__atlassian__getVisibleJiraProjects

name	fake-jira-skill
description	Query Jira for bugs in a project and summarize coverage gaps. Test fixture for e2e external-state field detection.
user-invocable	true
allowed-tools	Read, Write, Bash, mcp__atlassian__searchJiraIssuesUsingJql, mcp__atlassian__getVisibleJiraProjects

fake-jira-skill

Install options

Review the source files

Inputs

Steps

Environment

Inputs

Steps

Environment

fake-jira-skill

Source facts

Install options

Review the source files

More from this repository

More from this repository

Inputs

Steps

Environment

Inputs

Steps

Environment