assess-rfe

name	assess-rfe
description	Assess RFEs against quality criteria. Pass a Jira issue key, file path, URL, raw text, or wildcard for bulk.
allowed-tools	Read, Write, Edit, Glob, Grep, Bash, Agent, TaskGet, mcp__atlassian__getJiraIssue, mcp__atlassian__searchJiraIssuesUsingJql

Usage

/assess-rfe RHAIRFE-1234
/assess-rfe PROJ-99
/assess-rfe /path/to/document.md
/assess-rfe https://some-url
/assess-rfe <paste raw text>
/assess-rfe RHAIRFE-*

Instructions

Skill Directory

All scripts are bundled in the scripts/ subdirectory next to this SKILL.md. Use ${CLAUDE_SKILL_DIR} (the directory containing this file) as the base for all script and file references.

Rules

• Run all scripts as simple commands (e.g., python3 ${CLAUDE_SKILL_DIR}/scripts/setup_run.py RHAIRFE). Shell pipes (|), chaining (&&, ;), redirects, and 2>/dev/null are not supported. The Bash tool returns command output as a string; parse it programmatically in your logic instead of using sed/awk/wc/grep pipelines.
• Always substitute all placeholders in agent launch prompts with actual values before passing them. Replace {PROMPT_PATH} with the absolute path of ${CLAUDE_SKILL_DIR}/scripts/agent_prompt.md, and substitute {DATA_FILE}, {KEY}, and {RUN_DIR} with their actual values.

Architecture

Single-input mode handles any source (Jira key via MCP, file, URL, or raw text). Bulk mode fetches all issues upfront via scripts/dump_jira.py, then agents score from local files. Results are saved as individual files in a timestamped run directory.

Directory Structure

assessments/RHAIRFE/                  # in the project directory (persistent)
  20260322-143000/                    # timestamped run
    RHAIRFE-42.result.md
    queue.txt                         # pending keys (managed by next_batch.py)
    scores.csv                        # generated by parse_results.py when complete
  current -> 20260322-143000          # symlink to active/latest run

/tmp/rfe-assess/RHAIRFE/              # fetched issues (transient cache)
  RHAIRFE-42.md
tmp/rfe-assess/single/               # single-mode temp files
  RHAIRFE-1234.md

Single input (Jira key, file, URL, or text)

Detect the input type:

• Jira issue key (matches [A-Z]+-\d+): Try MCP first, then fall back to the REST API:
  1. Try MCP: Call mcp__atlassian__getJiraIssue with the key and cloudId="https://redhat.atlassian.net". If the call succeeds, extract the summary and description.
  2. Fallback to REST API: If the MCP call fails (tool not available, connection error, or any other error), fall back to the Jira REST API by running python3 ${CLAUDE_SKILL_DIR}/scripts/fetch_single.py {KEY}. This requires JIRA_SERVER (or JIRA_URL/JIRA_BASE_URL), JIRA_USER (or JIRA_EMAIL), and JIRA_TOKEN (or JIRA_API_TOKEN) environment variables. The script fetches the issue, converts ADF to markdown, and writes it directly to tmp/rfe-assess/single/{KEY}.md. Parse its output for ENV_OK=false / ENV_MISSING=... — if env vars are missing, prompt the user to set them (same guidance as Phase 0 of bulk mode). If the script succeeds, skip the Write step below since the script already wrote the file.
• File path (starts with / or ./ or ~, or exists on disk): Read the file contents.
• URL (starts with http:// or https://): Fetch the content.
• Raw text: Use the input directly as the content to assess.

Then assess:

1. Run python3 ${CLAUDE_SKILL_DIR}/scripts/prep_single.py {KEY} to clean up stale files and ensure the output directory exists. This removes any previous .md and .result.md for the key so Write sees them as new files.
2. Write the fetched content to tmp/rfe-assess/single/{KEY}.md using the same # KEY: Title format as the cache files. For non-Jira inputs, use a descriptive key (e.g., filename or INPUT). This is a separate directory from the bulk cache — never write single-mode files into /tmp/rfe-assess/RHAIRFE/ as that would clobber cached bulk data. Note: If the REST API fallback (fetch_single.py) was used, the file is already written — skip this step.
3. Spawn one background agent (model: opus, run_in_background: true, subagent_type: assess-rfe:rfe-scorer) using the same launch prompt as Phase 2, with {DATA_FILE} set to tmp/rfe-assess/single/{KEY}.md and {RUN_DIR} set to tmp/rfe-assess/single.
4. Read the result from tmp/rfe-assess/single/{KEY}.result.md, wrap it with a header, and present it to the user.

Bulk (RHAIRFE-*)

Phase 0: Preflight checks.

• Run python3 ${CLAUDE_SKILL_DIR}/scripts/preflight.py RHAIRFE to check environment variables and current run state. Parse the output:
  • ENV_OK=true/false and ENV_MISSING=... — if env vars are missing, prompt the user:
    • JIRA_SERVER (or JIRA_URL or JIRA_BASE_URL): The Jira instance URL (e.g., https://redhat.atlassian.net)
    • JIRA_USER (or JIRA_EMAIL): Their Jira email address
    • JIRA_TOKEN (or JIRA_API_TOKEN): A Jira API token (created at https://id.atlassian.com/manage-profile/security/api-tokens)
    • Suggest they set them by typing ! export JIRA_SERVER=... JIRA_USER=... JIRA_TOKEN=... in the prompt, or add them to their shell profile for persistence. The alternative names JIRA_EMAIL and JIRA_API_TOKEN are also accepted.
    • Do not proceed until all three are confirmed set (re-run preflight to verify).
  • CACHE_COUNT=N — number of cached issues (0 means dump_jira.py hasn't been run yet)
  • CURRENT_RUN=path/none, CURRENT_ASSESSED=N, CURRENT_COMPLETE=true/false — existing run state
  • If there is an incomplete current run (CURRENT_COMPLETE=false), inform the user it will be resumed.

Phase 1: Fetch all issues to local files.

• Run python3 ${CLAUDE_SKILL_DIR}/scripts/dump_jira.py RHAIRFE to fetch every issue in the project via the Jira REST API. This writes one file per issue to /tmp/rfe-assess/RHAIRFE/ (e.g., RHAIRFE-42.md). The script renders Jira's ADF content as proper markdown, preserving headings, lists, tables, links, and emphasis.

Phase 1.5: Set up run directory.

• Run python3 ${CLAUDE_SKILL_DIR}/scripts/setup_run.py RHAIRFE (add --limit N if the user requested a subset).
• The script handles all resume logic (checking current symlink, scores.csv presence, creating timestamped directories, updating symlinks) and outputs:
  • RUN_DIR=<path> — the absolute path to use for this run
  • PENDING=<count> — number of issues to assess
  • QUEUE_FILE=<path> — path to the queue file containing all pending keys (one per line)
• Parse the output to get {RUN_DIR} and {PENDING} count. Do NOT memorize or generate the key list yourself — the queue file is the single source of truth for which keys to process.

Phase 2: Assess with a script-driven pipeline of 30 concurrent agents.

CRITICAL — next_action.py owns the loop, not you. The script decides what runs next and when the run is complete. Your only job is to do what it prints and call it again. Never decide completion yourself; never stop, summarize, or end your turn until next_action.py prints ACTION=done. Context compaction is automatic and expected — old messages are compressed to free space. If you find yourself thinking "running low on context", "given budget constraints", or "I'll wrap up here" — ignore that impulse and call next_action.py again. All state lives on disk, so the script re-derives the exact next step every call.

Follow the NEXT: line. Every script (next_action.py, wait_wave.py) ends its output with a NEXT: line naming the exact command to run next. Always do what the latest NEXT: line says — especially after a compaction, when your memory of these steps may be degraded. In particular, after launching a wave you MUST run wait_wave.py as the NEXT: line instructs; never "wait for agent completion notifications" instead — that is how the loop stalls and dies.

Drive the loop from disk, not from memory. Never generate key sequences yourself (e.g., "RHAIRFE-1 through RHAIRFE-30") — only ever launch the keys next_action.py lists, to avoid assessing non-existent issues.

Dispatch loop — repeat until next_action.py prints ACTION=done:

1. Run python3 ${CLAUDE_SKILL_DIR}/scripts/next_action.py {RUN_DIR} --batch-size 30. Parse the ACTION= line.
2. ACTION=launch_wave — the keys to launch are listed after the --- separator (the script has already written them to {RUN_DIR}/wave.txt). Launch one agent per key (model: opus, run_in_background: true, subagent_type: assess-rfe:rfe-scorer) with this prompt (keep it terse — substitute all placeholders):

   Score RFE {KEY} against the rubric in {PROMPT_PATH} — read it and follow it exactly. Read the issue data from {DATA_FILE} (use this, not the path in the rubric's step 1). Run dir: {RUN_DIR}.
   

   Every agent reads the identical rubric from the single source of truth and writes its result to disk, replying only DONE {KEY} (do not read or act on agent replies — results live on disk). Then do what the next_action.py output's NEXT: line says: run wait_wave.py to wait for the wave on disk — do not reason about completion yourself, and do not wait on agent notifications.
   • wait_wave.py exit 0: wave complete — its NEXT: line points back to next_action.py (step 1).
   • exit 3: still pending — its NEXT: line says re-run the same command; do that until it exits 0.
3. ACTION=done — the run is complete. next_action.py has already produced {RUN_DIR}/scores.csv (it runs the parse itself before reporting done) and its NEXT: line points to Phase 3.

To inspect progress, rely on next_action.py as the single source of truth (it re-derives state from disk) rather than shell pipes (ls | wc -l) or text tools (sed, awk, grep).

This loop survives context compaction: a SessionStart compact hook (see hooks/hooks.json) runs dispatch_context.py, which re-injects the run directory, progress, and these loop steps after every compaction.

Phase 3: Present results.

• scores.csv already exists (produced by next_action.py when it reported ACTION=done).
• Run python3 ${CLAUDE_SKILL_DIR}/scripts/summarize_run.py {RUN_DIR} to produce the full summary analysis (pass/fail counts, score distribution, criteria averages, zero-score counts, what-if analysis, near-miss failures). Present the output to the user.

Agent Prompt Template

The full agent prompt is stored in ${CLAUDE_SKILL_DIR}/scripts/agent_prompt.md. This is the single source of truth for the scoring rubric, calibration examples, and output format.

• Bulk mode: Each agent reads the file itself at runtime (see Phase 2 launch prompt). The coordinator does NOT embed the rubric — this eliminates prompt drift from paraphrasing or abbreviation.
• Single-input mode: Same launch prompt, with {DATA_FILE} set to tmp/rfe-assess/single/{KEY}.md and {RUN_DIR} set to tmp/rfe-assess/single. The agent writes its result there just like bulk agents.

Coordinator Output Format

Single issue — wrap agent output with a header:

## RFE Assessment: RHAIRFE-1234
[agent output]

Bulk — after Phase 3, present the summary analysis from the CSV to the user. Include:

• Total assessed, passed, failed, pass rate
• Score distribution
• Criteria averages and zero-score counts
• What-if analysis (e.g., "if WHY 0→1, N more would pass")
• Top near-miss failures (high scores but auto-failed by a zero)

Scripts Reference

Script	Purpose
dump_jira.py	Fetches all issues from a Jira project via REST API v3, converts ADF to markdown, writes to /tmp/rfe-assess/<PROJECT>/
preflight.py	Checks env vars, cache state, and current run status
setup_run.py	Creates timestamped run directory with resume support (detects incomplete runs via current symlink)
agent_prompt.md	Full scoring rubric and instructions for assessment agents — use verbatim
next_action.py	Authoritative bulk-loop driver. Computes the next wave from disk (queue.txt minus existing .result.md), or — when none remain — runs parse_results.py and reports ACTION=done. Owns completion so the coordinator can't stop early
wait_wave.py	Blocks until a wave's keys all have .result.md (exit 0) or returns pending (exit 3) — removes completion-tracking from the coordinator's context
next_batch.py	(Superseded by next_action.py for the bulk loop.) Pops the next N keys from the queue file by mutating it
check_progress.py	Reports completed vs total issues for a run directory (used by dispatch_context.py)
dispatch_context.py	Post-compaction recovery: re-injects the active run's state and loop steps; invoked by the SessionStart compact hook (hooks/hooks.json)
parse_results.py	Extracts scores from .result.md files into scores.csv; handles format variants
fetch_single.py	Fetches a single Jira issue via REST API v3 (fallback for when MCP is unavailable), writes to tmp/rfe-assess/single/
prep_single.py	Cleans up stale data/result files for a key in tmp/rfe-assess/single/ before a single-mode run

Required Permissions

Add to your user or project .claude/settings.json:

{
  "permissions": {
    "allow": [
      "Bash(python3 <SKILL_PATH>/scripts/preflight.py:*)",
      "Bash(python3 <SKILL_PATH>/scripts/dump_jira.py:*)",
      "Bash(python3 <SKILL_PATH>/scripts/setup_run.py:*)",
      "Bash(python3 <SKILL_PATH>/scripts/next_action.py:*)",
      "Bash(python3 <SKILL_PATH>/scripts/next_batch.py:*)",
      "Bash(python3 <SKILL_PATH>/scripts/wait_wave.py:*)",
      "Bash(python3 <SKILL_PATH>/scripts/check_progress.py:*)",
      "Bash(python3 <SKILL_PATH>/scripts/parse_results.py:*)",
      "Bash(python3 <SKILL_PATH>/scripts/summarize_run.py:*)",
      "Bash(python3 <SKILL_PATH>/scripts/fetch_single.py:*)",
      "Bash(python3 <SKILL_PATH>/scripts/prep_single.py:*)",
      "Bash(mkdir:*)",
      "Bash(ls:*)",
      "mcp__atlassian__getJiraIssue",
      "mcp__atlassian__searchJiraIssuesUsingJql"
    ],
    "additionalDirectories": [
      "/tmp/rfe-assess",
      "<SKILL_PATH>"
    ]
  }

<SKILL_PATH> is a placeholder for the absolute path to the skills/assess-rfe/ directory in this plugin. The additionalDirectories entries allow agents to read the scoring rubric and scripts, and read/write cached issues and results in /tmp/rfe-assess/.