メニュー
Use when running Agent golden-case evaluation, checking intent accuracy, tool-call accuracy, RAG grounding, job matching, interview follow-up policy, or resume rubric compliance.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
| name | agent-evaluation-tool |
| description | Use when running Agent golden-case evaluation, checking intent accuracy, tool-call accuracy, RAG grounding, job matching, interview follow-up policy, or resume rubric compliance. |
Use this skill to run the project golden-case evaluator without relying on API 200s. It calls evals.agent.run_agent_eval and reports metrics for intent, tool routing, arguments, RAG grounding, job recommendations, interview follow-ups, and resume rubrics.
Run inside the API container:
docker compose exec api python /app/skills/agent-evaluation-tool/scripts/run_agent_eval.py
Inputs: optional --write-report to update docs/evaluation/agent-rag-eval-report.md.
The script emits compact JSON: summary.case_count, summary.metrics, summary.failed_cases, and optional report_path. If failed cases exist, the script exits with code 2.
Lead with failed cases if any, then summarize the most relevant metrics. Do not treat passing evals as proof that live model calls or external providers are healthy.
python skills/agent-evaluation-tool/scripts/run_agent_eval.py --self-test
python skills/agent-evaluation-tool/scripts/run_agent_eval.py --help
docker compose exec api python /app/skills/agent-evaluation-tool/scripts/run_agent_eval.py
Use when inspecting AI assistant file-based long-term memory, session JSONL archives, soft consolidation, Dream updates, or assistant context isolation.
Use when reading saved jobs, application status counts, manual follow-up notes, or the user's application tracking state.
Use when inspecting FastAPI routes, controllers, backend service boundaries, repository-backed endpoints, API surface, or route availability.
Use when inspecting or explaining Qingcheng AI chat complexity classification, prompt-template injection, SupervisorAgent routing, selected tool allowlists, or agent planning for a user message.
Use when changing or reviewing the Next.js frontend UI, Tailwind layout, chat/interview pages, resume/job/application screens, or frontend API wrappers.
Use when inspecting simulated interview Agent short-term session state, job-resume binding, question plan progress, answer signals, difficulty, follow-up strategy, or remaining focus.