メニュー
Summarize eval test results from the lucia agent evaluation suite into compact markdown. Parses DiskBasedReportingConfiguration JSON output and produces a metrics table, pass/fail summary, tool call chains, and failed scenario details.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
{what this skill teaches agents}
{what this skill teaches agents}
Run and debug GitHub Actions workflows locally using the `act` CLI tool. Covers runner image selection, secrets/vars handling, job targeting, event simulation, and common troubleshooting patterns. TRIGGER THIS SKILL WHEN: - Running or testing GitHub Actions workflows locally - Debugging CI/CD pipeline failures without pushing to GitHub - Validating workflow file changes before committing - Simulating different GitHub events (push, pull_request, workflow_dispatch) - Troubleshooting act runner image issues or missing tools SYMPTOMS THAT TRIGGER THIS SKILL: - Agent pushes workflow changes to test them instead of running locally - Agent doesn't know how to provide secrets or vars to local workflow runs - Agent can't reproduce a CI failure locally - Agent needs to run a specific job or workflow in isolation
Write integration tests using .NET Aspire's testing facilities with xUnit. Covers test fixtures, distributed application setup, endpoint discovery, and patterns for testing ASP.NET Core apps with real dependencies.
Design stable, compatible public APIs using extend-only design principles. Manage API compatibility, wire compatibility, and versioning for NuGet packages and distributed systems.
Write modern, high-performance C# code using records, pattern matching, value objects, async/await, Span<T>/Memory<T>, and best-practice API design patterns. Emphasizes functional-style programming with C# 12+ features.
| name | summarize-eval-results |
| description | Summarize eval test results from the lucia agent evaluation suite into compact markdown. Parses DiskBasedReportingConfiguration JSON output and produces a metrics table, pass/fail summary, tool call chains, and failed scenario details. |
| config | {"tool":"dotnet-script","script":"scripts/Summarize-EvalResults.csx","results_path":"%TEMP%/lucia-eval-reports/results"} |
Parses the JSON output from Microsoft.Extensions.AI.Evaluation.Reporting (DiskBasedReportingConfiguration) and produces a compact markdown report for analysis.
dotnet test on any eval test suite (Light, Music, Orchestrator)Latest execution (most common):
cd D:\github\seiggy\lucia-dotnet
dotnet script .copilot/skills/summarize-eval-results/scripts/Summarize-EvalResults.csx
Specific execution by ID:
dotnet script .copilot/skills/summarize-eval-results/scripts/Summarize-EvalResults.csx -- 20260218T182408
FindLight → SetLightState)| Metric | Type | Range | Description |
|---|---|---|---|
| Latency | Custom | 1-5 | Response time score (informational, never fails assertions) |
| Relevance | LLM Judge | 1-5 | Is the response relevant to the question? |
| Coherence | LLM Judge | 1-5 | Is the response well-structured and clear? |
| Task Adherence | LLM Judge | 1-5 | Did the agent accomplish what was asked? |
| Tool Call Accuracy | Built-in | True/False | Did the agent call the right tools? |
| A2A.Routing | Custom | 1-5 | Did the orchestrator produce a valid routing decision? |
| A2A.AgentTargeting | Custom | 1-5 | Were the expected agents selected? |
| A2A.AgentExecution | Custom | 1-5 | Did dispatched agents execute successfully? |
| A2A.Aggregation | Custom | 1-5 | Was a non-empty aggregated response produced? |
| A2A.Workflow | Custom | 1-5 | Composite score of all A2A sub-dimensions |
Scores < 4 are flagged as failed by test assertions (except Latency which is informational only).
Results are stored by the eval SDK at:
%TEMP%/lucia-eval-reports/results/<execution-id>/<scenario-name>/1.json
Each 1.json contains: scenarioName, messages (input), modelResponse (tool calls + final text), evaluationResult.metrics, chatDetails.turnDetails, and tags.