Run any Skill in Manus with one click

generate-report

Stars4

Forks1

UpdatedApril 15, 2026 at 23:15

Aggregate WebBench benchmark results into a comprehensive evaluation report. Triggers: "generate report", "create benchmark report", "summarize results", "aggregate scores", "produce evaluation report". Reads web-bench-results.jsonl, computes statistics by category/website/failure mode, and writes web-bench-report.md with pass rates, timing, token usage, and analysis. Does NOT execute or evaluate tasks — only aggregates existing results.

Installation

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

Run Skill in Manus

Source

lespaceman

lespaceman/athena-workflow-marketplace

View GitHub Repository View Creator Repositories

Download

Run Skill in Manus

Related occupationsSOC

Based on SOC occupation classification

Software DevelopersComputer and Mathematical Occupations·SOC 15-1252

File Explorer

2 files

SKILL.md

readonly

name	generate-report
description	Aggregate WebBench benchmark results into a comprehensive evaluation report. Triggers: "generate report", "create benchmark report", "summarize results", "aggregate scores", "produce evaluation report". Reads web-bench-results.jsonl, computes statistics by category/website/failure mode, and writes web-bench-report.md with pass rates, timing, token usage, and analysis. Does NOT execute or evaluate tasks — only aggregates existing results.
allowed-tools	Read Write Edit Bash Glob Grep

Generate WebBench Benchmark Report

Aggregate all results from web-bench-results.jsonl into a comprehensive markdown report.

Input

Results file: web-bench-results.jsonl — one JSON line per completed task

Result line schema:

{
  "id": 42,
  "url": "https://acehardware.com",
  "category": "READ",
  "task": "Navigate to...",
  "score": 1.0,
  "verdict": "PASS",
  "reasoning": "Successfully extracted all specs",
  "error": null,
  "duration_ms": 34200,
  "tokens_used": {"input": 12450, "output": 3200},
  "timestamp": "2026-03-19T14:30:00Z"
}

Report Generation

Use Node.js to compute all statistics:

node -e "
const fs = require('fs');
const results = fs.readFileSync('web-bench-results.jsonl','utf-8').trim().split('\n')
  .filter(l => l.trim()).map(JSON.parse);

const total = results.length;
const passed = results.filter(r => r.verdict === 'PASS').length;
const partial = results.filter(r => r.verdict === 'PARTIAL').length;
const failed = results.filter(r => r.verdict === 'FAIL').length;
const totalScore = results.reduce((s, r) => s + r.score, 0);

// Timing
const totalDurationMs = results.reduce((s, r) => s + (r.duration_ms || 0), 0);
const avgDurationMs = total ? totalDurationMs / total : 0;

// Tokens
const totalInputTokens = results.reduce((s, r) => s + ((r.tokens_used || {}).input || 0), 0);
const totalOutputTokens = results.reduce((s, r) => s + ((r.tokens_used || {}).output || 0), 0);
const totalTokens = totalInputTokens + totalOutputTokens;

// By category
const byCat = {};
for (const r of results) {
  if (!byCat[r.category]) byCat[r.category] = [];
  byCat[r.category].push(r);
}

// By website (top failures)
const bySite = {};
for (const r of results) {
  if (!bySite[r.url]) bySite[r.url] = [];
  bySite[r.url].push(r);
}

// Failure modes
const blockers = {};
for (const r of results) {
  if (r.error) blockers[r.error] = (blockers[r.error] || 0) + 1;
}

const catStats = {};
for (const [cat, rs] of Object.entries(byCat)) {
  catStats[cat] = {
    total: rs.length,
    passed: rs.filter(r => r.verdict === 'PASS').length,
    score: rs.reduce((s, r) => s + r.score, 0) / rs.length
  };
}

const siteFailures = Object.entries(bySite)
  .map(([site, rs]) => [site, rs.filter(r => r.verdict === 'FAIL').length])
  .sort((a, b) => b[1] - a[1]).slice(0, 20);

console.log(JSON.stringify({
  total, passed, partial, failed,
  totalScore, avgScore: total ? totalScore / total : 0,
  totalDurationMs, avgDurationMs,
  totalInputTokens, totalOutputTokens, totalTokens,
  byCategory: catStats,
  bySiteFailures: Object.fromEntries(siteFailures),
  blockers
}, null, 2));
"

Report Template

Write web-bench-report.md with this structure:

# WebBench Benchmark Report

**Date:** {timestamp}
**Agent:** {agent identifier}
**Dataset:** Halluminate/WebBench
**Tasks evaluated:** {total} / 2454

---

## Overall Results

| Metric | Value |
|--------|-------|
| **Pass Rate** | {passed}/{total} ({pass_pct}%) |
| **Partial Rate** | {partial}/{total} ({partial_pct}%) |
| **Fail Rate** | {failed}/{total} ({fail_pct}%) |
| **Average Score** | {avg_score:.2f} / 1.0 |
| **Total Duration** | {total_duration_formatted} |
| **Avg Duration/Task** | {avg_duration_formatted} |
| **Total Tokens** | {total_tokens:,} ({input_tokens:,} input + {output_tokens:,} output) |
| **Avg Tokens/Task** | {avg_tokens:,} |

## Results by Category

| Category | Total | Pass | Partial | Fail | Pass Rate | Avg Score |
|----------|-------|------|---------|------|-----------|-----------|
| READ | ... | ... | ... | ... | ...% | ... |
| CREATE | ... | ... | ... | ... | ...% | ... |
| UPDATE | ... | ... | ... | ... | ...% | ... |
| DELETE | ... | ... | ... | ... | ...% | ... |
| FILE_MANIPULATION | ... | ... | ... | ... | ...% | ... |

## Timing Breakdown

| Category | Avg Duration | Min | Max |
|----------|-------------|-----|-----|
| READ | ... | ... | ... |
| CREATE | ... | ... | ... |
| ... | | | |

## Token Usage Breakdown

| Category | Avg Input Tokens | Avg Output Tokens | Avg Total |
|----------|-----------------|-------------------|-----------|
| READ | ... | ... | ... |
| CREATE | ... | ... | ... |
| ... | | | |

## Top Failure Modes

| Failure Mode | Count | % of Failures |
|-------------|-------|---------------|
| Auth required | ... | ... |
| CAPTCHA | ... | ... |
| Site unavailable | ... | ... |
| Navigation failure | ... | ... |

## Worst Performing Websites (by failure count)

| Website | Tasks | Failures | Failure Rate |
|---------|-------|----------|-------------|
| ... | ... | ... | ... |

## Best Performing Websites (by pass rate, min 3 tasks)

| Website | Tasks | Pass Rate | Avg Score |
|---------|-------|-----------|-----------|
| ... | ... | ... | ... |

## Sample Failures

{Show 5-10 representative failures with task description, what went wrong, and verdict reasoning}

## Methodology

- **Execution:** One task per session via agent-web-interface browser automation
- **Evaluation:** LLM-as-judge with structured rubric (PASS=1.0, PARTIAL=0.5, FAIL=0.0)
- **Scoring dimensions:** Navigation, Comprehension, Completeness, Accuracy, Confirmation
- **Infrastructure blockers** (auth, CAPTCHA, site down) scored as FAIL but flagged separately

Output

File: web-bench-report.md in working directory
Report should be self-contained and readable without the raw JSONL data

Guardrails

Use the JSONL file as the sole source of truth — do not fabricate statistics
Format all durations as human-readable (e.g., "2h 34m 12s" not "9252000ms")
Format token counts with thousands separators
Round percentages to one decimal place
If results file has fewer than the expected total tasks, note this prominently in the report header

More from this repository

same repository

capture-feature-evidence

lespaceman/athena-workflow-marketplace

Deeply explore a live app feature or a single mapped sub-feature and capture grounded evidence for downstream planning in `docs/qa/exploration-report.md` or a scoped file under `docs/qa/exploration/`. Use after the requested scope is already known, especially after `map-feature-scope` has identified bounded sub-features for a large feature. This skill owns observed product evidence and blockers for the scoped area; it does not perform feature decomposition, frame exploratory charters, prioritize risks, or write coverage plans.

2026-07-014

map-feature-scope

lespaceman/athena-workflow-marketplace

Quickly map a broad product feature into concrete sub-features before deep exploration begins. Use when a requested feature may span multiple routes, tabs, overlays, roles, or primary interactive surfaces and the agent needs to identify bounded exploration units, shared state, and blockers before spawning parallel browser subagents. This skill owns `docs/qa/feature-map.md`; it does not produce the final deep exploration evidence, coverage plan, or test specs.

2026-07-014

exploratory-test-writer

lespaceman/athena-workflow-marketplace

Define an exploratory testing charter: frame what to investigate, why it matters, and which risk hypotheses to test before or alongside the shared testing workflow. Triggers include "exploratory testing for...", "what should I investigate in this app", "help me think about what could go wrong", "risk hypotheses for...", "testing charter for...", "where should I probe deeper", or any request that needs structured thinking about test intent before coverage planning begins. This skill owns exploratory intent, risk framing, investigation order, and concrete follow-up asks for missing evidence; it does NOT own exploration evidence (capture-feature-evidence), final coverage prioritization (plan-test-coverage), or test case specs (generate-test-cases).

2026-07-014

add-playwright-tests

lespaceman/athena-workflow-marketplace

THE DEFAULT ENTRY POINT for the Playwright execution layer in the split testing suite. Use to add, create, or set up Playwright end-to-end tests for a feature, page, or application. It orchestrates the full Playwright workflow after shared exploration and planning: analyze the Playwright codebase, consume `docs/qa/feature-map.md`, `docs/qa/exploration-report.md`, `docs/qa/coverage-plan.md`, and `docs/qa/test-cases/*.md`, write production-grade test code, review it, and verify it with real execution. Delegates to `analyze-test-codebase`, `write-test-code`, `review-test-code`, and `fix-flaky-tests`, while relying on shared `map-feature-scope`, `capture-feature-evidence`, `plan-test-coverage`, `generate-test-cases`, and `review-test-cases`.

2026-07-014

analyze-test-codebase

lespaceman/athena-workflow-marketplace

Scans and reports on an existing Playwright test codebase. This skill should be used to inspect the current Playwright setup before writing, reviewing, or fixing tests, and should be loaded early when working in a new project to understand existing patterns. Covers: Playwright config, page objects, fixtures, helpers, auth/globalSetup patterns, test conventions, and directory structure. Triggers: "understand", "check", "show me", "inspect", or "analyze" the current test setup, config, infrastructure, patterns, or conventions. In the full add-tests workflow, this skill serves as an early read-only analysis sub-step before planning detailed coverage or writing code. This skill examines existing files and outputs a structured report. It does NOT write or fix tests, install Playwright, or explore live websites — for live-site exploration, use `capture-feature-evidence` so evidence is captured in `docs/qa/exploration-report.md`.

2026-07-014

review-test-code

lespaceman/athena-workflow-marketplace

Quality review of Playwright test code before final execution signoff. This skill should be used when implementation review of executable Playwright tests is needed, not for diagnosis of runtime flakiness. Triggers: "review test code", "review Playwright tests", "check test quality", "audit test implementation", "review my tests before merging", "check test code for issues", "review e2e tests", "code review Playwright", "are my tests stable", "check for brittle selectors", "review before running tests". Quality gate after write-test-code — catches brittle selectors, force:true misuse, networkidle overuse, Tailwind utility class selectors, exact numeric assertions, missing teardown, parallel-unsafe mutations, hardcoded data, missing assertions, test coupling, and convention divergence. Review-only — does NOT rewrite tests, does NOT run tests. Use fix-flaky-tests for fixing, write-test-code for rewriting.

2026-07-014

name	generate-report
description	Aggregate WebBench benchmark results into a comprehensive evaluation report. Triggers: "generate report", "create benchmark report", "summarize results", "aggregate scores", "produce evaluation report". Reads web-bench-results.jsonl, computes statistics by category/website/failure mode, and writes web-bench-report.md with pass rates, timing, token usage, and analysis. Does NOT execute or evaluate tasks — only aggregates existing results.
allowed-tools	Read Write Edit Bash Glob Grep

Generate WebBench Benchmark Report

Aggregate all results from web-bench-results.jsonl into a comprehensive markdown report.

Input

Results file: web-bench-results.jsonl — one JSON line per completed task

Result line schema:

{
  "id": 42,
  "url": "https://acehardware.com",
  "category": "READ",
  "task": "Navigate to...",
  "score": 1.0,
  "verdict": "PASS",
  "reasoning": "Successfully extracted all specs",
  "error": null,
  "duration_ms": 34200,
  "tokens_used": {"input": 12450, "output": 3200},
  "timestamp": "2026-03-19T14:30:00Z"
}

Report Generation

Use Node.js to compute all statistics:

node -e "
const fs = require('fs');
const results = fs.readFileSync('web-bench-results.jsonl','utf-8').trim().split('\n')
  .filter(l => l.trim()).map(JSON.parse);

const total = results.length;
const passed = results.filter(r => r.verdict === 'PASS').length;
const partial = results.filter(r => r.verdict === 'PARTIAL').length;
const failed = results.filter(r => r.verdict === 'FAIL').length;
const totalScore = results.reduce((s, r) => s + r.score, 0);

// Timing
const totalDurationMs = results.reduce((s, r) => s + (r.duration_ms || 0), 0);
const avgDurationMs = total ? totalDurationMs / total : 0;

// Tokens
const totalInputTokens = results.reduce((s, r) => s + ((r.tokens_used || {}).input || 0), 0);
const totalOutputTokens = results.reduce((s, r) => s + ((r.tokens_used || {}).output || 0), 0);
const totalTokens = totalInputTokens + totalOutputTokens;

// By category
const byCat = {};
for (const r of results) {
  if (!byCat[r.category]) byCat[r.category] = [];
  byCat[r.category].push(r);
}

// By website (top failures)
const bySite = {};
for (const r of results) {
  if (!bySite[r.url]) bySite[r.url] = [];
  bySite[r.url].push(r);
}

// Failure modes
const blockers = {};
for (const r of results) {
  if (r.error) blockers[r.error] = (blockers[r.error] || 0) + 1;
}

const catStats = {};
for (const [cat, rs] of Object.entries(byCat)) {
  catStats[cat] = {
    total: rs.length,
    passed: rs.filter(r => r.verdict === 'PASS').length,
    score: rs.reduce((s, r) => s + r.score, 0) / rs.length
  };
}

const siteFailures = Object.entries(bySite)
  .map(([site, rs]) => [site, rs.filter(r => r.verdict === 'FAIL').length])
  .sort((a, b) => b[1] - a[1]).slice(0, 20);

console.log(JSON.stringify({
  total, passed, partial, failed,
  totalScore, avgScore: total ? totalScore / total : 0,
  totalDurationMs, avgDurationMs,
  totalInputTokens, totalOutputTokens, totalTokens,
  byCategory: catStats,
  bySiteFailures: Object.fromEntries(siteFailures),
  blockers
}, null, 2));
"

Report Template

Write web-bench-report.md with this structure:

# WebBench Benchmark Report

**Date:** {timestamp}
**Agent:** {agent identifier}
**Dataset:** Halluminate/WebBench
**Tasks evaluated:** {total} / 2454

---

## Overall Results

| Metric | Value |
|--------|-------|
| **Pass Rate** | {passed}/{total} ({pass_pct}%) |
| **Partial Rate** | {partial}/{total} ({partial_pct}%) |
| **Fail Rate** | {failed}/{total} ({fail_pct}%) |
| **Average Score** | {avg_score:.2f} / 1.0 |
| **Total Duration** | {total_duration_formatted} |
| **Avg Duration/Task** | {avg_duration_formatted} |
| **Total Tokens** | {total_tokens:,} ({input_tokens:,} input + {output_tokens:,} output) |
| **Avg Tokens/Task** | {avg_tokens:,} |

## Results by Category

| Category | Total | Pass | Partial | Fail | Pass Rate | Avg Score |
|----------|-------|------|---------|------|-----------|-----------|
| READ | ... | ... | ... | ... | ...% | ... |
| CREATE | ... | ... | ... | ... | ...% | ... |
| UPDATE | ... | ... | ... | ... | ...% | ... |
| DELETE | ... | ... | ... | ... | ...% | ... |
| FILE_MANIPULATION | ... | ... | ... | ... | ...% | ... |

## Timing Breakdown

| Category | Avg Duration | Min | Max |
|----------|-------------|-----|-----|
| READ | ... | ... | ... |
| CREATE | ... | ... | ... |
| ... | | | |

## Token Usage Breakdown

| Category | Avg Input Tokens | Avg Output Tokens | Avg Total |
|----------|-----------------|-------------------|-----------|
| READ | ... | ... | ... |
| CREATE | ... | ... | ... |
| ... | | | |

## Top Failure Modes

| Failure Mode | Count | % of Failures |
|-------------|-------|---------------|
| Auth required | ... | ... |
| CAPTCHA | ... | ... |
| Site unavailable | ... | ... |
| Navigation failure | ... | ... |

## Worst Performing Websites (by failure count)

| Website | Tasks | Failures | Failure Rate |
|---------|-------|----------|-------------|
| ... | ... | ... | ... |

## Best Performing Websites (by pass rate, min 3 tasks)

| Website | Tasks | Pass Rate | Avg Score |
|---------|-------|-----------|-----------|
| ... | ... | ... | ... |

## Sample Failures

{Show 5-10 representative failures with task description, what went wrong, and verdict reasoning}

## Methodology

- **Execution:** One task per session via agent-web-interface browser automation
- **Evaluation:** LLM-as-judge with structured rubric (PASS=1.0, PARTIAL=0.5, FAIL=0.0)
- **Scoring dimensions:** Navigation, Comprehension, Completeness, Accuracy, Confirmation
- **Infrastructure blockers** (auth, CAPTCHA, site down) scored as FAIL but flagged separately

Output

File: web-bench-report.md in working directory
Report should be self-contained and readable without the raw JSONL data

Guardrails

Use the JSONL file as the sole source of truth — do not fabricate statistics
Format all durations as human-readable (e.g., "2h 34m 12s" not "9252000ms")
Format token counts with thousands separators
Round percentages to one decimal place
If results file has fewer than the expected total tasks, note this prominently in the report header