reports-and-explainability

name	reports-and-explainability
description	Add metrics to execution reports or debug the explainability pipeline. Use when modifying pkg/explain data types, updating HTML report generation, or wiring new scorer signals into the report output. Covers per-hunt reports and aggregate index.html.

Reports and Explainability

ManulEngine (Go) produces two report artifacts:

1. Per-hunt HTML report — one file per hunt run, generated by report.GenerateHTML().
2. Aggregate index.html — a summary table of all hunts in a parallel run, generated by report.GenerateIndex().

Both are pure string-builder HTML — no template engine, no external dependencies.

Data flow

runtime.executeCommand
        │
        ▼
explain.ExecutionResult  (per-command)
        │
        ▼
explain.HuntResult       (aggregated per-hunt)
        │
        ├─► report.GenerateHTML()   → report_{{title}}_{{timestamp}}_{{seq}}.html
        └─► report.GenerateIndex()  → index.html

explain package (pkg/explain/explain.go)

ExecutionResult — per-command record

Key fields:

• Step — original DSL text
• CommandType — classified kind (navigate, click, fill, verify, ...)
• TargetQuery, TypeHint — what the scorer was asked to find
• CandidatesConsidered — total elements from the probe
• RankedCandidates []Candidate — top-N with full ScoreBreakdown
• WinnerXPath, WinnerScore — chosen element
• ActionPerformed, ActionValue — what was executed
• Success, Error — outcome
• DurationMS — wall-clock time
• ScreenshotPath — optional screenshot reference
• ProbeMetadata map[string]any — optional debug metadata from JS probe

HuntResult — per-hunt aggregate

Key fields:

• Title, Context, HuntFile — from @title / @context headers
• TotalSteps, Passed, Failed — counters
• Results []ExecutionResult — in execution order
• TotalDurationMS — overall wall-clock time
• Success — true if all commands passed
• SoftErrors []string — accumulated VERIFY SOFTLY warnings

ScoreBreakdown — per-candidate scoring

16 fields documenting every signal contribution:

• Text channel: ExactTextMatch, NormalizedTextMatch, LabelMatch, PlaceholderMatch, AriaMatch, DataQAMatch
• Attributes: IDMatch
• Semantic: TagSemantics, TypeHintAlignment
• Penalty: VisibilityScore, InteractabilityScore
• Proximity: ProximityScore
• Final: Total, RawScore

Rule: If you add a new scoring signal, you MUST add it to ScoreBreakdown and update the HTML renderer so --explain remains reviewable.

Report generation (pkg/report/)

GenerateHTML(result, outDir) (html.go)

• Creates outDir if missing
• Filename: report_{{sanitized_title}}_{{timestamp}}_{{seq}}.html
• reportSeq is an atomic.Uint64 preventing overwrites when parallel workers run hunts with the same title in the same second
• HTML is hand-built via strings.Builder with embedded CSS

CSS class conventions:

• .result-ok / .result-fail — per-step outcome
• .stat.pass .num { green #2ecc71 } / .stat.fail .num { red #e74c3c }
• .soft-errors — yellow warning block for VERIFY SOFTLY
• .badge.pass / .badge.fail — overall hunt outcome badge

Screenshots are rendered as <img src="../{{ScreenshotPath}}"> — the ../ relativizes from the reports/ subdirectory to the working directory where screenshots are stored.

GenerateIndex(summaries, outDir) (index.go)

• Accepts []RunSummary where each entry links a HuntResult to its per-hunt report path
• Computes aggregate stats: total hunts, passed/failed, step counts, total duration
• Renders a dark-themed HTML table with worker IDs and report links
• filepath.Rel + filepath.ToSlash ensures cross-platform relative links

Adding a new metric — the checklist

If you add a new field to ExecutionResult or HuntResult:

• Add the field to pkg/explain/explain.go with appropriate json tags
• Populate it in pkg/runtime/runtime.go during command execution
• Update pkg/report/html.go to display the new data (if user-visible)
• Update pkg/report/index.go if the metric should appear in the aggregate table
• Ensure the field is included in any JSON serialization if external tools consume it

Common pitfalls

Mistake	Consequence
Adding a score signal but not to ScoreBreakdown	--explain can't justify the ranking; users lose trust
Forgetting atomic.Uint64 seq suffix	Parallel workers overwrite each other's reports
Absolute screenshot paths in HTML	Reports break when moved to another machine
Not escaping strings with html.EscapeString	XSS risk if hunt files contain user-controlled text
Modifying HuntResult fields without updating GenerateIndex	Aggregate table shows stale or zero values

Key files

• pkg/explain/explain.go — ExecutionResult, HuntResult, ScoreBreakdown, Candidate
• pkg/report/html.go — GenerateHTML, per-hunt styling
• pkg/report/index.go — GenerateIndex, aggregate table
• pkg/report/run_history.go — run history persistence
• pkg/runtime/runtime.go — where ExecutionResult values are populated