agentevals-website-contribution

name	agentevals-website-contribution
description	Contribution workflow for the agentevals-agent — branch naming, Hugo front matter, adding/editing docs pages, and PR conventions for the agentevals-dev/website repo.

AgentEvals Website Contribution Runbook

The agentevals-agent operates only on agentevals-dev/website (a custom Hugo docs site, no external theme, deployed to agentevals.dev via GitHub Pages). Never touch any other repo.

Golden rules

1. Read before write — get_file_contents on a file before create_or_update_file.
2. Never commit to the default branch — always work on a feature branch.
3. Confirm destructive ops — deleting branches, closing issues, merging, force-push.

Branch + commit conventions

• Branch names: docs/<topic>, fix/<topic>, feat/<topic> (e.g. docs/add-rubric-guide, fix/typo-quick-start).
• Conventional commits: docs:, fix:, feat:, chore:.
• Create the branch with create_branch off the default branch first.

Adding a new documentation page

1. create_branch.
2. Create content/docs/<slug>.md with required Hugo front matter:

   ---
   title: "Page Title"
   weight: <next-number>   # controls sidebar/order; check siblings first
   description: "One-line summary shown in the docs index card grid."
   ---
   

3. Body in Markdown. Use fenced code blocks for YAML/CLI examples.
4. push_files, then create_pull_request with a clear summary and any linked issue.

Existing docs and their weights: quick-start(1), configuration(2), examples(3), ci-cd(4), mcp-server(5), web-ui(6). New pages should pick an unused weight.

Editing existing content

• Doc pages: edit content/docs/*.md (keep front matter intact).
• Homepage: edit layouts/index.html (it's an HTML template, NOT markdown).
• Layouts: layouts/docs/list.html (index), layouts/docs/single.html (page), layouts/_default/baseof.html (shell + theme toggle).
• Site config: hugo.toml (params include github + discord links).

Pull request hygiene

• Title uses conventional-commit prefix.
• Body: what changed, why, and Closes #<issue> when applicable.
• Return the PR/issue URL to the user.
• Only merge_pull_request after explicit user confirmation.

AgentEvals domain quick reference (for writing accurate docs)

• EVAL.yaml is the core spec: name, version, description, execution.target, suite-level assert, and tests[] (each with id, criteria, input, rubrics).
• 12 evaluator types incl. code_judge, llm_judge, rubric, composite, tool_trajectory, field_accuracy, execution_metrics, skill_trigger.
• Verdicts: pass (≥0.8), borderline (≥0.6), fail (<0.6). Scores are 0.0–1.0.
• Aggregation: weighted_average (default), minimum, maximum, safety_gate, all_or_nothing.
• AgentV (agentv.dev) is the CLI that executes EVAL.yaml.