| name | eval-baseline |
| description | Run ADK evaluations and capture current scores as a baseline before making changes |
| argument-hint | [workitem-path] |
| allowed-tools | Read, Write, Bash, Glob |
Eval Baseline Capture
Capture eval scores for $ARGUMENTS.
Steps
- Find all evalset files:
ls tests/eval/data/*.evalset.json
- For each evalset, run:
adk eval mortgage_concierge tests/eval/data/<file> \
--config_file_path=tests/eval/data/test_config.json \
--print_detailed_results
- Parse output to extract:
tool_trajectory_avg_score
response_match_score
- Per-query breakdown
- Store results in
.workitems/<current>/eval-baseline.json:
{
"timestamp": "<ISO-8601>",
"evalsets": {
"<evalset-name>": {
"tool_trajectory_avg_score": 0.85,
"response_match_score": 0.72
}
},
"thresholds": {
"tool_trajectory_avg_score": 0.8,
"response_match_score": 0.7
}
}
- Report scores in a table format
Regression Check
When run AFTER changes, compare against existing baseline:
- Flag any metric that dropped below baseline
- Flag any metric that dropped below config thresholds
- Display delta table (before/after/change)