| name | run-metrics |
| description | Run metrics reports for APIView Copilot. Use for: run metrics, metrics report, generate metrics, monthly metrics, metrics for March, metrics for January, adoption metrics, comment quality, quality trends, save metrics, metrics charts. |
| argument-hint | Month name (e.g. 'March') or date range (e.g. '2025-03-01 to 2025-03-31') |
Run Metrics
When to Use
- Generating monthly or custom-range metrics reports
- Reviewing adoption rates and comment quality across languages
- Producing charts for metrics visualization
- Reviewing companion quality trends across recent months
- Saving finalized metrics to the database (only when user explicitly requests)
Defaults
Unless the user says otherwise, always apply these defaults:
- Environment:
production
- Charts: Always include
--charts
- Languages: All languages (do not pass
--exclude)
- Save: Do NOT pass
--save unless the user explicitly asks to save (e.g. "save it", "persist", "write to DB")
- Format: JSON output (UTF-8 encoded)
- Quality trends follow-up: If the user asks for metrics but does not explicitly ask for quality trends, ask a short follow-up after sharing the metrics results: "Do you want the quality trends chart for the same period too?"
- Quality trends auto-run: If the user explicitly asks for quality trends, comment trends, or asks broadly for charts/trends/quality, also run the companion
quality-trends report for the same period.
Date Resolution
The user will typically specify a calendar month by name (e.g. "March", "January 2025"). Resolve to the full month date range:
| User says | start_date | end_date |
|---|
| "March" (current year) | YYYY-03-01 | YYYY-03-31 |
| "January 2025" | 2025-01-01 | 2025-01-31 |
| "March 1 to March 15" | YYYY-03-01 | YYYY-03-15 |
| "2025-06-01 to 2025-06-30" | 2025-06-01 | 2025-06-30 |
When only a month name is given without a year, use the current year. Be careful with month lengths (28/29/30/31 days).
Running Metrics
Step 1: Run the Command
Show the resolved command and run it immediately in a single foreground terminal invocation with a 180-second timeout (timeout: 180000). Before running, clean up stale output so you never accidentally present old results.
Important: Do NOT use 2>&1 — that merges stderr log messages (e.g. "Saved: output\charts\adoption.png") into the JSON output file, producing invalid JSON. Pipe stdout through Out-File -Encoding UTF8 so the output file is always valid UTF-8 JSON.
Full terminal command (cleanup + run):
New-Item -ItemType Directory -Path output -Force | Out-Null; if (Test-Path output/metrics_output.json) { Remove-Item output/metrics_output.json }; if (Test-Path output/charts) { Remove-Item output/charts/* -Force }; python cli.py report metrics -s <start_date> -e <end_date> --charts | Out-File -Encoding UTF8 output/metrics_output.json
After the command completes, read the output file with read_file to get the JSON results — do NOT use a separate terminal command. Then use view_image (not a terminal command) to display the chart PNGs. This means the entire workflow requires only one terminal invocation.
Examples
# March 2025, production, all languages, with charts (typical request)
python cli.py report metrics -s 2025-03-01 -e 2025-03-31 --charts
# Staging environment
python cli.py report metrics -s 2025-03-01 -e 2025-03-31 --charts --environment staging
# Exclude specific languages
python cli.py report metrics -s 2025-03-01 -e 2025-03-31 --charts --exclude Java Golang
# Custom date range
python cli.py report metrics -s 2025-03-10 -e 2025-03-20 --charts
# Save to database (ONLY when user explicitly requests)
python cli.py report metrics -s 2025-03-01 -e 2025-03-31 --charts --save
Companion Quality Trends
The quality-trends report is the companion chart view for metrics work.
- If the user explicitly asks for quality trends, comment trends, or a broader trends/chart view, run it automatically for the same time window.
- If the user only asked for metrics, ask whether they want the quality trends chart too after presenting the metrics result.
Translating the date window
The quality-trends command uses --months plus an optional --end-date, not a start date.
Convert the requested metrics window into an inclusive calendar-month count:
| Window | Use for --months |
|---|
| 2026-03-01 to 2026-03-31 | 1 |
| 2026-03-01 to 2026-03-15 | 1 |
| 2026-01-15 to 2026-03-02 | 3 |
Use the same resolved end date from the metrics request for --end-date.
Quality trends command
if (Test-Path output/charts/comment_bucket_trends.png) { Remove-Item output/charts/comment_bucket_trends.png -Force }; python cli.py report quality-trends --months <month_count> --end-date <end_date>
Optional additions:
- --environment staging
- --languages Python CSharp C# Java TypeScript JavaScript Go
- --exclude-human
- --neutral
After it completes, summarize the terminal output and use view_image to show the saved chart at output/charts/comment_bucket_trends.png.
Follow-up: "Save it"
If the user asks to save after a metrics run (e.g. "okay save it", "persist that", "write to DB"), re-run the same command with --save appended. Keep all other flags identical to the previous run.
Available Flags
| Flag | Type | Default | Description |
|---|
--start-date / -s | string | required | Start date (YYYY-MM-DD) |
--end-date / -e | string | required | End date (YYYY-MM-DD) |
--environment | string | production | production or staging |
--exclude | list | none | Space-separated language names to exclude |
--charts | flag | off | Generate PNG charts to output/charts/ |
--save | flag | off | Persist metrics to Cosmos DB (never use unless user explicitly requests) |
Chart Outputs
When --charts is enabled, 4 PNGs are saved to output/charts/:
- adoption.png — Copilot vs non-Copilot reviews by language
- comment_quality.png — Comment quality breakdown (upvoted, implicit good/bad, downvoted, deleted)
- human_copilot_split.png — Human vs AI comment split
- human_comments_comparison.png — Human comments with vs without Copilot
Gotchas
- Redirect stdout to a file: Always pipe through
| Out-File -Encoding UTF8 output/metrics_output.json and read the file afterward. Do NOT use > which produces UTF-16 in PowerShell 5.1.
- Clean up before running: Delete the previous
output/metrics_output.json and output/charts/* before running. This prevents presenting stale results if the command fails silently.
- Do NOT use
2>&1: This merges stderr log lines (e.g. "Saved: ...") into the JSON output file, producing invalid JSON. Only redirect stdout.
- Use foreground with timeout: Run with
isBackground=false and timeout: 180000 (3 min). Do NOT use isBackground=true and poll — that causes repeated user approval prompts.
- Use
New-Item -ItemType Directory not mkdir: mkdir is aliased differently across shells. Use New-Item -ItemType Directory -Path output -Force | Out-Null for reliable directory creation.
- Read results with
read_file and view_image: After the command finishes, use read_file for the JSON and view_image for charts. Do NOT launch additional terminal commands to read the file.
- Use
python cli.py not .\avc: The avc.bat script may resolve to system Python. Use python cli.py report metrics ... to ensure the correct environment.
- Quality trends is separate: It uses
report quality-trends with --months and optional --end-date, not --start-date.
- Month end dates: February has 28/29 days, April/June/Sept/Nov have 30 days. Get it right.
- Built-in exclusions:
c, c++, typespec, swagger, xml are always excluded automatically — no need to add them.
