| name | cost-latency-analysis |
| description | How to compute model latency and estimated $ cost from PilotSwarm
observability data. Read this before reporting that a model is
"slow" or "expensive" — most apparent slowness is orchestration
overhead, not model inference, and most cost numbers are guesses
unless they reference a real published price card.
|
Cost & Latency Analysis
You are the agent-manager. When investigating reliability, cost, or
performance, follow this skill.
Latency: prefer assistant.usage.duration
PilotSwarm records two different "durations" per turn. Do not confuse
them:
| Source | What it measures | When to use |
|---|
runTurn activity span (execution history) | Total wall-clock time the activity ran, including dehydrate, hydrate, snapshot, blob I/O, scheduling. | Operator-facing "how long did this turn take end-to-end". Useful for orchestration-overhead investigations. |
assistant.usage.duration (assistant event) | Time spent inside the model call itself as reported by the LLM provider. | Model-latency comparisons. The only fair number to use when comparing models, providers, or context sizes. |
runTurn spans can materially overstate model latency — sometimes
2–5× — because they include dehydrate/hydrate, snapshot serialization,
blob storage round-trips, retry backoff, and tool-execution time.
Rule of thumb:
- Comparing "is gpt-5.4 slower than gpt-5.4-mini?" → use
assistant.usage.duration.
- Investigating "why does this turn take 30 seconds?" when the model
number is small → look at the
runTurn span and compare to the
assistant span. The delta is the orchestration overhead.
Where to read it from
- Per-turn:
read_agent_events filtered to event_types: ["assistant"],
then read usage.duration (often in milliseconds — confirm units in
the actual payload, do not assume).
- For roll-ups, request a derived field on the management surface and
expose it as a tool (see the Observability Surface for the Agent
Tuner rule in
.github/copilot-instructions.md). Do not summarize
latency by averaging runTurn spans — it will mislead.
Cost: estimate, do not guess
Token counts come from read_session_metric_summary /
read_fleet_stats and are reliable. Per-token prices change
constantly and do not live in PilotSwarm. Always derive cost from a
linked, dated snapshot of each provider's price card.
Default approach:
- Read the model name from the metric summary (or from the assistant
event's
model field for per-turn cost).
- Look up the per-million-token input + output price from the
provider's published page (links below). Note the date you looked
it up.
- Cost = (
tokens_input × $/M-input + tokens_output × $/M-output)
÷ 1,000,000.
- If the model offers prompt caching (Claude, GPT-5.4 family), apply
the discounted cache-read rate to
tokens_cache_read. Cache writes
are often billed at standard input rate.
- Report the price source and date alongside the dollar figure.
Stable price-card sources
These are the canonical pages to consult. Do not invent or memoize
numbers — re-fetch on each report.
Example
session: 22013ffb
model: azure-openai:gpt-5.4
tokens: input 28,634 output 4,224 cache_read 16,700
report:
- input cost: 28634 × $X/M = $...
- output cost: 4224 × $Y/M = $...
- cache-read cost: 16700 × $Z/M = $...
total ≈ $0.0XX (price source: openai.com/api/pricing, fetched <date>)
What to never do
- Never quote a per-token dollar cost without naming the price source
and the date you fetched it.
- Never compare model latency using
runTurn spans alone.
- Never claim Copilot per-token cost in dollars — Copilot pricing is
not per token.
- Never average across mixed providers without tagging each row by
model and provider — you will average $30/M-token Opus calls with
$0.10/M-token nano calls and report a number that is meaningless.
