| name | latency-budgeting |
| description | Allocate and validate an end-to-end latency objective across serial and parallel work, network, queues, dependencies, computation, rendering, retries, and tail behavior using measured service contributions. |
Latency Budgeting
Use when a user/system journey has a latency target that spans multiple components or when teams need to know where optimization effort matters.
Procedure
- Define the end-to-end operation, start/finish boundaries, target percentile/distribution, workload, and whether the objective concerns response, first useful result, completion, frame, or another latency definition.
- Trace the critical path and identify work that is serial, parallel, queued, cached, speculative, or asynchronous/outside the user's wait.
- Measure current latency contribution at each major boundary under representative conditions, including client/network, gateway, services, databases, external APIs, queues, serialization, and rendering as relevant.
- Account for tail amplification. Several dependencies with individually acceptable percentiles can produce worse end-to-end tail behavior, especially under fan-out.
- Allocate budgets based on technical/product constraints and optimization opportunity rather than dividing the total evenly by component count.
- Include queueing, retries, timeouts, connection/setup, cache misses, cold starts, and other paths that consume the objective under realistic failure/scale conditions.
- Define component-level indicators/gates that reveal when one layer consumes too much of the end-to-end budget while preserving the overall user outcome as the authority.
- Optimize or redesign the dominant serial/variable contributions first, then remeasure because improving one layer can expose another.
- Revisit budgets when architecture, workload, geography, providers, or product expectations change; do not fossilize an early allocation as a permanent SLA.
Decision rules
- End-to-end latency is not the sum of every component duration when work overlaps; use the actual critical path.
- Tail latency and variance often matter more than average latency for user experience and timeout policy.
- A component meeting its local budget does not excuse a missed user-level objective.
- Fleet placement can affect network/data-locality latency, but placement policy remains Fleet-owned; report measurable constraints or preferences.
Quality gate
The latency budget is useful when the end-to-end boundary and percentile are explicit, critical-path contributions are measured, tail/queue/retry behavior is included, allocations reflect real constraints, local signals connect to the user objective, and optimization priorities follow the measured largest contributors.
