throttling-backoff-handler

name	throttling-backoff-handler
description	Playbook for designing a correct Graph API throttling (HTTP 429) response handler: reading Retry-After, exponential backoff with jitter, per-resource throttle budgets, and the SDK built-in retry middleware that makes hand-rolling unnecessary for most cases. Owned by graph-api-engineer.

Throttling Backoff Handler

When to invoke

• A Graph integration is receiving 429 Too Many Requests responses.
• Designing a new high-volume Graph integration and want to build throttle-handling in from the start.
• Evaluating whether to use SDK retry middleware or hand-roll backoff.
• A batch request returns 429 on some sub-requests.

The Graph throttling contract

Graph throttles per application, per tenant, per resource type, per time window ([verify-at-build] — limits change; see Microsoft Graph throttling guidance). When throttled:

• HTTP status: 429 Too Many Requests
• Header: Retry-After: <seconds> (always present; always honour it)
• You MUST wait at least Retry-After seconds before retrying — not "a few seconds."

Do not retry immediately. An immediate retry on a 429 consumes another throttle unit and restarts the window. This is the single most common cause of a 429 cascade.

Step 1 — Use the SDK's built-in retry handler

Before hand-rolling backoff, check whether the SDK handles it:

SDK	Built-in retry middleware
Microsoft Graph Python SDK (msgraph-sdk)	Yes — RetryHandler is included by default in GraphServiceClient
Microsoft Graph JS/TS SDK (@microsoft/microsoft-graph-client)	Yes — RetryHandlerOptions middleware
Microsoft Graph .NET SDK	Yes — RetryHandler in GraphClientFactory.CreateDefaultHttpHandlers()

If you are using the official SDK, enable the retry handler and configure its options:

from msgraph import GraphServiceClient
from msgraph.core import GraphClientFactory
from kiota_http.middleware import RetryHandler, RetryHandlerOption

retry_options = RetryHandlerOption(max_retries=5, retry_time_limit=180)
# The SDK's default client already includes RetryHandler — verify version-specific docs
client = GraphServiceClient(credential, ["https://graph.microsoft.com/.default"])

If the SDK does not provide a retry handler, or you are calling the REST API directly, implement the pattern below.

Step 2 — Correct exponential backoff with jitter

import time, random, httpx

def graph_get_with_retry(url: str, headers: dict, max_retries: int = 5) -> dict:
    attempt = 0
    while attempt <= max_retries:
        resp = httpx.get(url, headers=headers)

        if resp.status_code == 200:
            return resp.json()

        if resp.status_code == 429:
            retry_after = int(resp.headers.get("Retry-After", 30))
            # Exponential backoff: max(Retry-After, 2^attempt) + jitter
            wait = max(retry_after, (2 ** attempt)) + random.uniform(0, 1)
            print(f"Throttled. Waiting {wait:.1f}s (attempt {attempt+1}/{max_retries})")
            time.sleep(wait)
            attempt += 1
            continue

        if resp.status_code in (500, 502, 503, 504):
            # Transient server errors — retry with backoff
            wait = (2 ** attempt) + random.uniform(0, 1)
            time.sleep(wait)
            attempt += 1
            continue

        # Non-retriable error
        resp.raise_for_status()

     RuntimeError()

Key points:

• Always honour Retry-After as the floor — don't backoff less than what Graph specified.
• Add jitter (random offset) to prevent thundering-herd if multiple workers are throttled simultaneously.
• Cap retries — infinite retry loops can hold threads indefinitely and mask bugs.
• Log the wait — silent retries hide throttling from monitoring.

Step 3 — Per-resource throttle budgets

Different Graph resources have different throttle budgets (all [verify-at-build]):

Resource	Approximate limit	Notes
Users, Groups (directory)	12 000 req / 10 min per app per tenant	Higher for read-only
Mail, Calendar (Exchange Online)	10 000 req / 10 min per app	Per-mailbox sublimits apply
Files / SharePoint	1 200 req / min per app	Per-site sublimits
Teams	15 req / sec per app	Real-time messaging is especially tight

Do not treat all 429s the same — a Teams throttle and a directory throttle have different windows. Log the resource path alongside the 429 to identify which budget is exhausted.

Step 4 — Proactive throttle avoidance

Technique	Effect
$batch 20 requests per round-trip	Reduces round-trips; does not reduce throttle-unit consumption
$select explicit fields	Lighter responses; slightly lower service cost per request
Delta queries for "what changed"	Replaces N per-item polls with one differential query
Stagger workers with asyncio.sleep(0.1) between calls	Spreads load across the throttle window
Off-peak batch jobs	Avoid the 9 AM–12 PM usage spike when other apps share the tenant budget

Batch-level vs sub-request 429

When a $batch call has sub-requests that are throttled:

• The overall batch response is HTTP 200 (the batch call itself succeeded).
• Individual throttled sub-requests have status 429 in the responses array.
• Extract the Retry-After from the sub-request's headers object (not the top-level HTTP headers).
• Re-issue only the throttled sub-requests after the delay — do not re-send the whole batch.

Pitfalls

• Retrying on 429 immediately — the most common mistake; always sleep Retry-After seconds minimum.
• Treating Retry-After as advisory — it is a contract; Graph's throttling window does not reset until the specified time.
• Using a fixed sleep (e.g., time.sleep(1)) instead of reading Retry-After — under-waits on long throttle windows, over-waits when Retry-After is small.
• Not logging 429s — silent retries hide a growing throttle problem until it becomes a production incident.
• Using max_retries=0 to "fail fast" in a batch pipeline — one throttled call aborts the entire job; a few retries with correct backoff are almost always correct.