| name | hgraf-add-gantt-overlay |
| description | Pattern for adding a new per-agent Gantt overlay — client telemetry source → proto field → server storage → bus delta → frontend renderer layer → uiStore toggle. |
hgraf-add-gantt-overlay
When to use
You want to add a new per-agent time-series overlay drawn inside the Gantt lanes — examples: context window fill (already exists; see ContextWindowSample in proto/harmonograf/v1/types.proto and bus.py:25-36 DELTA_CONTEXT_WINDOW_SAMPLE), token rate, error density, thinking depth, memory footprint. An overlay is a sequence of (agent_id, timestamp, value) samples, rendered as a sparkline or heatmap inside or above each agent's row.
If your data is span-scoped (per-tool-call), you do not want an overlay — you want a span attribute or a span kind (see hgraf-add-proto-field). Overlays are for continuous or sampled signals.
Prerequisites
make proto works and you understand the three-way regen flow (see hgraf-add-proto-field).- You have read the Gantt renderer at
frontend/src/gantt/renderer.ts:1-97 — three canvas layers (background / blocks / overlay) with shared viewport state.
- You understand the
SessionBus fan-out at server/harmonograf_server/bus.py:65-120.
Step-by-step
1. Client: emit the sample
Pick where the data is produced. For LLM-centric signals, the hook is usually in client/harmonograf_client/adk.py around the after_model_callback or before_model_callback seams. For process-centric signals (CPU, memory), add a periodic sampler to client/harmonograf_client/heartbeat.py — it already runs on a timer and can piggyback sample emission onto the heartbeat tick.
Use the existing ContextWindowSample plumbing as the template. Grep for ContextWindowSample across client/harmonograf_client/ and server/harmonograf_server/ to see every site you will touch.
2. Proto: add the sample message
Edit proto/harmonograf/v1/types.proto:
message TokenRateSample {
string agent_id = 1;
google.protobuf.Timestamp at = 2;
double tokens_per_second = 3;
int64 window_ms = 4;
}
Add an entry on the telemetry-up oneof in proto/harmonograf/v1/telemetry.proto so clients can stream it. Add a frontend-facing wrapper in frontend.proto that the server pushes out on WatchSession updates (pattern: one of the SessionUpdate variants — see the existing ContextWindowSample field).
Run make proto and verify all three generation targets produced output.
3. Server: storage
Add a storage dataclass next to ContextWindowSample in server/harmonograf_server/storage/base.py. Add an insert method in the sqlite store (storage/sqlite.py). Add a new table to SCHEMA at sqlite.py:45-170 (or a new column to an existing table if the signal is sparse and you want per-span attribution).
Add a DELTA_TOKEN_RATE_SAMPLE constant to server/harmonograf_server/bus.py:25-36 and publish it from the ingest path (ingest.py) when the client sends a new sample.
4. Server: ingest + fan-out
In server/harmonograf_server/ingest.py, find the oneof dispatch in the StreamTelemetry handler. Add a branch for your new message type that:
- Persists to storage (via the store method you added).
- Publishes a
Delta on the per-session SessionBus with your new DELTA_* kind.
The bus fan-out is already wired — WatchSession subscribers pick up your delta automatically as long as the SessionUpdate message in frontend.proto has a field for it. If not, add the field and add a mapping in server/harmonograf_server/rpc/frontend_service.py (or equivalent — grep for WatchSession).
5. Frontend: receive and store
frontend/src/rpc/hooks.ts and frontend/src/rpc/convert.ts handle the inbound stream. Add a mapping from the new pb variant to a domain type, and append samples into a new store (create frontend/src/state/tokenRateStore.ts following the pattern of sessionsStore.ts:6-18).
For high-frequency samples, keep the store size bounded — use a ring buffer keyed on agent_id. The existing context-window implementation should be a good template.
6. Frontend: renderer layer
Extend frontend/src/gantt/renderer.ts. There are three canvas layers (lines 1-97): background, blocks, overlay. You have two options:
A. Overlay on the existing overlay canvas. Preferred when the signal is a thin sparkline that should float above the blocks. Add a draw call in the overlay pass, subscribe to your new store, translate (timestamp → x) using viewport state, translate (value → y within the agent's row).
B. New dedicated canvas. Required if your overlay needs independent hit-testing or repaints on a different cadence. Add a fourth canvas layer, initialize it in GanttCanvas.tsx, and give it its own RAF loop so background repaints don't trigger yours.
Tradeoff: Option A is simpler but couples your repaint rate to the existing overlay (runs on every hover change — potentially wasteful). Option B is harder to wire but decouples. Start with A and migrate to B only if performance profiling shows the overlay dominating.
Hit-testing (hover tooltips for overlay samples) goes through spatialIndex.ts — add a new bucket for your overlay points if you want them clickable.
7. Frontend: uiStore toggle
Add a boolean toggle to frontend/src/state/uiStore.ts (pattern: readTaskPlanVisible/writeTaskPlanVisible at lines 18-89 — they persist to localStorage):
const TOKEN_RATE_OVERLAY_KEY = "harmonograf.ui.tokenRateOverlay";
const readTokenRateOverlay = () => localStorage.getItem(TOKEN_RATE_OVERLAY_KEY) === "1";
const writeTokenRateOverlay = (v: boolean) => localStorage.setItem(TOKEN_RATE_OVERLAY_KEY, v ? "1" : "0");
Add the field and setter to UiState in the same file. Wire a toggle into the AppBar or a view-options drawer (frontend/src/components/shell/AppBar.tsx). The renderer reads the flag and skips the overlay draw call when disabled.
8. Tests
- Client: round-trip test in
client/tests/test_transport_protocol.py — assert the new message serializes and the transport mock delivers it.
- Server: storage round-trip test + bus delivery test (e.g.
server/tests/test_bus.py if present, else test_ingest.py).
- Frontend:
frontend/src/__tests__/ vitest for the store ingestion + a snapshot/regression test of the renderer draw path if one exists.
Verification
make proto
cd client && uv run --with pytest --with pytest-asyncio python -m pytest -q
cd server && uv run --with pytest --with pytest-asyncio python -m pytest -q
cd frontend && pnpm lint && pnpm build
make demo
Common pitfalls
- Coordinate-space bugs. The Gantt has multiple coordinate systems: (wall-clock ms → viewport pixels), (lane index → row y), (sample value → overlay y). Every new overlay reinvents the last one; read
renderer.ts and viewport.ts carefully and reuse the helpers. A 3px offset that looks fine at one zoom will be hilariously wrong at another.
- Repaint cost. If you naively call the overlay draw on every store update (e.g. 10Hz samples per agent × 8 agents = 80Hz), you will tank the RAF loop. Batch into the existing RAF; do not schedule your own.
- Store unbounded growth. A 30-minute session at 1Hz per agent is trivial; a 30-minute session at 100Hz per agent × 8 agents is 1.4M samples. Use a ring buffer. See
client/harmonograf_client/buffer.py for a pattern.
- Backpressure. The bus drops samples under load (see
Subscription.dropped counter at bus.py:51-63). Your overlay will have holes; do not draw fake zeros — render gaps.
- SessionBus delta kind collision. Pick a
DELTA_* constant name that does not collide with existing ones (bus.py:25-36). The protocol is string-keyed; collisions fail silently.
- uiStore persistence key. If you reuse an existing localStorage key prefix, you will silently read the old value as your new toggle. Use a dedicated, specific key.
- Gantt row height changes. If your overlay needs vertical space, you will be tempted to grow the row height. Row heights are defined near
renderer.ts:55-96 and are used by the spatial index, the minimap (frontend/src/components/Gantt/Minimap.tsx), and the hover hit-testing. Changing them has ripple effects — audit all three before committing.
- Missing schema version bump. If you added a sqlite table, old databases will not have it. Follow the migration guidance in
hgraf-add-proto-field step 5 — ALTER TABLE / CREATE TABLE IF NOT EXISTS, never DROP.
