script-kit-devtools

name	script-kit-devtools
description	Agent-facing DevTools for Script Kit GPUI: use protocol, MCP, and CLI primitives to inspect, control, measure, debug, benchmark, and prove real app UI behavior from bug reports without defaulting to prewritten agentic-testing recipes.

Script Kit DevTools

This skill owns the agent-facing DevTools layer for Script Kit GPUI. It treats agentic-testing recipes as regression packs and smoke tests; the primary interface is direct protocol/MCP/CLI inspection, interaction, measurement, comparison, and proof against the real app.

Use this skill when a user reports a UX/UI bug, shares a screenshot, asks an agent to investigate the app like DevTools, or wants to expand the automation surface that agents use to debug Script Kit.

Core Model

Think Chrome DevTools for Script Kit, not a script catalog.

The loop is:

1. Intake the bug report, screenshot, observed behavior, expected behavior, surface hints, and safety constraints.
2. Form a lightweight hypothesis.
3. Open or reveal the real app through the real user entry path.
4. Use DevTools primitives to inspect semantic state, layout, text, focus, popups, scroll, screenshots, and target identity.
5. Produce red proof or classify the blocker.
6. Identify likely code owner and missing primitive if proof is blocked.
7. After a code fix, rerun the same primitive stack for green proof.
8. Promote only stable, valuable flows into agentic-testing recipes.

Agent Feature-Verification Loop

When an agent is implementing a feature and needs to invoke APIs, push values, and verify UI state, prefer writing a small throwaway verification script over issuing many one-shot CLI calls. Each one-shot CLI invocation costs ~0.5-2s of process/session overhead; a driver script runs the same steps at ~10-50ms per step inside one process.

The loop:

1. Edit Rust code.
2. Build with ./scripts/agentic/agent-cargo.sh build --bin script-kit-gpui.
3. Write a probe script that imports Driver from scripts/devtools/driver.ts, drives the exact user path (setFilterAndWait, simulateKey, batch, waitForState), asserts on getState/getElements output, and prints one JSON receipt.
4. Run it in one shell call and read the receipt. Iterate.

Binary-path: with no explicit override, the driver and session.sh auto-pick the freshest (by mtime) of target/debug/script-kit-gpui (owned by ./dev.sh) and target-agent/pools/agent-debug/debug/script-kit-gpui (agent-cargo's pool), and print which they chose to stderr — read that line and confirm it names the binary you just built. Pin explicitly with Driver.launch({ binary: ... }) or SCRIPT_KIT_GPUI_BINARY=... when the choice matters (e.g. another loop may rebuild either path mid-run). For a stable per-task path that survives later rebuilds, export an artifact clone: SCRIPT_KIT_AGENT_ARTIFACT_NAME=<task> ./scripts/agentic/agent-cargo.sh build --bin script-kit-gpui then launch target-agent/artifacts/<task>/script-kit-gpui.

Pi Sidecar Availability (Agent Chat)

Dev/agent-built binaries run outside a .app bundle, so the bundled Contents/MacOS/pi sidecar never resolves; debug builds fall back to target/pi-sidecar/pi, then ~/dev/pi_agent_rust/target/{release,debug}/pi (see src/ai/agent_chat/pi/binary.rs). When this skill is invoked, run bash scripts/agentic/ensure-pi-sidecar.sh once before launching or driving the app — it exits instantly when a Pi binary already resolves and otherwise builds the repo-local sidecar via scripts/prepare-pi-sidecar.sh. Without it, cmd+enter / Agent Chat surfaces show "Pi Agent Chat is unavailable", which is an environment gap, not an app bug. ./dev.sh runs the same check automatically.

Driver rules of thumb:

• Use sandboxHome: true unless the bug specifically needs real user data; it keeps runs reproducible and protects real Script Kit state.
• For live Agent Chat/brain probes add seedAgentAuth: true (or run bash scripts/agentic/seed-sandbox-home.sh <sandbox-home>) — it APFS-clones ~/.pi plus ~/.codex/{auth.json,config.toml} into the sandbox in one call.
• Use await driver.waitForSettle() instead of hardcoded sleeps before the first submit after opening a surface (the old sleep(1500) set_model-race workaround): it polls until N consecutive identical states and typically returns in ~300ms.
• Whole-scenario batch (setInput → waitFor → select) is one round trip and the fastest proof shape; per-command calls are still fast and easier to interleave with assertions.
• Always await driver.close() (use try/finally) so no app instance outlives the probe.
• The one-shot CLIs below remain correct for single inspections, strict target-identity receipts, and red/green compare artifacts. scripts/agentic/root-source-filter-matrix.ts is the migration template for porting a session.sh script to the driver.

Parallel Feature Loops

Multiple feature loops may run simultaneously on the same checkout. The isolation contract:

• Driver sessions are conflict-free by default: sessionName is a label, and every Driver.launch derives a unique artifact directory (/tmp/sk-driver-sessions/<name>-<pid>-<n>-<ts>), so concurrent drivers — even with identical names, even in one process — never share app.log, protocol bus, or sandbox HOME. Read the actual path from driver.sessionDir. Only an explicit sessionDir opts out.
• Always pair sandboxHome: true with parallel loops so app instances never contend on real Script Kit databases.
• Builds: the shared agent pool serializes under one lock and produces ONE binary — two loops building different edits would overwrite each other. Do NOT give each loop its own pool (a pool costs tens of GB and the disk-budget eviction in agent-cargo.sh will reap extras). Instead, build in the shared pool and export a per-loop binary clone: SCRIPT_KIT_AGENT_ARTIFACT_NAME=<loop-name> ./scripts/agentic/agent-cargo.sh build --bin script-kit-gpui, then Driver.launch({ binary: "target-agent/artifacts/<loop-name>/script-kit-gpui" }). The artifact is an instant APFS copy-on-write clone (~0 bytes), is replaced atomically on rebuild, and is never overwritten by other loops' builds. Loops with divergent source edits serialize on the pool lock — acceptable; relaunch to pick up new code.
• Legacy session.sh sessions are addressed purely by name: parallel loops MUST pass loop-unique session names (e.g. suffix the loop id/pid); reusing a name resumes or clobbers the other loop's session. Receipt scripts that write fixed .test-output/<tool> paths should namespace by session the way root-source-filter-matrix.ts does.
• Screen-level proofs do not parallelize: only one window is frontmost, so show/windowFocused waits, native input, and screenshot identity checks from concurrent loops race each other. Keep parallel loops on hidden-window protocol proofs (state/elements/layout/batch — none require show) and serialize the focus/visual proof at the end.

Proven: 3 concurrent drivers with the same label ran 30 batch scenarios in ~1s with distinct artifact dirs; 2 concurrent matrix runs both passed 48/48 with zero cross-talk and no slowdown.

Use These Skills Together

These are global (user-level) skills, not files in this repo — if one is not installed in your session, continue with this skill's primitives and say so.

• $protocol-automation when changing stdin JSON, Message, getState, getElements, getLayoutInfo, waitFor, batch, parse receipts, or target identity.
• $agentic-testing when the task needs a regression recipe, screenshot proof, process/session cleanup, or an existing stress scenario.
• $platform-windowing-macos when native windows, AppKit focus, AX, screen capture, or OS-level targeting owns the behavior.
• $testing-quality-gates when choosing narrow checks for protocol, Rust, Bun, or runtime proof.

Current Primitive Families

Prefer direct primitives over prewritten recipes:

• Driver library: scripts/devtools/driver.ts is the persistent, event-driven protocol driver for multi-step and high-volume work. It owns the app process directly (stdin pipe in, stdout pipe out, responses matched by requestId with no polling and no per-command subprocess), so a full round trip costs single-digit-to-tens of milliseconds instead of the ~0.5-2s per command of subprocess-per-step flows. Import Driver from scenario scripts (Driver.launch({ sandboxHome: true }), then setFilterAndWait, getState, waitForState, batch, simulateKey, close). bun scripts/devtools/driver.ts smoke is the standalone health check; bun scripts/agentic/driver-benchmark.ts is the throughput receipt (~100 filter scenarios in ~6s vs ~0.5s/scenario via session.sh). Use the driver when a task runs more than a handful of protocol steps; use the one-shot CLIs below for single inspections and strict-receipt proofs.
• Schema CLI: bun scripts/devtools/schema.ts reports the shared fail-closed receipt envelope, classifications, target identity fields, primitive schemas, and minimum acceptance bar for DevTools coverage.
• Targets CLI: bun scripts/devtools/targets.ts list|inspect exposes target discovery and strict target identity receipts so later inspect, measure, act, and compare commands do not parse target identity out of broad reports.
• Surface CLI: bun scripts/devtools/surface.ts inspect --surface <SurfaceKind> joins strict target identity to the generated surface contract, dismiss policy, runtime surface fields, capabilities, and missing primitive list.
• Elements CLI: bun scripts/devtools/elements.ts snapshot returns a target-scoped semantic tree with stable ids, labels, selected/focused ids, duplicate-id checks, and explicit missing bounds warnings.
• Layout CLI: bun scripts/devtools/layout.ts measure returns target identity plus component bounds, regions, clipping, overlap pairs, and resize-pressure metrics for visual/layout bug reports.
• Scroll CLI: bun scripts/devtools/scroll.ts inspect returns target identity plus scroll top, viewport/content heights, safe viewport, selected row visibility, footer occlusion, and overflow pressure.
• Focus CLI: bun scripts/devtools/focus.ts inspect returns target identity plus window focus, focused/selected semantic ids, focused/selected nodes, active footer, and keyboard ownership policy.
• Text CLI: returns target identity plus input/selected text fingerprints, text node lengths, footer label text, and explicit text-bounds gaps.

Investigation Contract

Every user-filed UX/UI bug investigation should produce an artifact or report with:

• intake: user report, screenshot paths, observed/expected behavior, suspected surface, safety constraints
• hypothesis log: current hypothesis, pivots, disproved explanations
• interaction transcript: action intent, control channel, target, command/input, visible result, receipt
• measurements: layout/text/scroll/focus/popup/screenshot metrics using stable field names
• classification: reproduced, not-reproduced, fixed, blocked-by-missing-primitive, blocked-by-unsafe-operation, needs-user-info
• likely owner: file/function/surface contract to inspect before editing
• proof plan: exact red and green commands/receipts
• cleanup: stopped sessions/processes and remaining windows

Do not call an investigation green because a recipe passed. Green means the same user-path symptom or measurement that failed red now passes after the fix.

Decision Rules

• Start protocol/MCP-first. Use direct DevTools primitives before reaching for a long recipe.
• Use devtools.inspect first when the surface is unfamiliar, because it tells the agent what target it actually found and which follow-up primitives are supported or missing.
• Pass the user bug text into devtools.inspect --bug when available. The report should preserve the suspected surface, primitive stack, visible-window proof, missing primitive details, and recipe boundary before any fix is attempted.
• Open the real app for UX/UI reports. Protocol-only setup is fine, but visible/user-path proof must be present for visual or interaction bugs.
• Use recipes only when they match the bug directly or as regression proof after the investigation has isolated the issue.
• If direct primitives are missing, stop and name the missing primitive precisely.
• Never hide missing DevTools coverage behind screenshots, sleeps, native input, or broader recipes.
• Escalate to native input only when the bug depends on OS delivery, real focus, pointer behavior, AX, or screen-capture permission.

Coverage Audit

Read references/devtools-coverage-audit.md when planning new DevTools surface work, expanding coverage, or deciding whether a user bug is blocked by missing instrumentation.

Read references/devtools-api-coverage-map.md when the work needs Chrome-DevTools-level breadth, Notes coverage, Dictation coverage, or the next protocol/API primitive. The checked-in coverage command is the machine-readable companion to that map.

Read references/devtools-surface-inventory.md when planning broad DevTools expansion, preparing Oracle prompts, or deciding which app surface families still lack direct primitives.

Read references/devtools-oracle-buildout-plan.md for the Oracle-reviewed primitive sequence, shared receipt envelope, and surface-specific proof requirements.

Read references/oracle-devtools-scenario-iterations.md when translating user bug reports into coverage work. It preserves the 50 Oracle-planned UX/UI bug scenarios and the DevTools fields needed to investigate them.

Use the audit as a living map, not a final verdict. Update it when new protocol/MCP primitives land or when a recipe reveals a missing primitive.

Completion Gates

For DevTools surface changes:

• Add source-contract tests for protocol schema, routing, target identity, and fail-closed behavior.
• Produce at least one direct primitive receipt, not only a scripted recipe receipt, when behavior is runtime-observable.