Ejecuta cualquier Skill en Manus
con un clic

Ejecuta cualquier Skill en Manus con un clic

submit-smoke

Estrellas4

Forks0

Actualizado10 de junio de 2026 a las 17:42

Cheaply prove the PTY turn-submission mechanism actually lands turns BEFORE any timed or headline run spends real tokens. Runs .skills/submit-smoke/scripts/submit_smoke.sh — one tiny passthrough Haiku block — and checks two independent signals: the driver's own "confirmed N/M" tally AND the count of real (non-ping) records the proxy logged. PASS only when every turn confirmed and the proxy logged at least that many real turns. Also reports which Enter encoding won (pure \r vs an escalated \n / \r\n fallback). Run this first whenever the driver, the submit helper (benchmark/lib/turn_submit.js), or the claude TUI version changes.

Instalación

Instalar con Codex o Claude Copia este prompt, pégalo en Codex, Claude u otro asistente, y deja que revise la página de la skill y la instale por ti.

Ejecutar en Manus

Fuente

zapgun-ai

zapgun-ai/clawback

Abrir repositorio de GitHub Ver repositorios del creador

Descarga

Ejecutar en Manus

Ocupaciones relacionadasSOC

Basado en la clasificación ocupacional SOC

Desarrolladores de softwareOcupaciones informáticas y matemáticas·SOC 15-1252

Explorador de archivos

2 archivos

SKILL.md

readonly

name

submit-smoke

description

clawback submit-mechanism smoke

Run .skills/submit-smoke/scripts/submit_smoke.sh from the project root. It is the fast gate that answers ONE question before a real benchmark: does Enter actually submit?

Why this exists

The driver used to write ${prompt}\r in a single chunk; Claude Code's bracketed-paste path coalesced it and the \r landed as a newline INSIDE the input box, so ~6 of 8 turns silently never ran. The box sat at "turn 0%" and the settle-on-silence heuristic could not tell "claude finished" from "claude never started." The fix (benchmark/lib/turn_submit.js) types the text, pauses, sends Enter as its OWN keystroke, and CONFIRMS each turn against the proxy's --turn-log before advancing — escalating \r -> \n -> \r\n only when confirmation never arrives. This skill verifies that fix end to end on real claude, cheaply, so a swallowed Enter can never silently under-drive a headline run again.

Usage

# default: 3 turns, passthrough, Haiku, port 8799
.skills/submit-smoke/scripts/submit_smoke.sh

# bump the confirmation window for a slower model (Opus streams later):
.skills/submit-smoke/scripts/submit_smoke.sh --model claude-opus-4-7 --confirm-sec 20 --turns 3

How to read the verdict

The script prints a verdict block and exits non-zero on FAIL:

driver said: confirmed N/M turn — the driver's own tally (a turn is confirmed when a new non-ping turn-log record OR a sustained PTY output burst appears within --confirm-sec).
real turn-log recs: K — non-ping /v1/messages records the proxy actually wrote. Because the arm is passthrough (no keep-alive pings), this is an exact count of turns that truly hit Anthropic.
Enter encodings that won — a histogram. All "\r" means CR-as-its-own- keystroke submits once separated from the text (the happy path). Any "\n" / "\r\n" winning means CR alone still did not submit and the fallback carried it — a real finding to note before the big run.

PASS = confirmed N/M with N==M, AND real records >= turns, AND the driver exited 0. Anything else is FAIL (it points you at drive.log and proxy.log in the run dir).

Cost & safety

Spends real Anthropic tokens for --turns turns via the real claude binary — keep it tiny (default 3) and on Haiku. Runs on port 8799 by default so it never collides with a dev proxy on 8787. Each run writes a self-contained runs/submit-smoke-<ts>/ (turn-log, proxy log, drive log, state) at 0600.

When to run it

After editing benchmark/bin/drive_pty.js or benchmark/lib/turn_submit.js.
After a claude CLI upgrade (the TUI's paste / keyboard handling can move).
As the first step of any .skills/ab session — a green submit smoke is the precondition for trusting an A/B block's turn counts.

Más de este repositorio

mismo repositorio

zapgun-ai/clawback

Run a counterbalanced A/B knob block end to end and produce a report. Orchestrates the whole validation loop the other skills do piecemeal — for each arm it starts a clawback proxy with that arm's knobs and its OWN --state file (fresh SESSION KEY, no clawback-side carry-over), drives N turns of load through it (pty or replay driver), stops it, then analyzes the per-arm turn-logs and renders the charts. No cache-salt: production forwards none, so neither does the harness. Default block is A0 (passthrough baseline) vs A5 (recommended stack); the analyzer pools all passthrough turns into the baseline and compares each treatment arm against it, so the block must include A0 and may carry several treatment arms at once. Use this to run the full A/B runbook in one command. The pty driver spends real Anthropic tokens (one run per arm) — pin --model.

2026-06-104

armstat

zapgun-ai/clawback

Fast, read-only per-arm turn-log summary — billable turn count, httpStatus tally (catches rate-limiting / non-200s mid-run), cache_read/creation sums with the ephemeral 5m-vs-1h split (a direct knob-sanity check: A2/A3/A5 should show ephemeral_1h > 0, A0/A1/A4 should not), token-weighted hit rate, and thinkingBudget. Unlike the heavyweight `bench` analyzer it does no bootstrap/CI/pricing, so it is cheap to run at each arm boundary WHILE a suite is still in flight. Safe on a live, actively-appended NDJSON file (a partial trailing line is skipped). Use it to spot-check a just-finished arm before the next one starts.

2026-06-104

bench

zapgun-ai/clawback

Analyze clawback turn-logs (NDJSON) to compare treatment vs passthrough arms. Produces report.md, report.csv, summary.json, manifest.json. Headline is billable input tokens reclaimed per turn (full-rate quota saved, no pricing) with a bootstrap 95% CI; per-turn $ and hit-rate are a demoted cost appendix, all stratified by inter-turn gap bucket. Use after the operator has collected counter-balanced windows.

2026-06-104

capture

zapgun-ai/clawback

Capture a REAL Claude Code /v1/messages body as a replay fixture, so the HTTP replay arm (.skills/replay) faithfully reproduces Claude Code's cache_control breakpoint structure (a hand-authored fixture makes the 1h-TTL knob look like a no-op). Starts a clawback proxy with --capture-body, drives ONE real claude turn through it via the PTY driver, verifies the dump carries cache_control breakpoints, and promotes it to the fixture path. Spends real Anthropic tokens (exactly one turn; pin a cheap model with --model). Use once before replay-based benchmarking, or to refresh the fixture when Claude Code changes its request shape.

2026-06-104

card-tones

zapgun-ai/clawback

Render the clawback share card in BOTH tones (neg + pos) as a DESIGN render-only proof — NOT a benchmark result — so the share-card atmosphere (deep-purple LENS_CORE centre, win-only cyan corona, darkened gradients) can be eyeballed in one pass. neg uses a real run's amber regression card (purple core, no corona); pos is a SYNTHETIC fixture built only to exercise the renderer's win branch (purple core + cyan corona) — its number is fabricated, never published. Delegates to the shared preview_card.sh. Use when tweaking the share-card renderer's visuals and you want to see both tones side by side.

2026-06-104

drive

zapgun-ai/clawback

Drive the REAL claude binary through clawback's PTY channel at a scripted inter-turn gap schedule — the faithful load arm. Because the bytes on the wire are produced by genuine Claude Code, this is the only driver that exercises the 1h-TTL nested-cache-control rewrite (arm A2) authentically. Use for headline/moat numbers and to cross-check the HTTP replay arm (.skills/replay). Requires node-pty and a running clawback proxy.

2026-06-104

name

submit-smoke

description

clawback submit-mechanism smoke

Run .skills/submit-smoke/scripts/submit_smoke.sh from the project root. It is the fast gate that answers ONE question before a real benchmark: does Enter actually submit?

Why this exists

Usage

# default: 3 turns, passthrough, Haiku, port 8799
.skills/submit-smoke/scripts/submit_smoke.sh

# bump the confirmation window for a slower model (Opus streams later):
.skills/submit-smoke/scripts/submit_smoke.sh --model claude-opus-4-7 --confirm-sec 20 --turns 3

How to read the verdict

The script prints a verdict block and exits non-zero on FAIL:

driver said: confirmed N/M turn — the driver's own tally (a turn is confirmed when a new non-ping turn-log record OR a sustained PTY output burst appears within --confirm-sec).
real turn-log recs: K — non-ping /v1/messages records the proxy actually wrote. Because the arm is passthrough (no keep-alive pings), this is an exact count of turns that truly hit Anthropic.
Enter encodings that won — a histogram. All "\r" means CR-as-its-own- keystroke submits once separated from the text (the happy path). Any "\n" / "\r\n" winning means CR alone still did not submit and the fallback carried it — a real finding to note before the big run.

PASS = confirmed N/M with N==M, AND real records >= turns, AND the driver exited 0. Anything else is FAIL (it points you at drive.log and proxy.log in the run dir).

Cost & safety

When to run it

After editing benchmark/bin/drive_pty.js or benchmark/lib/turn_submit.js.
After a claude CLI upgrade (the TUI's paste / keyboard handling can move).
As the first step of any .skills/ab session — a green submit smoke is the precondition for trusting an A/B block's turn counts.