fault

name	fault
description	Inject HTTP errors, socket faults, or transport-layer drops on a player session via the harness CLI. Invoke when the user says "inject", "drop a 404", "fault on segments", "every N seconds", "kill the connection mid-body", "throttle", "reset faults", or otherwise wants to mutate the proxy's behaviour for one player. This skill owns the English-to-CLI-flag translation including the ambiguous-phrase tables; resolve disambiguation BEFORE running any harness command.
last_reviewed	"2026-05-19T00:00:00.000Z"

Inject faults via harness fault

Default target: https://dev.jeoliver.com:21000 (test-dev, public Let's Encrypt cert). Every mutation is checkpoint-protected by the CLI — harness undo <target> rolls back the most-recent change. You never have to manage state.

Conventions: this skill follows .claude/skills/CONVENTIONS.md. Most load-bearing for fault: every mutation is checkpoint-protected (harness undo rolls back); ALWAYS show current state (harness fault list <t>) before adding, removing, or clearing; lead every shell command with harness.

Wire-shape contract: every fault type produces a SPECIFIC on-the-wire pattern that characterization tests interpret against. Before reasoning about what a fault DOES, read .claude/standards/fault-injection-wire-contract.md — it documents the exact wire shape per fault AND the real-world failure mode each one models. Don't propose changes to applySocketFault or any fault-type case branch without reading that doc first.

Always do this first

Before any fault add, show the user what's already there:

harness --insecure fault list <target>

If rules are already active, surface them before adding new ones. The CLI's first-match-wins rule evaluation means a stale rule at the top of the list can mask a new one entirely.

After every successful add, echo the inverse:

added rule abc12345 — undo with `harness undo <target> --yes` or
                              `harness fault rm <target> abc12345`

Phrase disambiguation — read this before parsing the user

English asks routinely map to genuinely different harness behaviours. Resolve these before picking flags; ask if you can't.

"Drop"

User says	Means	Command
"drop a 404", "drop 500s"	HTTP error	harness fault add <t> --type 404 (or 500/503)
"drop packets", "blackhole it"	Transport drop (kernel discards)	harness shape <t> --transport-fault drop ⚠
"drop the connection mid-request"	Socket fault	harness fault add <t> --type request_body_reset
"drop the session"	Delete the player	harness players rm <t> — confirm first

⚠ Transport drop isn't exposed as a top-level shape flag yet — see "Known gaps" below. For now, use harness raw PATCH with {"shape":{"transport_fault":{"type":"drop"}}}.

"Every / per / 1-in"

User phrasing	Flags
"every 10 seconds", "1 every 10s"	--frequency 10 --mode seconds
"every 10 requests", "1 in 10"	--frequency 10 --mode requests
"X faults per second on every request"	--frequency X --mode failures_per_seconds (rare)
Bare "every 10"	Ambiguous — ask requests vs seconds.

The default --mode requests is what most operators mean. failures_per_seconds means X faults per second window, NOT "once every X seconds" — easy to get wrong; confirm if the user says it.

"Kill the network" / "break it" / "hose it"

Don't guess. Offer a menu:

• 100% transport drop → harness raw PATCH /api/v2/players/<id> --body '{"shape":{"transport_fault":{"type":"drop","frequency":1,"mode":"requests","consecutive":1}}}'
• Bandwidth = 0 → harness shape <t> --rate 0 (player will stall on next segment)
• 100% packet loss → harness shape <t> --loss 100
• All 5xx on every request → harness fault add <t> --type 503 --frequency 1 --mode requests

"All requests" vs "all sessions"

• "every request kind" / "any request" → omit --kind (matches all)
• "all sessions" / "every player" → iterate harness players list, run the mutation per id. Or create a group with harness groups create --members A,B,C and PATCH the group once.

Reset / clear / undo — three different things

Verb	Command	Behaviour
Undo ("undo that")	harness undo <t>	Replays the most-recent checkpoint. Restores exactly what was there.
Clear faults	harness fault clear <t>	Removes all fault rules. Doesn't restore prior state — wipes.
Wipe everything	harness players prune --yes	Destroys ALL players + state. Confirm before running.

fault clear does NOT undo a prior fault clear. Once cleared, undo will reverse the clear (restore the rules that existed before the clear); fault clear again would re-clear them.

Number/unit normalisation

User says	Flag value
"5mb", "5 megabits", "5 Mbps", "5M"	--rate 5
"100k", "100 kbps", "100 Kbits"	--rate 0.1
"1 gig", "1 Gbps"	--rate 1000
"100ms", "0.1s"	--delay 100
"5%", "5 percent loss"	--loss 5

For ambiguous units ("100k") pick the streaming-plausible value (kbps for bandwidth) and tell the user what you assumed.

Fault types

harness fault add <target> --type TYPE [--kind KIND] [--frequency N] [--mode MODE] [--consecutive N]

Available --type:

Type	Layer	Notes
403, 404, 500, 503	HTTP	Status code returned
connection_refused	HTTP	Server rejects the connection
dns_failure	HTTP	Synthetic DNS error
rate_limiting	HTTP	Returns 429 + Retry-After
corrupted	HTTP	Segment payload zero-filled (segment-only)
request_connect_delayed/hang/reset	Socket	TCP connect phase
request_first_byte_delayed/hang/reset	Socket	First-byte phase
request_body_delayed/hang/reset	Socket	Body-transfer phase
none	n/a	Disables a rule without removing it

Available --kind (request_kind filter; omit for all): segment, partial, manifest, master_manifest, init, audio_segment, audio_manifest.

corrupted is segment-only — pass --kind segment (CLI will reject with 400 otherwise).

Common recipes

Ask	Command
"404 every 10s on every request"	harness fault add ipad --type 404 --frequency 10 --mode seconds
"one-shot 404 on the next segment"	harness fault add ipad --type 404 --kind segment
"manifest timeout every 5 requests"	harness fault add ipad --type request_first_byte_hang --kind manifest --frequency 5
"10 consecutive 503s every minute"	harness fault add ipad --type 503 --frequency 60 --mode seconds --consecutive 10
"drop the manifest playlist"	harness fault add ipad --type 404 --kind manifest
"kill the connection mid-body"	harness fault add ipad --type request_body_reset
"503 only on 2160p segments"	harness fault add ipad --type 503 --kind segment --url-substr 2160p
"clear faults on the iPad"	harness fault clear ipad
"undo the last thing"	harness undo ipad --yes

Edit / remove individual rules

harness fault list ipad                          # shows short rule_id (8-char prefix ok)
harness fault edit ipad 20260517 --frequency 5   # change cadence without removing
harness fault rm ipad 20260517                   # delete one
harness fault clear ipad                         # delete all

The short rule_id from fault list is enough — the CLI expands prefixes.

Known gaps (CLI follow-ups, not skill bugs)

• harness shape doesn't expose --transport-fault drop|reject as top-level flags yet. For now use harness raw PATCH with the shape body, or wait for the CLI to add typed flags.
• harness shape doesn't expose --pattern (square_wave / ramp / pyramid) — for ramp behaviour use harness procedure abr-sweep --rates X,Y,Z --hold Ns instead.
• harness fault add for play-scoped (vs player-scoped) rules isn't wrapped yet — use harness raw POST /api/v2/plays/<play_id>/fault_rules for play-scope.

What NOT to do here

• Don't checkpoint first. The CLI does it automatically on every mutation. Skipping the harness checkpoint call isn't an oversight — it's redundant.
• Don't add a rule without showing the existing list first. Stale top-of-list rules silently consume requests that the operator thought their new rule would catch.
• Don't run players prune or players rm without explicit user confirmation. Even with --yes the CLI proceeds without asking; require the operator to say yes in the chat first.
• Don't echo "rule added" without the undo command. Every mutation reply ends with the inverse.

See also

• shape — bandwidth / delay / loss / patterns (separate skill)
• triage / investigate — find what broke before deciding to inject
• finding — capture what a fault test reproduced
• .claude/standards/avplayer-quirks.md — how iOS reacts to specific faults