reverse-eng-site-skill

name	reverse-eng-site-skill
description	Reverse-engineer a website's undocumented internal API by instrumenting browser traffic, replaying captured requests as direct HTTP, proving auth and response fidelity, and packaging a resilient client or site-specific skill. Use when asked to discover, automate, wrap, download from, or repair an integration with a website that lacks a public API or MCP.

Reverse Engineer Site APIs

Follow one transition: instrument, then replay. Use the logged-in browser as an instrumented probe to establish state and provoke requests. As soon as the target request is captured, end UI navigation and move execution to direct HTTP. The browser may remain the authenticated transport for direct requests or refresh authentication, but later steps do not resume clicking through the site.

1. Define the operation

Write one observable operation in this form:

Given <state>, perform <action>, obtain <exact result>.

Classify it as read-only or state-changing. For a state-changing operation, choose a reversible test object and keep bulk execution outside discovery.

Complete when: the operation is classified read-only or state-changing, the target result is explicit, and a state-changing operation has a reversible test action.

2. Capture one request

Use the user's existing authenticated browser session. Start network capture before the action, preserve the log, clear unrelated traffic, then perform the smallest UI action that produces the result once. Prefer a DevTools Protocol or HAR capture that includes request bodies and survives navigation.

Record:

• method, full URL, query parameters, status, and initiator
• request headers and body shape, with secret values redacted
• response content type, size, top-level shape, and two non-secret UI canaries
• whether cookies, authorization headers, CSRF tokens, or request signatures exist

If a HAR is available, run:

python <skill-dir>/scripts/har_candidates.py capture.har

Here and below, <skill-dir> means the directory containing this SKILL.md.

Write capture.har to a private temporary directory outside the repository. It may contain unredacted cookies and tokens. Read the Secret boundary before storing it.

Read references/browser-capture.md when the available browser tool hides bodies, loses events on navigation, or exposes multiple ways to access the logged-in session.

Complete when: at least one captured request can plausibly account for the exact target result and at least one UI canary has been saved. When the UI exposes two, reserve one as a fidelity holdout and do not use it to select the endpoint. Record when only one canary exists. A rendered page or DOM selector is not request evidence.

3. Replay the request outside the UI

Start from the captured request, not a guessed endpoint. Reproduce it with curl, httpx, or the site's native protocol. Preserve its method, encoding, cookies, headers, and body exactly for the first replay. Use the browser's cookie store or authenticated request context programmatically; exhaust those paths before asking the user to copy a cookie.

Read references/browser-capture.md for the ordered auth-acquisition ladder before asking the user for a cookie.

Compare capture and replay using status, byte length, stable keys, item count, and the captured canary. When a browser-only anti-bot or signed request blocks standalone HTTP, replay with fetch inside the authenticated browser and record that constraint instead of substituting DOM automation.

Complete when: a repeatable command returns the same target data, or the specific browser-bound requirement is demonstrated with request evidence.

4. Minimize and map

Remove nonessential headers one at a time. Vary one input at a time to map:

• identity and publication, workspace, account, or resource IDs
• pagination or cursors, sort, filters, search, and limits
• list, detail, download, and mutation endpoints
• REST, GraphQL operations, persisted queries, RPC batches, or streaming frames
• authentication lifetime, CSRF coupling, and refresh behavior

Compare authenticated and unauthenticated replays. A successful public list request proves only that the list is public; it says nothing about full detail, paid content, private fields, or mutations.

Read references/protocols.md when the request is GraphQL, a framework RPC, a Server Action, a batch, or a stream.

Complete when: the smallest known request is documented and every input needed for the user's operation has a known source.

5. Prove fidelity

Read references/security-and-validation.md before comparing samples or testing a write.

Compare the direct response with the UI canary or canaries saved during Step 2, using the reserved holdout when two were available. Perform this comparison without further UI navigation. Report exact response fields. Treat previews, summaries, search snippets, and list payloads as incomplete until a detail response proves otherwise. Preserve source data; format it rather than reconstructing missing text.

For writes, use one reversible sample, inspect the response, and read it back through an independent endpoint.

Complete when: two controlled samples reproduce the expected result and the auth/no-auth difference is measured. For writes, the sample is also verified by read-back.

6. Build the durable client

Implement the proven request with:

• secrets from environment variables or a browser session, never source files
• pagination with a tested stopping condition
• timeouts, bounded retries, and rate limiting
• explicit output formats that retain raw IDs and fields
• a low-cost auth check and the failure classifications in the validation reference
• dry-run or confirmation gates for state-changing and bulk operations

Keep capture artifacts out of version control until they are sanitized. Read references/security-and-validation.md before persisting requests, handling write endpoints, or declaring the client complete.

Complete when: the client runs from a fresh process, secrets are absent from tracked files and logs, pagination terminates correctly, and failures identify the rediscovery point. State-changing and bulk operations have a dry-run or confirmation gate.

7. Package the site-specific skill

Copy assets/site-skill-template and replace every placeholder. Include the verified operations, auth acquisition, runnable examples, evidence date, and a repair path that starts by recapturing traffic. Keep site facts in that site skill; keep this meta-skill generic.

Complete when: every template placeholder is replaced, a fresh agent can run one read-only operation without UI clicking, and can identify exactly how to recapture it if the site changes.

Handoff

Report:

Evidence	Required content
Captured request	action, method, redacted URL, status
Replay	command or script path and matching canary
Auth	mechanism, programmatic source, measured auth/no-auth difference
Surface	endpoints or operations, parameters, pagination
Validation	two samples and write read-back when applicable
Artifact	client path, site skill path, and secret scan result
Limits	anti-bot, rate, account, legal, or browser-bound constraints

Declare the result discovered after capture, proven after direct replay and fidelity checks, and durable only after the packaged client passes from a fresh process.