run-embedded-app-verify

name	run-embedded-app-verify
description	Verify a Shopify embedded-app change by driving the real, authenticated Shopify admin in the developer's browser. Use after changing app code when the user asks to verify, check, or test the change in the browser/admin, or asks to run the verify loop.

Verify an embedded Shopify app change in the real admin

Closed loop: preflight → dedicated verify window → navigate the embedded app → interact → assert against the plan → report. On mismatch: fix the code and loop.

Hard rules:

• The dev server is the developer's responsibility. NEVER start it, and NEVER probe it — do not curl the application_url from shopify.app.toml or any tunnel URL: shopify app dev mints a fresh tunnel URL per run, so the toml value may not be the one actually serving. Judge dev-server health only by what the app iframe renders (step 4). If it is down, tell the user and stop.
• NEVER touch browser tabs other than the verify tab you created. The rest of the browser belongs to the developer.
• Delete every screenshot you take once the report is delivered (see step 6).

1. Load project config

Read .claude/shopify-verify.json in the project root:

{ "appHandle": "...", "storeDomain": "...", "iframeSelector": "iframe[name=\"app-iframe\"]" }

If the file is missing, run the setup-embedded-app-verify skill flow first (same plugin), then continue.

2. Preflight

1. Determine the verification URL first — the preflight opens the browser directly on it when it has to launch one: https://admin.shopify.com/store/<storeDomain>/apps/<appHandle>/<page-path> (<page-path> = the app route relevant to the change being verified).

2. Run:

   node ${CLAUDE_PLUGIN_ROOT}/scripts/ensure-browser.mjs --browser "${user_config.browser}" --mode "${user_config.mode}" --port "${user_config.cdp_port}" --url "<the verification URL>"
   

   (The ${user_config.*} values are substituted into this skill at load time; blank or literal placeholders are fine — the script defaults to chrome / profile / 9222.)

   • Exit 0 → CDP is live. Note the BROWSER_STATE: line in the output — launched-at-url, launched, or reused — step 3 branches on it.
   • Non-zero → show the script's stderr message to the user verbatim and stop. (CDP_BLOCKED_DEFAULT_PROFILE means: tell the user to switch the plugin's mode option to "profile" via /plugin → configure. BROWSER_MISMATCH means: a browser other than the configured one owns the CDP port — the user must quit it or change the browser/cdp_port option.) Do NOT probe the dev server or any tunnel URL here (hard rule above) — the app iframe render in step 4 is the only dev-server health check.

3. Open the verify window

Keep the developer's windows untouched.

Preflight said BROWSER_STATE: launched-at-url (profile mode): the browser opened directly at the verification URL — its only tab IS the verify window. browser_tabs (action: list), select that tab, grab its targetId with browser_run_code_unsafe, and skip to step 4:

async (page) => {
  const session = await page.context().newCDPSession(page);
  const { targetInfo } = await session.send("Target.getTargetInfo");
  await session.detach();
  return targetInfo.targetId;
}

Preflight said BROWSER_STATE: launched or reused: the browser's tabs belong to the developer (an attach-mode relaunch restores their session — those tabs are NOT yours, whatever they show). browser_tabs (action: list), then:

Case A — the list shows only blank tabs (about:blank / chrome://new-tab-page — a browser nobody is using): reuse that startup window instead of opening a second one. Select it, grab its targetId with browser_run_code_unsafe:

async (page) => {
  const session = await page.context().newCDPSession(page);
  const { targetInfo } = await session.send("Target.getTargetInfo");
  await session.detach();
  return targetInfo.targetId;
}

then browser_navigate it to the verification URL.

Case B — any non-blank tabs exist (attach mode / browser already in use): open a dedicated verify window directly at the verification URL — no about:blank hop — with browser_run_code_unsafe (the code is invoked with the current page as its single argument):

async (page) => {
  const session = await page.context().newCDPSession(page);
  const { targetId } = await session.send("Target.createTarget", {
    url: "<the verification URL above>",
    newWindow: true,
  });
  await session.detach();
  return targetId;
}

Then browser_tabs (action: list) and select the newly added entry — the tab that was not in the list before creation (new tabs are appended at the end). Do not pick the first URL match: the developer may already have a tab open on the same admin page.

In all cases: save the targetId — step 6 closes the window with it. Every subsequent navigation/interaction happens in this tab only.

4. Drive the app

1. The verify window already loads the verification URL from step 3 — wait for it. On later loop iterations, reload / browser_navigate this same tab instead of opening a new window.
2. If the URL redirects to accounts.shopify.com: the Shopify session expired.
   • attach mode → ask the user to log into the Shopify admin in their browser, wait, retry.
   • profile mode → leave the verify window open on the login page, ask the user to log in there once, wait for their confirmation, retry.
3. Wait for the app iframe (iframeSelector from config), then take an iframe-scoped browser_snapshot. If it shows a tunnel/connection error instead of the app UI — "server IP address could not be found", ERR_NAME_NOT_RESOLVED, ERR_CONNECTION_REFUSED, a Cloudflare tunnel error page (e.g. error 1033), or a bare "can't be reached" page — the app's dev server is not running or not reachable. Report exactly that to the user, ask them to check that their dev server is running, and stop. Do not curl anything and do not start the server (hard rule above). The embedded app lives entirely inside that iframe — target all app selectors through it. Exception — App Bridge UI renders in the top admin document, outside the iframe: modals, toasts, the save bar, and title-bar actions will NOT appear in an iframe-scoped snapshot. If UI you expect is missing from the scoped snapshot, take an unscoped browser_snapshot (whole page) and interact with it there.
4. Interact per the plan: browser_snapshot first, then click/type/select, browser_wait_for after actions that trigger loading. Target syntax: interaction tools take target = the ref exactly as printed in the latest snapshot (e.g. f6e393), or a plain unique CSS selector. Never ref=f6e393 and never iframe[...] >> [ref=...] — both fail ("Unknown engine ref" / "does not match any elements"). Refs go stale after navigation or re-render: re-snapshot and use the fresh ref.
5. Evidence: accessibility snapshots plus screenshots. Pass a bare filename to browser_take_screenshot — the server saves it under the plugin's output directory, outside the project tree. Remember every saved path (the tool result prints it).

5. Assert and loop

Compare what the page shows against the expected behavior (the plan, or the change just made). Judge like a reviewer: exact copy, state transitions, toasts, network side effects visible in the UI.

• PASS → report what was verified with evidence, go to step 6.
• FAIL → report the exact mismatch (expected vs observed), fix the code, then re-verify: the dev server hot-reloads, so a reload of the admin page (step 4.1) picks up the change.
• Loop guard: 3 consecutive failures on the same assertion → stop, report all evidence, hand control back to the user.

6. Cleanup

1. Close the verify window — unless the user asked to keep it open — with browser_run_code_unsafe and the targetId saved in step 3:

async (page) => {
  const session = await page.context().newCDPSession(page);
  await session.send("Target.closeTarget", { targetId: "<targetId from step 3>" });
}

Never use browser_close or close-by-index instead: both act on the MCP's notion of the current tab, which closes the wrong tab — or nothing — when the developer has many tabs open. If the tool errors because its own page just closed, the window did close: that is success. 2. Delete every screenshot taken during this run. Exception: if the user asked to keep them, move them to <project>/.claude/verify-screenshots/<YYYYMMDD-HHmmss>/ and say where they are.

Failure handling

Symptom	Action
ensure-browser.mjs non-zero	Show its message verbatim; stop
Redirect to accounts.shopify.com	Session expired → step 4.2
Iframe renders tunnel/connection error (DNS not found, connection refused, Cloudflare 1033)	Dev server not running/reachable → report to user, ask them to check it, stop. Never curl a tunnel URL, never start the server
Iframe never appears	Screenshot the admin page; report what actually rendered (404 / install prompt / error banner); if 404, the app may not be installed on this store — point the user to the install link in their dev server output
MCP tool errors Browser context management is not supported on first call	A stale or foreign Chromium holds the CDP port in a restricted state. Ask the user to quit the browser that owns the port, rerun the preflight (it relaunches a clean one), retry
3 consecutive assertion failures	Stop and report; do not thrash