browser-workflow-skill-builder

# Browser Workflow Skill Builder Create or improve browser automation skills for logged-in web apps. Use this when building skills that read a page, choose the correct item, generate or prepare content, click/type/upload, or submit/post/publish. The core pattern is: **save context -> analyze DOM/CSS -> write skill-local JS extraction -> verify same item by hash -> click/type with generic browser tools -> screenshot/context verify**. Designing the *command surface* of a `co <thing>` CLI and its SKILL.md, rather than a browser workflow on top of one? That is the sibling skill `cli-skill-design`. ## Architecture Rule Do not add site-specific Python to the core browser tool. Use generic browser primitives — run each as `co browser <function>`, positional args in order, options as `--flag=value`: ```bash co browser save_page_context <name> co browser take_screenshot co browser run_page_script <script_path> '<args_json>' co browser run_frame_script <script_path> '<args_json>' --frame_url_contains=<substring> --frame_name=<name> co browser click_element_by_selector <selector> --index=<n> --text='<visible text>' co browser type_text_by_selector <selector> '<text>' --index=<n> co browser click_element_near_selector <anchor_selector> <target_selector> --target_text='<text>' --container_selector=<selector> co browser upload_file_by_selector <selector> <file_path> --frame_name=<name> co browser upload_file_after_click_by_selector <click_selector> <file_path> --text='<button text>' --frame_name=<name> ``` Put site-specific selectors and DOM logic in the target skill: ```text .co/skills/<target-skill>/ SKILL.md scripts/ extract-<items>.js verify-<item>.js ``` `run_page_script(...)` runs JS in the main document of the current browser page, so it shares the same browser context, login cookies, current tab, and DOM state. `run_frame_script(...)` runs the same page-function script across matching frames and returns per-frame results. Use it when `elements.json` shows the target element has a non-main `frame`, when the UI is rendered through iframe/frame-like surfaces, or when main-document selectors cannot see a visible modal/composer. ## Build Workflow 1. Create or update `.co/skills/<target-skill>/SKILL.md`. 2. Start with safe read-only browser analysis: - navigate to the target page - take a screenshot - run `co browser save_page_context <target>_before` - inspect saved `page.html`, `styles.css`, and `elements.json` 3. Identify stable selectors: - Prefer semantic selectors: `aria-label`, `role`, `contenteditable`, `placeholder`, `data-testid`, stable text. - Avoid generated class names unless there is no alternative. - For repeated items, identify the container selector, body/text selector, author/title selector, and primary action selector. 4. Write `scripts/extract-<items>.js`. - It must return structured JSON, not prose. - Include normalized text, a deterministic `text_hash`, action indexes, and visible bounds. - Filter ads/promoted/sidebar items in JS when possible. 5. Write `scripts/verify-<item>.js`. - It must accept expected author/title/text/hash from `args`. - It must rescan the current DOM and return `ok: true` only when the same item is still visible and actionable. 6. Update `SKILL.md` to make the scripts mandatory gates before clicks/submits. The target `SKILL.md` must also include a trust-the-scripts rule: during a live run, do not read, `ls`, or `glob` the skill's own script files, and do not run `node --check` or local tests. Live testing on 2026-06-12 showed an agent spending its first ~10 iterations reading every script before acting. Scripts are trusted by path; if one fails twice, save context and stop — fix it outside the live run. 7. Run a safe extraction test first: no click, no type, no submit. 8. Run a full workflow only after extraction and verification are proven. If browser setup itself fails with profile locks, closed browser context, or Playwright sync/async errors, stop and report the browser failure. Do not write fallback browser runners, do not switch to ad hoc Python/Node Playwright scripts, and do not inspect browser tool source during a target skill run. ## Discovery Protocol Before writing automation, gather evidence from the real page: 1. Open the target page in the browser workflow. 2. Take a normal viewport screenshot, not only a full-page screenshot. 3. Save context: ```bash co browser save_page_context <site>_<workflow>_before ``` 4. Inspect `elements.json` first for clickable actions, aria labels, positions, text, button order, and the `frame` field. 5. If the target has a non-main `frame` or appears in a shadow/interop surface, plan to use `run_frame_script(...)` and shadow-aware JS traversal. Do not assume `document.querySelectorAll(...)` from the main page can see it. 6. Inspect `page.html` for stable DOM relationships around the target item and action button. - For frame/shadow UIs, `page.html` may not include the interactive internals. Treat `elements.json` plus screenshots as stronger evidence. 7. Inspect `styles.css` only when visibility, sticky layout, or hidden duplicate elements make DOM matching ambiguous. 8. Write down the exact selector set in the target skill: - item container selector - item text selector - author/title/channel selector - action selector and action text - editor/input selector - submit selector and submit text - frame name/url filter if `run_frame_script(...)` is required - whether scripts must traverse open shadow roots - upload trigger selector and file input selector if local file upload is part of the workflow - whether the editor is plain text or a rich text editor that preserves HTML For feeds, modals, infinite scroll, and dashboards, the same visible label often appears many times. Treat button indexes as valid only when they come from the same extraction/verification script that selected the item. ## Selector Heuristics Prefer selectors in this order: 1. `aria-label` with a meaningful stable phrase. 2. `role` plus text or nested semantic structure. 3. `contenteditable`, `placeholder`, `name`, `type`, `data-testid`. 4. Stable link URL patterns or form attributes. 5. Text filter passed separately: `co browser click_element_by_selector <selector> --text='...'`. 6. Generated classes only as a last resort and only after screenshot/context evidence shows no semantic alternative. Do not use CSS classes that look generated as the primary selector for social feeds. They usually change between sessions. Do not assume a visible control is a `<button>`. Social apps often use `div[role="button"]` or frame-provided controls for labels such as `Start a post`. Match by semantic role and visible text before falling back to tag selectors. Do not hardcode temporary context locators such as `[data-browser-agent-id="318"]` into a target skill. They are useful evidence for one saved context, not stable selectors. When there are many matching buttons, prefer one of these patterns: ```bash # extract JS returns action_index among all visible matching action buttons co browser click_element_by_selector button --index=<action_index> --text='<Action>' ``` or: ```bash co browser click_element_near_selector '<editor/input selector>' button \ --target_text='<Submit/Post/Comment>' \ --container_selector='<item/form container selector>' ``` The first pattern is for opening the item editor/action. The second pattern is for final submit near an already active editor. ## Extraction Script Contract The extraction script should look like: ```js (args) => { const maxItems = args.maxItems || 3; // scan visible containers return { ok: true, items: [ { item_index: 0, author: "Name or channel", text: "Exact visible content used for generation", text_hash: "stable hash of normalized text", action_index: 0, has_action: true, visible_bounds: { x: 0, y: 0, width: 0, height: 0 } } ], selected_item: null }; } ``` Rules: - Normalize whitespace before hashing. - Hash the exact text sent to the content-generation skill. - Return the action index among the same selector/text set that `click_element_by_selector(...)` will use. - Include enough identity to disambiguate repeated items: author/title/channel, timestamp if stable, URL/id if available, visible bounds. - Return `ok: false` with a reason when no safe item is found. - Do not click from JS. - Filter things that should never be acted on: ads, promoted/sponsored items, sidebars, notifications, navigation, suggested people, unrelated cards. The extractor should be conservative. It is better to return `ok: false` than to act on the wrong item. ## Verify Script Contract The verify script should look like: ```js (args) => { const expectedHash = args.expected_text_hash; const expectedText = args.expected_text; const expectedAuthor = args.expected_author; // rescan visible DOM return { ok: true, matched_item: { item_index: 0, author: expectedAuthor, text: expectedText, text_hash: expectedHash, action_index: 0, has_action: true } }; } ``` Rules: - `ok: true` only when author/title and text/hash match. - Include the current action index from the current DOM. - If the feed moved, the modal changed, or the item is gone, return `ok: false`. - The browser workflow must stop on `ok: false`. - Return `scanned[]` on failure so the next debugging pass can see what the script did find. - Do not accept fuzzy matches for final submit. Fuzzy matching is allowed only during discovery, not during action. ## Match Contract For any workflow where content is generated for a page item, keep this state: ```text verified_item_author_or_title verified_item_text verified_item_text_hash generated_content ``` Required gates: 1. Extract JS returns the item text/hash. 2. Content generation receives exactly `verified_item_text`. 3. Verify JS confirms the same item before opening the editor/form/action. 4. Verify JS confirms the same item again before final submit/post/upload. If any gate fails, stop before click/type/submit. For generated replies/comments, the prompt to the writer skill must receive exactly `verified_item_text`, not a paraphrase, screenshot summary, or page-wide text. This prevents "good comment, wrong item" failures. ## Browser Action Pattern After JS verification, use generic browser tools: ```bash co browser click_element_by_selector button --index=<matched_item.action_index> --text=Comment co browser type_text_by_selector '<editor selector>' '<generated_content>' co browser click_element_near_selector '<editor selector>' button \ --target_text='<submit text>' --anchor_index=-1 \ --container_selector='<item container selector>' \ --require_anchor_text=true --wait_ms=2500 --verify_anchor_text_cleared=true ``` For uploads, use the equivalent stable input/button selectors and keep the same extraction/verification gates around the item or form being acted on. Local file uploads cannot be completed by browser JavaScript alone. Browser-side JS cannot set a real local path on `<input type="file">`. Use generic browser upload primitives: ```bash co browser upload_file_after_click_by_selector '<upload button selector>' '<absolute local file path>' \ --text='<Upload/Add file/Upload from computer>' --frame_name='<optional frame name>' ``` or, when a file input is directly available: ```bash co browser upload_file_by_selector 'input[type="file"]' '<absolute local file path>' \ --frame_name='<optional frame name>' ``` Wrap uploads with skill-local JS scanners such as `verify-upload-controls.js` and `verify-upload-complete.js` that report visible upload triggers, `input[type=file]` metadata, media previews, and upload completion state. Save screenshot/context immediately after upload. If upload verification fails, stop before submit/publish. For final side-effect actions in frame/shadow UIs, prefer a skill-local JS click script over generic CSS clicking: ```bash co browser run_frame_script .co/skills/<target-skill>/scripts/click-<action>.js \ '{"expected_text":"...","expected_text_hash":"..."}' --frame_name='<optional frame name>' ``` The click script must verify the exact editor/composer content or item identity, find the enabled visible submit button near that verified context, click exactly once, and return `ok: true` only when it actually clicked. The workflow must require both `matched_frame.result.ok === true` and `matched_frame.result.clicked === true` before reporting success. For multi-step publish flows, model every side-effect button as a separate one-shot gate. Example: article editor `Next` is not the same action as final `Publish`. Use `click-next.js` after exact editor verification, save the share/publish surface context, then use a separate `click-final-publish.js` that verifies the expected title/content is visible in that final surface before clicking `Publish`. For testing multi-step publish flows without publishing, add an explicit debug flag such as `debug_next: true`: it may click the non-final `Next`, save context/screenshot of the final publish surface, report visible buttons, and stop before final `Publish`. For reactions/likes, treat the reaction as the same pattern with no generated content: ```text extract reaction targets -> verify same target by author/text/hash -> click verified reaction index -> verify the unreacted selector disappeared for that target ``` Example target skill: ```text .co/skills/linkedin-thumbup/ SKILL.md scripts/ extract-like-targets.js verify-like-target.js ``` The reaction script is just another action variant. Do not add `like_linkedin_post()` to the browser core. ## Skill-Local JS Pattern Each target skill should include scripts next to `SKILL.md`: ```text .co/skills/<target-skill>/scripts/extract-items.js .co/skills/<target-skill>/scripts/verify-item.js ``` Use the templates from this skill when starting: ```text .co/skills/browser-workflow-skill-builder/scripts/extract-items-template.js .co/skills/browser-workflow-skill-builder/scripts/verify-item-template.js ``` The scripts must be page functions: ```js (args) => { return { ok: true }; } ``` They are run with: ```bash co browser run_page_script .co/skills/<target-skill>/scripts/extract-items.js '{"maxItems":3}' ``` For frame-aware workflows, run the same script with: ```bash co browser run_frame_script .co/skills/<target-skill>/scripts/verify-item.js \ '{"expected_text_hash":"..."}' --frame_url_contains='' --frame_name='' ``` Extraction and verification scripts should be deterministic and small. Avoid network requests, timers, mutation, clicks, typing, or localStorage writes. They should read DOM only and return JSON. Final `click-<action>.js` scripts are the exception: they may click only after exact verification passes. They must never perform broad searches such as "first Post button" without checking the matching editor/item context. If modals, editors, or submit buttons may live inside open shadow roots, include a small deep traversal helper in the script: ```js const queryAllDeep = (root, selector) => { const out = [...root.querySelectorAll(selector)]; for (const el of root.querySelectorAll('*')) { if (el.shadowRoot) out.push(...queryAllDeep(el.shadowRoot, selector)); } return out; }; ``` Browser-side scripts run in the page, not Node. Do not use `require`, `fs`, `process`, or other Node APIs inside `.co/skills/<target-skill>/scripts/*.js`. ## Rich Text Editors Do not assume a rich text editor renders Markdown when raw Markdown is typed or inserted. Live LinkedIn article testing showed raw Markdown stayed literal (`##`, `**bold**`, bullets, and Markdown links were displayed as text). For rich editors: 1. Parse source content outside the browser when needed. 2. Produce both rendered HTML and expected visible text. 3. Insert HTML through a paste-like path, `document.execCommand('insertHTML', ...)`, or direct `innerHTML` only inside a guarded fill script. 4. Verify against expected visible text, not the original Markdown syntax. 5. Return formatting evidence from verification: heading count, bold/italic count, link count, unordered/ordered list count, list item count, and code/pre count. Example script contract: ```text