actionbook-web-test

name	actionbook-web-test
description	Run browser-based web tests against websites using Actionbook CLI. Activate when the user wants to test a website workflow, run smoke tests, verify a user flow, check if a web application works, run regression tests, or validate browser-based interactions. Supports test definition, execution, assertion, reporting, and json-ui visual report generation.

When to Use This Skill

Activate when the user:

• Asks to "test", "verify", "check", or "validate" a website workflow
• Wants to run smoke tests or health checks on a web application
• Needs to verify a user flow works end-to-end (login, checkout, search, etc.)
• Asks to "run regression tests" or "does this still work?"
• Wants to confirm a deployment didn't break functionality
• Needs to monitor a website's functionality on a schedule
• Builds browser-based test suites without writing Playwright/Cypress code

What actionbook-web-test Provides

actionbook-web-test transforms web tests from coded test scripts into declarative YAML workflows executed by AI agents via Actionbook CLI.

Benefit	How
AI-native recovery	When a selector fails, the agent snapshots the live page and finds the equivalent element
Actionbook-managed selectors	Pre-verified selectors with health scores — no manual maintenance
Cross-project reusability	YAML workflows work anywhere Actionbook CLI is installed
No test framework required	No Playwright/Cypress/Jest setup — just actionbook browser commands
Human-readable tests	YAML workflows are readable by non-developers
Visual test reports	json-ui powered HTML reports with metrics, step details, and failure screenshots

Test Workflow Format

Tests are defined as YAML files in a tests/ directory. Each file describes one test workflow.

name: example-test
description: What this test verifies
url: https://example.com
tags: [smoke, critical]
timeout: 30000  # ms, default 30000

# Pre-fetch verified selectors from Actionbook
actions:
  - "example.com:/:default"

# Environment variables (support {{env.VAR}} templates)
env:
  USERNAME: "test-user"
  PASSWORD: "{{env.TEST_PASSWORD}}"

# Browser setup options
setup:
  headless: true
  auto_dismiss_dialogs: true
  no_animations: true

# Ordered test steps
steps:
  - name: Open page
    action: open
    url: "https://example.com"

  - name: Verify loaded
    assert:
      - type: element-exists
        selector: "#main-content"

Full schema reference: workflow-format.md

Step Types

Each step has a name and either an action (browser command) or assert (verification checks).

Actions → CLI Command Mapping

Action	CLI Command	Required Fields
open	actionbook browser open <url>	url
click	actionbook browser click "<selector>"	selector
fill	actionbook browser fill "<selector>" "value"	selector, value
type	actionbook browser type "<selector>" "value"	selector, value
select	actionbook browser select "<selector>" "value"	selector, value
hover	actionbook browser hover "<selector>"	selector
press	actionbook browser press <key>	key
wait	actionbook browser wait "<selector>"	selector
wait-fn	actionbook browser wait-fn "<expression>"	expression
wait-idle	actionbook browser wait-idle	—
wait-nav	actionbook browser wait-nav	—
snapshot	actionbook browser snapshot	—
screenshot	actionbook browser screenshot	—
text	actionbook browser text [selector]	selector (optional)
eval	actionbook browser eval "expression"	expression
upload	actionbook browser upload "<selector>" "<file-path>"	selector, file_path
scroll	actionbook browser scroll <direction>	direction (up/down/top/bottom/to)
emulate	actionbook browser emulate <device>	device
info	actionbook browser info "<selector>"	selector
console	actionbook browser console --level error	—
close	actionbook browser close	—

Step Options

- name: Accept cookies if present
  action: click
  selector: "[data-testid='cookie-accept']"
  on_fail: continue     # skip | abort (default) | continue
  retry: 1              # override retry count
  timeout: 5000         # step-level timeout override
  condition: element-exists "[data-testid='cookie-banner']"

Assertion Types

Steps can include assert blocks to verify expected outcomes. Common types listed below; see assertion-types.md for the complete reference.

Type	Description	CLI Mapping
text-contains	Element text contains string	browser text "<selector>" + string check
text-equals	Element text exactly matches	browser text "<selector>" + exact match
text-matches	Text matches regex pattern	browser text "<selector>" + regex
url-contains	Current URL contains string	browser eval "location.href"
url-equals	Current URL exactly matches	browser eval "location.href"
element-exists	Element present in DOM	browser wait "<selector>" --timeout 5000
element-not-exists	Element NOT present	browser eval "!document.querySelector(...)"
element-visible	Element is visible	browser eval visibility check
element-hidden	Element hidden or absent	Inverse of element-visible
element-count	Element count matches condition	browser eval "querySelectorAll(...).length"
attribute-equals	Element attribute matches	browser eval "getAttribute(...)"
attribute-contains	Attribute contains substring	browser eval "getAttribute(...)"
page-title-contains	Page title contains string	browser eval "document.title"
eval-truthy	JS expression evaluates truthy	browser eval "<expression>"
console-no-errors	No JS errors in console	browser console --level error
network-no-failures	No HTTP 4xx/5xx errors	Network monitoring via CDP
screenshot-match	Visual regression comparison	browser screenshot + pixel diff
performance-under	Performance metric under threshold	browser eval performance timing

Assertion Examples

# Text assertions
- name: Verify welcome message
  assert:
    - type: text-contains
      selector: "[data-testid='welcome']"
      value: "Welcome back"

# URL assertions
- name: Verify redirect
  assert:
    - type: url-contains
      value: "/dashboard"

# Element count with operator
- name: Verify search results
  assert:
    - type: element-count
      selector: ".search-result"
      operator: ">="
      value: 5

# JS evaluation
- name: Verify cart state
  assert:
    - type: eval-truthy
      expression: "JSON.parse(localStorage.getItem('cart')).items.length > 0"

Execution Flow

Step 0: Pre-flight Checks

Before running any test, verify the environment is ready:

# 1. Check browser connection
actionbook browser status
# If no browser → open one with setup flags

# 2. Verify target site is reachable (fast check, no rendering)
actionbook browser fetch <url> --format text --timeout 10000 --lite
# If fails → report site unreachable, skip all tests for this domain

# 3. Start console error monitoring
actionbook browser console --level error --duration 0 &
# Capture JS errors throughout the test session

Pre-flight failures should be reported clearly — distinguish "test failed" from "environment broken".

Step 1: Discover

Parse YAML workflow files from the tests/ directory. Filter by --filter flag (matches tags or name).

# Run all tests
/actionbook-web-test run tests/

# Run smoke tests only
/actionbook-web-test run tests/smoke/

# Filter by tag
/actionbook-web-test run tests/ --filter critical

Step 2: Setup

For each workflow:

1. Pre-fetch selectors: actionbook search + actionbook get "<action-id>" for each entry in actions
2. Resolve template variables ({{env.VAR}}, {{timestamp}}, etc.)
3. Restore auth state if setup.profile is specified (cookies/storage from previous session)
4. Open browser with configured flags:

   actionbook --auto-dismiss-dialogs --no-animations browser open <url>
   

5. If setup.emulate is set, apply device emulation:

   actionbook browser emulate iphone-14
   

Step 3: Execute

For each step in order:

1. Check condition (if present) — skip step if condition is false
2. Pre-check element (for interaction steps): use info to verify element state

   actionbook browser info "<selector>"
   # Returns: bounding box, visibility, enabled state, attributes
   

3. Translate action to actionbook browser CLI command
4. Execute the command
5. If step has assert block: run each assertion check
6. On PASS: log success, continue to next step
7. On FAIL: enter recovery (Step 4) or handle per on_fail setting
8. After the last step of each test (regardless of PASS/FAIL/SKIP): capture a screenshot of the current page state. This screenshot will be embedded in the report under that test's section.

   # Auto-capture at end of each test — save to a per-test temp file
   actionbook browser screenshot /tmp/test-<test-name>-final.png
   base64 -i /tmp/test-<test-name>-final.png | tr -d '\n' > /tmp/test-<test-name>-final-b64.txt
   

Smart Waits: Always prefer wait-fn over eval "setTimeout":

# BAD: blind delay
actionbook browser eval "new Promise(r => setTimeout(r, 800))"

# GOOD: wait for condition
actionbook browser wait-fn "document.querySelector('#sidebar').offsetWidth < 100" --timeout 5000

# GOOD: wait for element state change
actionbook browser wait-fn "document.querySelector('.loading').style.display === 'none'" --timeout 10000

# GOOD: wait for URL change after click
actionbook browser wait-fn "window.location.href.includes('/dashboard')" --timeout 10000

Step 4: Recover

Error	Recovery Strategy	Retries
Selector not found	snapshot → find equivalent selector → retry step	1
Navigation timeout	wait "<selector>" --timeout 15000 → retry (use wait instead of wait-idle in extension mode)	1
Element not clickable	scroll to "<selector>" + wait → retry	1
Element not visible	info "<selector>" to check state → scroll/wait → retry	1
Login wall detected	Check cookies list → if no auth, pause for user to log in, resume	0 (manual)
Anti-bot / CAPTCHA	Add --stealth, fingerprint rotate → retry	1
Assertion failure	Screenshot + log actual vs expected (genuine failure)	0
Browser crash	Re-open browser, restart from failed step	1

Selector recovery detail:

When a selector from Actionbook or the workflow YAML fails at runtime:

# 1. Snapshot the live page
actionbook browser snapshot --interactive --compact --max-tokens 800

# 2. Find the equivalent element in the snapshot output
# 3. Use the new selector to retry the failed step

Step 5: Teardown

# Capture any accumulated JS errors before closing
actionbook browser console --level error

# Close browser
actionbook browser close

Always close the browser, even on test failure.

Step 6: Report

Generate test results in the requested format. See Report Generation for details.

Selector Strategy

Selectors come from three sources: Actionbook API (verified, health-scored), workflow YAML (static), and live snapshot (runtime fallback).

Priority	Source	When to Use
1	actionbook search + get	Build phase — discover and pre-fill selectors for target pages
2	data-testid / aria-label	Stable attributes written directly in workflow YAML
3	CSS selector	Specified directly in workflow steps
4	actionbook browser snapshot	Runtime fallback when all above selectors fail

Test Construction Flow

Tests are built using Actionbook selectors, not hand-written:

# 1. Search for the target page's action
actionbook search "reddit homepage sidebar navigation search" --domain reddit.com

# 2. Get the full page structure with verified selectors
actionbook get "reddit.com:/search/:default"
# → Returns page structure with inline CSS selectors:
#   Sidebar container: #left-sidebar-container
#   Collapse button: #flex-nav-collapse-button
#   Feed sort links: a[href*='/hot/?feed=home']