ios-sim

name	ios-sim
description	Control the iOS Simulator — tap, type, swipe, screenshot, record video, install and launch apps. Use when interacting with the iOS simulator during React Native development. Triggers on "simulator", "tap", "swipe", "screenshot of simulator", "install app", "launch app", "what's on screen", "record video", "accessibility tree", "tap by label", "tap by id", "list elements", "go back", "scroll to top", "scroll to bottom".
allowed-tools	Bash, Read, Agent

Skill: ios-sim

On load: Read ../../.claude-plugin/plugin.json from this skill's base directory. Display ios-sim v{version} before proceeding.

Control the iOS Simulator through shell scripts wrapping xcrun simctl and AXe.

IMPORTANT: Never load screenshots or accessibility trees in the main context.
Always dispatch a subagent for visual/inspection tasks.

---

Used By

Skill	What it uses
a11y-audit	ui.sh describe-all for accessibility tree capture
layout-check	capture.sh view + ui.sh describe-all for screenshot and element positions
diagnose	capture.sh view to read error text from the simulator screen
deeplink-test	Screenshot capture to verify the screen after firing a deep link

---

Prerequisites

• macOS with Xcode installed
• A booted iOS simulator (xcrun simctl list devices to check)
• AXe (brew install cameroncooke/axe/axe) — required for all ui.sh commands (tap, tap-label, tap-id, type, swipe, describe-all, describe-point, list, back, scroll). capture.sh view works without AXe.
• See ${CLAUDE_SKILL_DIR}/references/troubleshooting.md if anything is missing

Scripts

All scripts live at ${CLAUDE_SKILL_DIR}/scripts/. Run them with Bash.

Routing Table

Intent	Workflow	Why
See what's on screen	view	Image stays in subagent
Find UI elements/coordinates	inspect	JSON tree stays in subagent
Multi-step UI interaction	interact	Entire loop stays in subagent
Simple one-shot command	direct	No image/tree involved

Direct Commands (safe for main context)

Intent	Script	Example
Get booted simulator ID	device.sh booted	device.sh booted
Open Simulator app	device.sh open	device.sh open
Save screenshot to file	capture.sh screenshot <path>	capture.sh screenshot /tmp/shot.png
Start video recording	capture.sh record	capture.sh record
Stop video recording	capture.sh stop	capture.sh stop
Install app bundle	app.sh install <path>	app.sh install /path/to/App.app
Launch app by bundle ID	app.sh launch <id>	app.sh launch com.example.app
Tap element by accessibility label	ui.sh tap-label <label>	ui.sh tap-label "Login"
Tap element by accessibility ID	ui.sh tap-id <id>	ui.sh tap-id "submit-button"
List on-screen elements (Controls/Content)	ui.sh list	ui.sh list
Tap the back/navigation button	ui.sh back	ui.sh back
Scroll to top or bottom of list	ui.sh scroll top|bottom	ui.sh scroll top

---

Workflows

1. View (screenshot analysis)

When: "What's on the simulator screen?", "How does it look?", "Is there an error?"

Main agent never loads the image. Haiku does the analysis.

Dispatch Agent:
  subagent_type: general-purpose
  model: haiku
  description: "ios-sim: analyze screenshot"
  prompt: |
    1. Run: ${CLAUDE_SKILL_DIR}/scripts/capture.sh view
       This outputs a file path to a compressed JPEG.
    2. Read that file path with the Read tool to see the image.
    3. Analyze and return:
       - Overview: What app/screen is visible (1-2 sentences)
       - Key elements: Buttons, text, inputs, navigation items
       - State: Errors, loading, forms filled, current tab
       - Coordinates: Notable interactive elements with approximate point positions
    4. If the user asked something specific, answer that directly.
    Return text only. Be concise.

    USER QUESTION: [insert user's question here]

2. Inspect (accessibility tree)

When: "What elements are on screen?", "Find the login button", "Where should I tap?"

The accessibility tree JSON can be massive. Parse it in a subagent.

Dispatch Agent:
  subagent_type: general-purpose
  model: haiku
  description: "ios-sim: inspect UI elements"
  prompt: |
    1. Run: ${CLAUDE_SKILL_DIR}/scripts/ui.sh describe-all
       This outputs the full accessibility tree as JSON.
    2. Parse the JSON and return a structured summary:
       - Screen dimensions (from root frame)
       - Interactive elements: buttons, text fields, switches, links
         Format each as: "Label" [type] at (x, y) — size WxH
       - Current focus/selection state
       - Navigation structure (tabs, headers, back buttons)
    3. If looking for a specific element, report its exact coordinates.
    Return text only. Be concise.

    LOOKING FOR: [insert what the user needs to find]

3. Interact (multi-step UI automation)

When: "Tap the login button", "Fill in the form", "Navigate to settings"

Combines view + inspect + actions in a subagent loop. The entire interaction stays isolated — main context only gets the final result.

Dispatch Agent:
  subagent_type: general-purpose
  description: "ios-sim: UI interaction"
  prompt: |
    You are automating the iOS Simulator. Scripts are at:
    ${CLAUDE_SKILL_DIR}/scripts/

    Available commands:
    - capture.sh view               → compressed screenshot (read the output path to see it)
    - ui.sh describe-all            → full accessibility tree JSON
    - ui.sh describe-point X Y      → element at coordinates
    - ui.sh tap X Y                 → tap at point coordinates
    - ui.sh tap X Y --duration S    → long press
    - ui.sh type "text"             → type ASCII text (max 500 chars)
    - ui.sh swipe X1 Y1 X2 Y2      → swipe gesture
    - ui.sh tap-label "label"       → tap element by accessibility label (no coordinate lookup needed)
    - ui.sh tap-id "id"             → tap element by accessibility ID (no coordinate lookup needed)
    - ui.sh list                    → compact table of on-screen elements grouped by Controls/Content
    - ui.sh back                    → heuristic back-button finder and tap (scores by label/position)
    - ui.sh scroll top|bottom       → repeated swipes with stabilization detection (max 10 swipes)

    TASK: [insert what the user wants to do]

    WORKFLOW:
    1. First capture.sh view to see current state
    2. Use ui.sh describe-all if you need exact coordinates
    3. Perform the requested actions
    4. capture.sh view again to verify the result
    5. Return a text summary of what you did and the final state

    RULES:
    - Use POINT coordinates from the accessibility tree, not pixel coordinates
    - After each action, verify the result before proceeding
    - If something fails, try describe-all to re-orient
    - Return text summary only — do not include base64 image data

---

Tips

• All scripts auto-detect the booted simulator. Pass --udid <UUID> to target a specific device.
• Screenshots are 3x pixel resolution. The accessibility tree reports point coordinates. Always use point coordinates for tap/swipe.
• ui.sh type only accepts ASCII printable characters (max 500 chars).
• Video recording runs in the background. Use capture.sh stop to finish.
• For the interact workflow, omit model to use the user's current model (better reasoning for complex multi-step tasks).

Context Efficiency

Item	Size	In Main Context?
Screenshot JPEG	~100-300 KB	NEVER — haiku subagent only
Accessibility tree JSON	~10-100 KB	NEVER — subagent only
Subagent text summary	~200-800 chars	YES
Direct commands (device, app)	~50-200 chars	YES