baguette-ios-simulator

name	baguette-ios-simulator
description	Headless iOS Simulator manager with host-side HID input injection, 60fps streaming, and device farm web UI for iOS 26
triggers	["control iOS simulator headlessly","inject touch input into simulator","stream simulator screen","manage simulator device farm","tap swipe gesture simulator programmatically","boot simulator without GUI","simulator web UI dashboard","simulate multi-finger pinch zoom on iOS"]

Baguette iOS Simulator Manager

Skill by ara.so — Daily 2026 Skills collection.

Baguette is a Swift CLI tool that creates, boots, and shuts down iOS Simulator devices, streams their screens at 60fps, and injects taps/swipes/multi-finger gestures entirely headlessly — no Simulator.app GUI required. It also serves a self-contained web UI for single-device and multi-device (farm) control.

Requirements

• Apple Silicon Mac only
• macOS 15+
• Xcode 26 (links against private SimulatorKit/CoreSimulator frameworks)

Install

brew install tddworks/tap/baguette

Build from Source

git clone https://github.com/tddworks/baguette
cd baguette
make           # release build via ./build.sh
swift test     # run the test suite

Key CLI Commands

Device Management

# List all simulators (default + custom sets)
baguette list

# Boot a simulator headlessly (no Simulator.app window)
baguette boot --udid <UDID>

# Shutdown a simulator
baguette shutdown --udid <UDID>

Screen Streaming

# Stream frames to stdout as MJPEG (default)
baguette stream --udid <UDID> --fps 60 --format mjpeg

# Stream as H.264/AVCC
baguette stream --udid <UDID> --fps 60 --format avcc

# Pipe MJPEG to ffplay for local preview
baguette stream --udid <UDID> --fps 30 --format mjpeg | ffplay -i -

One-Shot Gesture Input

Coordinates are in device points; --width/--height are the simulator screen size in points.

# Tap at a point
baguette tap --udid <UDID> --x 219 --y 478 --width 438 --height 954

# Tap with custom duration
baguette tap --udid <UDID> --x 219 --y 478 --width 438 --height 954 --duration 0.1

# Swipe from top to bottom (scroll down)
baguette swipe --udid <UDID> \
  --startX 219 --startY 190 \
  --endX 219 --endY 760 \
  --width 438 --height 954

# Pinch to zoom in (startSpread < endSpread)
baguette pinch --udid <UDID> \
  --cx 219 --cy 478 \
  --startSpread 60 --endSpread 200 \
  --width 438 --height 954

# Two-finger pan
baguette pan --udid <UDID> \
  --x1 175 --y1 478 \
  --x2 263 --y2 478 \
  --dx 0 --dy -100 \
  --width 438 --height 954

# Hardware buttons
baguette press --udid <UDID> --button home
baguette press --udid <UDID> --button lock

Streaming Gesture Input (stdin JSON)

For real-time or scripted gesture sequences, pipe newline-delimited JSON to baguette input:

baguette input --udid <UDID>

Each line gets an ack: {"ok":true} or {"ok":false,"error":"..."}.

Web UI Server

# Start the web server (default port 8421)
baguette serve

# Custom port and host
baguette serve --port 9000 --host 0.0.0.0

# Custom device set
baguette serve --port 8421 --device-set /path/to/device-set

# Open single-device dashboard
open http://localhost:8421/simulators

# Open multi-device farm dashboard
open http://localhost:8421/farm

DeviceKit Chrome/Bezel Data

# Print bezel layout JSON for a booted device
baguette chrome layout --udid <UDID>

# Write composite PNG (device screenshot + bezel) to stdout
baguette chrome composite --udid <UDID> > screenshot.png

# By device name (no UDID needed)
baguette chrome layout --device-name "iPhone 17 Pro"
baguette chrome composite --device-name "iPhone 17 Pro" > iphone17pro_bezel.png

Wire Protocol — Streaming Input via stdin

Send newline-delimited JSON to baguette input --udid <UDID>:

{"type":"tap", "x":219, "y":478, "width":438, "height":954, "duration":0.05}

{"type":"swipe", "startX":219, "startY":760, "endX":219, "endY":190, "width":438, "height":954, "duration":0.3}

{"type":"touch1-down"    
    
      

      
      
        

      

 
 

  

WebSocket Protocol (for Web UI / Custom Clients)

Connect to ws://localhost:8421/simulators/<UDID>/stream?format=mjpeg (or avcc).

Server → Client (binary frames)

• MJPEG: raw JPEG bytes per message
• AVCC: 1-byte tag prefix:
  • 0x01 — avcC description
  • 0x02 — keyframe
  • 0x03 — delta frame
  • 0x04 — JPEG seed frame (renders before H.264 IDR)

Client → Server (text JSON)

{"type":"set_bitrate", "bps": 2000000}
{"type":"set_fps",     "fps": 30}
{"type":"set_scale",   "scale": 0.5}
{"type":"force_idr"}
{"type":"snapshot"}

Gesture input messages (same format as stdin wire protocol above) are also accepted over the WebSocket.

Web UI Routes

Method	Path	Description
GET	/	Redirects → /simulators
GET	/simulators	Device list HTML
GET	/simulators.json	{running: [...], available: [...]}
GET	/simulators/:udid	Stream page HTML
POST	/simulators/:udid/boot	Boot device
POST	/simulators/:udid/shutdown	Shutdown device
GET	/simulators/:udid/chrome.json	Bezel layout JSON
GET	/simulators/:udid/bezel.png	Rasterized bezel PNG
WS	/simulators/:udid/stream	Live stream + input
GET	/farm	Multi-device farm HTML

Code Examples

Scripting Gestures from Swift

import Foundation

// Build a gesture sequence as newline-delimited JSON
func makeGestureScript() -> String {
    let gestures: [[String: Any]] = [
        // Boot sequence: tap the app icon
        ["type": "tap", "x": 100, "y": 200, "width": 390, "height": 844, "duration": 0.05],
        // Scroll down in a list
        ["type": "swipe", "startX": 195, "startY": 600,
         "endX": 195, "endY": 200, "width": 390, "height": 844, "duration": 0.4],
        // Pinch to zoom
        ["type": "pinch", "cx": 195, "cy": 422,
         "startSpread": 50, "endSpread": 180, "width": 390, "height": 844],
        
        [: , : ]
    ]
     gestures.compactMap { dict -> ? 
          data   .data(withJSONObject: dict)  {   }
         (data: data, encoding: .utf8)
    }.joined(separator: )
}


 (: , : )   {
     process  ()
    process.executableURL  (fileURLWithPath: )
    process.arguments  [, , udid]

     inputPipe  ()
     outputPipe  ()
    process.standardInput  inputPipe
    process.standardOutput  outputPipe

     process.run()

     inputData  (script  ).data(using: .utf8)
    inputPipe.fileHandleForWriting.write(inputData)
    inputPipe.fileHandleForWriting.closeFile()

    process.waitUntilExit()

     outputData  outputPipe.fileHandleForReading.readDataToEndOfFile()
     acks  (data: outputData, encoding: .utf8)  
    ()
}

Listing and Booting Simulators

import Foundation

struct SimulatorInfo: Codable {
    let running: [Device]
    let available: [Device]

    struct Device: Codable {
        let udid: String
        let name: String
        let os: String
        let state: String
    }
}

func fetchSimulators(port: Int = 8421) async throws -> SimulatorInfo {
    let url = URL(string: "http://localhost:\(port)/simulators.json")!
    let (data, _) = try await URLSession.shared.data(from: url)
    return try JSONDecoder().decode(SimulatorInfo.self, from: data)
}

func bootSimulator(udid: String, port: Int = 8421) async  {
     url  (string: )
     request  (url: url)
    request.httpMethod  
     (, response)    .shared.data(for: request)
     (response  ).statusCode    {
         (.badServerResponse)
    }
}


 {
     sims    fetchSimulators()
    ()
      first  sims.available.first {
          bootSimulator(udid: first.udid)
        ()
    }
}

Connecting to a Live Stream WebSocket

import Foundation

func connectToSimulatorStream(udid: String, port: Int = 8421) {
    let url = URL(string: "ws://localhost:\(port)/simulators/\(udid)/stream?format=mjpeg")!
    let session = URLSession(configuration: .default)
    let task = session.webSocketTask(with: url)
    task.resume()

    // Reduce bitrate and fps for a thumbnail
    let setFps = URLSessionWebSocketTask.Message.string(
        #"{"type":"set_fps","fps":15}"#
    )
    let setBitrate = URLSessionWebSocketTask.Message.string(
        #"{"type":"set_bitrate","bps":500000}"#
    )
    task.send(setFps) { _ in }
    task.send(setBitrate) { _ in }

    // Receive MJPEG frames
    func receiveFrame() {
        task.receive { result in
            switch result {
            case .success(.data(let jpegData)):
                // jpegData is a raw JPEG — display in NSImageView / UIImageView
                print("Received frame: \(jpegData.count) bytes")
                receiveFrame()
             .success(.string( text)):
                ()
                receiveFrame()
             .failure( error):
                ()
            }
        }
    }
    receiveFrame()
}

Sending Gestures over WebSocket

func sendTapOverWebSocket(task: URLSessionWebSocketTask, x: Double, y: Double,
                           width: Double, height: Double) {
    let gesture: [String: Any] = [
        "type": "tap",
        "x": x, "y": y,
        "width": width, "height": height,
        "duration": 0.05
    ]
    guard let data = try? JSONSerialization.data(withJSONObject: gesture),
          let json = String(data: data, encoding: .utf8) else { return }
    task.send(.string(json)) { error in
        if let error { print("Send error: \(error)") }
    }
}

func sendPinchOverWebSocket(task: URLSessionWebSocketTask,
                             cx: Double, cy: Double,
                             startSpread: Double, endSpread: Double,
                             : , : ) {
     gesture: [: ]  [
        : ,
        : cx, : cy,
        : startSpread,
        : endSpread,
        : width, : height
    ]
      data   .data(withJSONObject: gesture),
           json  (data: data, encoding: .utf8)  {  }
    task.send(.string(json)) {   }
}

Shell Script: Full Automation Flow

#!/usr/bin/env bash
set -euo pipefail

# Start the baguette server in background
baguette serve --port 8421 &
SERVER_PID=$!
sleep 1

# Get the first available simulator UDID
UDID=$(baguette list | grep "iPhone 17 Pro" | head -1 | awk '{print $1}')
echo "Using simulator: $UDID"

# Boot it
baguette boot --udid "$UDID"
sleep 3

# Run a tap sequence
baguette tap --udid "$UDID" --x 195 --y 422 --width 390 --height 844

# Pipe a gesture script
cat <<EOF | baguette input --udid "$UDID"
{"type":"tap","x":195,"y":200,"width":390,"height":844}
{"type":"swipe","startX":195,"startY":600,"endX":195,"endY":200,"width":390,"height":844,"duration":0.3}
{"type":"button","button":"home"}
EOF

# Capture a screenshot with bezel
baguette chrome composite --udid "$UDID" > screenshot.png
echo "Screenshot saved to screenshot.png"

# Shutdown
baguette shutdown --udid "$UDID"
kill $SERVER_PID

Configuration

Environment Variables

Variable	Description
BAGUETTE_WEB_DIR	Override the served web root (e.g. point to Sources/Baguette/Resources/Web for live UI iteration without rebuilding)

# Live-iterate on web UI without rebuilding
export BAGUETTE_WEB_DIR="$(pwd)/Sources/Baguette/Resources/Web"
baguette serve

Device Sets

# Use a custom simulator device set
baguette serve --device-set /path/to/my-device-set
baguette list  # uses default device set

Common Patterns

CI/CD: Boot, Test, Shutdown

#!/usr/bin/env bash
UDID=$(xcrun simctl list devices available -j | \
  python3 -c "import sys,json; devs=[d for v in json.load(sys.stdin)['devices'].values() for d in v if 'iPhone 17' in d['name'] and d['isAvailable']]; print(devs[0]['udid'])")

baguette boot --udid "$UDID"

# Run your XCTest suite or UI tests here
xcodebuild test -scheme MyApp -destination "id=$UDID"

baguette shutdown --udid "$UDID"

Streaming to a File

# Capture 10 seconds of MJPEG to file
baguette stream --udid <UDID> --fps 30 --format mjpeg \
  | head -c $((10 * 30 * 50000)) > recording.mjpeg

# Convert to MP4 with ffmpeg
baguette stream --udid <UDID> --fps 30 --format mjpeg \
  | ffmpeg -f mjpeg -i - -t 10 -c:v libx264 output.mp4

Multi-Finger Gesture Sequence (Real-Time)

# Send a real-time 2-finger swipe up (simulate pull-to-refresh for two fingers)
cat <<'EOF' | baguette input --udid <UDID>
{"type":"touch2-down","x1":160,"y1":600,"x2":230,"y2":600,"width":390,"height":844}
{"type":"touch2-move","x1":160,"y1":500,"x2":230,"y2":500,"width":390,"height":844}
{"type":"touch2-move","x1":160,"y1":400,"x2":230,"y2":400,"width":390,"height":844}
{"type":"touch2-move","x1":160,"y1":300,"x2":230,"y2":300,"width":390,"height":844}
{"type":"touch2-up","x1":160,"y1":300,"x2":230,"y2":300,"width":390,"height":844}
EOF

Troubleshooting

"Command not found: baguette"

# Ensure Homebrew's bin is in PATH
export PATH="/opt/homebrew/bin:$PATH"
# Or check install location
brew --prefix tddworks/tap/baguette

Simulator Won't Boot

# Check Xcode 26 is selected (required for SimulatorKit frameworks)
xcode-select -p
# Should show Xcode 26 path, e.g. /Applications/Xcode-26.0.app/Contents/Developer
sudo xcode-select -s /Applications/Xcode-26.0.app/Contents/Developer

# Verify the UDID exists
baguette list

Stream Connects but No Frames

# Make sure the simulator is booted (not just created)
baguette boot --udid <UDID>
sleep 3
baguette stream --udid <UDID> --fps 30 --format mjpeg | xxd | head

Input Gestures Not Registering

• Coordinates must be in device points, not pixels. For a 3x display at 390pt wide, pixel width is 1170 — always use point values.
• Ensure --width and --height match the simulator's actual screen size in points (check with baguette chrome layout --udid <UDID>).
• Only home and lock buttons are functional on iOS 26 (press command).

Web UI Not Updating

# Use BAGUETTE_WEB_DIR for live file iteration
export BAGUETTE_WEB_DIR="$(pwd)/Sources/Baguette/Resources/Web"
baguette serve
# Edit .html/.js/.css files and hard-refresh the browser

Port Already in Use

# Check what's using port 8421
lsof -i :8421
# Use a different port
baguette serve --port 9000
open http://localhost:9000/simulators

Build Failures (Source Build)

# Ensure arm64e target and Xcode 26 SDK
xcrun --sdk macosx --show-sdk-path
# Should show macOS 15+ SDK from Xcode 26
swift --version
# Should show Swift 6.1+
make clean && make