| name | flutter-mcp-toolkit-control |
| description | Drive a running Flutter app — tap, scroll, type, fill forms, hot-reload, navigate. Use when you need to interact with the UI. |
When to use
Use this skill when you need to drive a running Flutter app as a user would:
- Tap buttons, icons, list items, or any interactive widget.
- Type text into fields, submit forms, clear inputs.
- Scroll or swipe to reveal off-screen content.
- Navigate between routes programmatically (push, pop, popUntil).
- Dismiss dialogs and bottom sheets.
- Press keyboard keys (Enter, Escape, Tab, arrows, ASCII chars).
- Hot-reload or hot-restart after editing Dart source files.
- Combine reload + screenshot + semantics in one round-trip for fast iteration.
Selectors
Most interaction tools target a widget by ref — a short string like "s_0" returned by semantic_snapshot. For visible widgets, call semantic_snapshot, scan the returned nodes, find the right ref, then pass it. For off-screen targets with stable semantics text or identifier, use reveal_search; it performs a bounded snapshot → match → scroll loop and returns a fresh ref/snapshotId.
Snapshot node fields to filter on:
| Want to find | Scan field | Example value |
|---|
| By visible label / text | label | "Login" |
| By value or hint | value / hint | "user@example.com" |
| By tooltip | tooltip | "Close" |
| By widget key | key | "[<'submitBtn'>]" |
| By semantic role / type | flags or actions | ["tap"] |
Example — find the "Login" button ref:
semantic_snapshot()
→ nodes: [{ref:"s_0", label:"Login", actions:["tap"]}, ...]
tap_widget(ref: "s_0")
Pass snapshotId (from the snapshot response) to any interaction call. If the tree has changed, the call returns stale_snapshot with both IDs so you know to re-snapshot. Refs are only valid against the most recent snapshot.
Recipes
Tap a widget by text
semantic_snapshot()
→ find node where label == "Submit" → ref "s_3"
tap_widget(ref: "s_3", snapshotId: <id>)
Fill a login form
semantic_snapshot() → email ref "s_1", password ref "s_2"
fill_form(fields: [{ref:"s_1", text:"user@example.com"}, {ref:"s_2", text:"secret"}], snapshotId: <id>)
→ one round-trip; stops on first failure
Scroll to find an item
scroll(direction: "down", distance: 300)
semantic_snapshot() → item now visible → ref "s_5"
tap_widget(ref: "s_5")
Reveal an off-screen field by identifier
reveal_search(query: "greeting_input_field", matchBy: "identifier", direction: "down", maxAttempts: 4)
→ returns ref "s_14", snapshotId 2, match, attempts trace
enter_text(ref: "s_14", snapshotId: 2, text: "hello")
Wait for a widget to appear
wait_for(predicate: {kind: "text", text: "Welcome"}, timeoutMs: 8000)
→ returns fresh snapshot when text appears
tap_widget(ref: <ref from wait_for snapshot>)
Navigate to a route
navigate(action: "push", route: "/settings", arguments: {tab: "account"})
semantic_snapshot() → fresh refs in the new screen
Hot reload after a code change
hot_reload_and_capture()
→ screenshot + semantic snapshot + errors in one call
Press the back hardware button
press_key has no Back key. Use navigate(action: "pop") for Navigator pop; handle_dialog(action: "dismiss") for dialogs; press_key(key: "Escape") on desktop.
navigate(action: "pop")
Tool reference
tap_widget
Tap a widget by ref. ref • string • required. snapshotId • integer • optional. connection • object • optional.
{"name": "tap_widget", "arguments": {"ref": "s_3", "snapshotId": 7}}
Returns: {"via": "semantic_action", "ref": "s_3"} — Failures: stale_snapshot, ref_not_found
long_press
Long-press a widget by ref. ref • string • required. snapshotId • integer • optional. connection • object • optional.
{"name": "long_press", "arguments": {"ref": "s_2"}}
Returns: {"via": "semantic_action"} — Failures: stale_snapshot, ref_not_found
enter_text
Enter text into a text field; taps to focus before typing. ref • string • required. text • string • required. snapshotId • integer • optional. connection • object • optional.
{"name": "enter_text", "arguments": {"ref": "s_1", "text": "hello@example.com"}}
Returns: {"via": "editable_state"} — Failures: stale_snapshot, ref_not_found
reveal_search
Find a semantic target that may be off-screen. query • string • required. matchBy • string • optional (text|identifier|label|value|hint, default text). direction • string • optional (up|down|left|right, default down). maxAttempts • integer • optional • max 10. distance • number • optional. connection • object • optional.
{"name": "reveal_search", "arguments": {"query": "greeting_input_field", "matchBy": "identifier", "direction": "down", "maxAttempts": 4}}
Returns: {"ref": "s_14", "snapshotId": 2, "match": {...}, "attempts": [...]} — Failures: missing_query, target_not_found, scroll_blocked
fill_form
Batch text entry: fills multiple fields in one call. Stops on first failure. snapshotId validated on first field only. fields • array of {ref, text} • required. snapshotId • integer • optional. connection • object • optional.
{"name": "fill_form", "arguments": {"fields": [{"ref":"s_1","text":"user"},{"ref":"s_2","text":"pass"}], "snapshotId": 5}}
Returns: {"filled": 2} — Failures: stale_snapshot, ref_not_found
scroll
Scroll to reveal content. "down" reveals content below (finger swipes up). direction • string • required (up|down|left|right). ref • string • optional (falls back to screen center). distance • number • optional • default 300. snapshotId • integer • optional. connection • object • optional.
{"name": "scroll", "arguments": {"direction": "down", "ref": "s_0", "distance": 500}}
Returns: {"via": "semantic_action"} — Failures: ref_not_found, stale_snapshot
swipe
High-velocity fling. Same direction model as scroll. Always Tier 2 pointer events. direction • string • required. ref • string • optional. distance • number • optional • default 300. snapshotId • integer • optional. connection • object • optional.
{"name": "swipe", "arguments": {"direction": "left", "ref": "s_4"}}
Returns: {"via": "pointer_events"} — Failures: ref_not_found, web_gesture_not_supported
drag
Drag from one widget to another. Always Tier 2. fromRef • string • required. toRef • string • required. snapshotId • integer • optional. connection • object • optional.
{"name": "drag", "arguments": {"fromRef": "s_2", "toRef": "s_7"}}
Returns: {"via": "pointer_events"} — Failures: ref_not_found, web_gesture_not_supported
hover
Synthesize a mouse hover. Desktop/web only — no hover concept on mobile. ref • string • required. snapshotId • integer • optional. connection • object • optional.
{"name": "hover", "arguments": {"ref": "s_5"}}
Returns: {"via": "pointer_events"} — Failures: ref_not_found, platform error on mobile
press_key
Synthesize key press (down+up). Accepted: Enter Escape Tab Backspace Delete Space ArrowUp ArrowDown ArrowLeft ArrowRight plus single ASCII (a-z 0-9). key • string • required. ctrl/shift/alt/meta • boolean • optional • default false. connection • object • optional.
{"name": "press_key", "arguments": {"key": "Enter"}}
Returns: {"key": "Enter"} — Failures: unsupported_key, no_focus
wait_for
Wait for a UI predicate; returns fresh semantic snapshot. Predicates: {kind:"text",text} | {kind:"noText",text} | {kind:"time",ms} | {kind:"stable",stableWindowMs}. predicate • object • required. timeoutMs • integer • optional • default 5000 • max 30000. connection • object • optional.
{"name": "wait_for", "arguments": {"predicate": {"kind": "text", "text": "Dashboard"}, "timeoutMs": 8000}}
Returns: fresh semantic snapshot — Failures: timeout, invalid_predicate
navigate
Drive the registered Navigator. Requires MCPToolkitBinding.instance.navigatorKey = key in the app. action • string • required (push|pop|popUntil). route • string • required for push/popUntil. arguments • object • optional (for push). connection • object • optional.
{"name": "navigate", "arguments": {"action": "push", "route": "/profile", "arguments": {"userId": "42"}}}
Returns: {"action": "push", "route": "/profile"} — Failures: navigator_not_configured, route_not_found
handle_dialog
Dismiss the topmost popup/dialog route. Only action: "dismiss" supported. Requires navigatorKey = key on MCPToolkitBinding.instance in the app. action • string • required (must be "dismiss"). connection • object • optional.
{"name": "handle_dialog", "arguments": {"action": "dismiss"}}
Returns: {"dismissed": true} — Failures: navigator_not_configured, no_dialog
hot_reload_flutter
Hot reload the app. Preserves state. force • boolean • optional • default false (reload even without source changes). connection • object • optional.
{"name": "hot_reload_flutter", "arguments": {}}
Returns: "Hot reload completed" + report JSON — Failures: vm_not_connected, compilation_error
hot_restart_flutter
Full restart. App state not preserved. No required params. connection • object • optional.
{"name": "hot_restart_flutter", "arguments": {}}
Returns: {"report": {"type": "Success", "success": true}} — Failures: vm_not_connected
hot_reload_and_capture
Hot reload then capture screenshot + semantics + errors in one call. compress • boolean • default true. includeSemantics • boolean • default true. includeErrors • boolean • default true. errorsCount • integer • default 4. connection • object • optional.
{"name": "hot_reload_and_capture", "arguments": {"includeErrors": true}}
Returns: screenshot (base64) + semantic snapshot + errors — Failures: vm_not_connected, compilation_error
Patterns
Always wait_for before tap_widget after navigation
After navigate(action: "push") the new route's widgets are not in the tree yet. Use wait_for with a text predicate to confirm the destination has rendered, then snapshot and act.
navigate(action: "push", route: "/checkout")
wait_for(predicate: {kind: "text", text: "Order Summary"}, timeoutMs: 5000)
semantic_snapshot() → tap target widgets
Prefer fill_form over multiple enter_text calls
Each enter_text is a separate VM round-trip. fill_form sends all field/text pairs in one call; snapshotId is checked once (on the first field). For any form with 2+ fields, always prefer fill_form.
# Avoid: 2 round-trips
enter_text(ref: "s_1", text: "Alice")
enter_text(ref: "s_2", text: "secret")
# Prefer: 1 round-trip
fill_form(fields: [{ref: "s_1", text: "Alice"}, {ref: "s_2", text: "secret"}])
After hot_reload_*, wait for the new tree before continuing
Hot reload completes asynchronously. Use wait_for(predicate: {kind:"stable", stableWindowMs:300}) to confirm the tree has settled before calling semantic_snapshot. Or use hot_reload_and_capture which returns a post-reload snapshot directly.
hot_reload_flutter()
wait_for(predicate: {kind: "stable", stableWindowMs: 300})
semantic_snapshot() → interact with reloaded widgets
# Or in one call (preferred):
hot_reload_and_capture() → screenshot + semantics + errors already post-reload
