scrcpy-claw

name

scrcpy-claw

description

Lobster Android Assistant (龙虾安卓助手) - A comprehensive Android device control skill. Provides touch control, keyboard input, system operations, screen analysis, AI-assisted automation, and script recording/playback through ADB and scrcpy integration. Use this skill when the user wants to control Android devices, automate testing, perform remote control, or build automation workflows. 当用户想要控制安卓设备、自动化测试、远程操作或构建自动化流程时，使用此技能。

龙虾安卓助手 / Lobster Android Assistant

一个强大的 Android 设备控制技能，通过 ADB 和 scrcpy 集成实现全面的设备控制能力。

A powerful Android device control skill with comprehensive capabilities through ADB and scrcpy integration.

功能特性 / Features

🎯 触摸控制 / Touch Control

• 点击、双击、长按 / Tap, double-tap, long press
• 滑动、手势操作 / Swipe, gestures
• 多点触控、缩放手势 / Multi-touch, pinch-to-zoom

⌨️ 键盘输入 / Keyboard Input

• 文本输入（支持中文） / Text input (Chinese supported)
• 按键事件、组合键 / Key events, key combinations
• 特殊按键（Home、Back、音量等）/ Special keys (Home, Back, Volume, etc.)

📱 系统控制 / System Control

• 电源管理 / Power management
• 应用启动与切换 / App launch and switching
• 截图、录屏 / Screenshot, screen recording
• 剪贴板操作 / Clipboard operations

🤖 AI 辅助自动化 / AI-Assisted Automation

• 屏幕分析与 UI 元素识别 / Screen analysis and UI element detection
• 智能决策与操作建议 / Intelligent decision-making
• 自然语言指令执行 / Natural language command execution

📹 Scrcpy 集成 / Scrcpy Integration

• 实时屏幕镜像 / Real-time screen mirroring
• 低延迟控制 / Low-latency control
• 完整控制消息支持 / Full control message support

🔄 脚本录制回放 / Script Recording & Playback

• 操作录制 / Action recording
• 脚本保存与加载 / Script save and load
• 变速回放 / Variable speed playback

快速开始 / Quick Start

基础使用 / Basic Usage

from scripts.adb_controller import ADBController

# 初始化控制器 / Initialize controller
controller = ADBController()

# 触摸操作 / Touch operations
controller.tap(500, 800)              # 点击 / Tap
controller.swipe(100, 500, 500, 500)  # 滑动 / Swipe
controller.long_press(500, 800, 1000) # 长按 / Long press

# 键盘操作 / Keyboard operations
controller.press_home()               # 返回主页 / Go home
controller.press_back()               # 返回键 / Back
controller.input_text("Hello")        # 输入文本 / Input text

# 系统操作 / System operations
controller.screenshot("screen.png")   # 截图 / Screenshot
controller.start_app("com.android.settings")  # 启动应用 / Launch app

AI 自动化 / AI Automation

from scripts.adb_controller import ADBController
from scripts.ai_controller import ScreenAnalyzer, AIDecisionEngine

controller = ADBController()
analyzer = ScreenAnalyzer()
engine = AIDecisionEngine(controller, analyzer)

# 执行自动化任务 / Execute automation
engine.run_automation("打开设置并启用深色模式")
# Open settings and enable dark mode

脚本录制 / Script Recording

from scripts.adb_controller import ADBController, ActionRecorder

controller = ADBController()
recorder = ActionRecorder(controller)

# 开始录制 / Start recording
recorder.start_recording()
recorder.record_action('tap', {'x': 500, 'y': 800})
recorder.stop_recording()

# 保存并回放 / Save and playback
recorder.save_to_file("script.json")
actions = recorder.load_from_file("script.json")
recorder.playback(actions, speed=2.0)

环境要求 / Requirements

• Python 3.7+
• Android SDK (adb 命令可用 / adb command available)
• 已开启 USB 调试的 Android 设备 / Android device with USB debugging enabled

安装说明 / Installation

1. 确保已安装 ADB 并添加到系统 PATH Ensure ADB is installed and added to system PATH

adb version

2. 连接 Android 设备并授权 USB 调试 Connect Android device and authorize USB debugging

adb devices

3. 加载技能即可使用 Load the skill and start using

API 参考 / API Reference

ADBController

主要设备控制器 / Primary device controller.

方法 Method	描述 Description
tap(x, y, duration_ms)	点击坐标 / Tap at coordinates
swipe(x1, y1, x2, y2, duration_ms)	滑动手势 / Swipe gesture
long_press(x, y, duration_ms)	长按操作 / Long press
input_text(text)	输入文本 / Input text
keyevent(keycode)	发送按键事件 / Send key event
press_home()	返回主页 / Press HOME
press_back()	返回键 / Press BACK
screenshot(path)	截图 / Take screenshot
start_app(package)	启动应用 / Launch app

ScrcpyController

高级控制器（需 scrcpy 服务）/ Advanced controller (requires scrcpy server).

方法 Method	描述 Description
start_server()	启动 scrcpy 服务 / Start scrcpy server
connect()	连接到设备 / Connect to device
scroll(x, y, h, v)	滚动操作 / Scroll operation
rotate_device()	旋转屏幕 / Rotate screen
multi_touch(points)	多点触控 / Multi-touch

AIDecisionEngine

AI 决策引擎 / AI decision engine.

方法 Method	描述 Description
analyze_and_suggest(goal)	分析并建议操作 / Analyze and suggest
execute_suggestion(suggestion)	执行建议 / Execute suggestion
run_automation(goal)	执行自动化任务 / Run automation

使用场景 / Use Cases

自动化测试 / Automated Testing

# 运行测试序列 / Run test sequence
controller.start_app("com.example.app")
controller.tap(500, 800)
controller.input_text("test input")
controller.press_back()

远程控制 / Remote Control

# 镜像用户指令 / Mirror user commands
def handle_command(cmd):
    if cmd.type == 'tap':
        controller.tap(cmd.x, cmd.y)
    elif cmd.type == 'type':
        controller.input_text(cmd.text)

AI 辅助操作 / AI-Assisted Operations

# 让 AI 自动完成任务 / Let AI complete tasks
engine.run_automation("打开微信发送消息给张三")
# Open WeChat and send message to Zhang San

故障排除 / Troubleshooting

设备未找到 / Device Not Found

adb devices
adb kill-server && adb start-server

权限被拒绝 / Permission Denied

在设备上启用 USB 调试：设置 > 开发者选项 > USB 调试 Enable USB debugging on device: Settings > Developer Options > USB Debugging

中文输入问题 / Chinese Input Issues

建议安装 ADBKeyboard 以支持中文输入： Install ADBKeyboard for Chinese input support:

adb install ADBKeyboard.apk
adb shell ime set com.android.adbkeyboard/.AdbIME

技术架构 / Technical Architecture

┌─────────────────────────────────────────────────────────────┐
│                     龙虾安卓助手                              │
│                    Lobster Android Assistant                 │
├─────────────────────────────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐  │
│  │ ADB 控制器  │  │ Scrcpy 控制器│  │   AI 决策引擎       │  │
│  │ADB Controller│ │Scrcpy Ctrl │  │AI Decision Engine  │  │
│  └──────┬──────┘  └──────┬──────┘  └──────────┬──────────┘  │
│         │                │                     │             │
│         └────────────────┼─────────────────────┘             │
│                          │                                   │
│                   ┌──────┴──────┐                            │
│                   │  Android 设备│                            │
│                   │Android Device│                           │
│                   └─────────────┘                            │
└─────────────────────────────────────────────────────────────┘

版本信息 / Version Info

• 版本 Version: 1.0.0
• 作者 Author: Lobster Team
• 许可证 License: MIT

更新日志 / Changelog

v1.0.0

• 初始版本发布 / Initial release
• ADB 完整控制支持 / Full ADB control support
• Scrcpy 集成 / Scrcpy integration
• AI 自动化功能 / AI automation features
• 脚本录制回放 / Script recording & playback