edlsh/omp-computer-use

edlsh のプロフィール

記録された最新のソース活動: 2026年7月13日 09:17

SkillsMP カタログ更新: 2026年10月17日 00:09

収集済み skills: 1

GitHub スター: 0
GitHub フォーク: 1

リポジトリの README

README の元ファイル：README.md

README の元ファイルを読む

README 全文を表示

# omp-computer-use Native [Oh My Pi](https://github.com/can1357/oh-my-pi) extension for state-scoped desktop control on macOS and Windows. `omp-computer-use` lets OMP agents inspect application windows, find controls, click, type, scroll, wait for UI changes, and verify observable outcomes. It keeps complete immutable observations behind compact `stateId`-scoped views so concurrent calls cannot silently act on stale UI. The macOS helper requires macOS 14 or newer. ## When to use it Prefer a reliable API, MCP server, file operation, terminal command, or OMP's dedicated browser tool when one fits. Computer use is for software whose usable interface is the app on screen. The extension adds: - first-class OMP tools with read/write/exec approval tiers; - macOS Accessibility and ScreenCaptureKit integration; - Windows UI Automation and native input integration; - immutable UI states with stale-write rejection; - cached search, expansion, inspection, and paginated text; - checked action batches, semantic postconditions, and successor diffs; - optional OCR-backed fused observations; - managed CDP browser roots sharing the same state model. ## Install ```bash omp install git:github.com/edlsh/omp-computer-use ``` Start OMP interactively after installation: ```bash omp ``` On macOS, the first computer-use tool call registers the native helper and asks for: - **Accessibility** - **Screen Recording**, called **Screen & System Audio Recording** on newer macOS releases Grant both permissions to: ```text /Applications/pi-computer-use.app ``` The helper retains the upstream app identity so upgrades do not unnecessarily replace the macOS permission target. Enable each toggle, return to OMP, and choose **Recheck (restarts helper)**. On Windows, run OMP in an unlocked interactive desktop session. The package does not include a Windows executable; its first live tool call builds the included Rust helper, so Rust and `cargo` must be on `PATH`. The helper is then installed under: ```text %USERPROFILE%\.omp\agent\helpers\omp-computer-use\windows-bridge.exe ``` Use `/computer-use` inside OMP to show the effective configuration and its sources. ## Core workflow ```text find_roots → observe_ui → search_ui / expand_ui / inspect_ui → act_ui ``` An `@eN` element ref belongs only to the `stateId` that returned it. Continue from the successor state after every mutation; re-observe after uncertain outcomes or external UI changes. Main tools: - `find_roots` - `observe_ui` - `search_ui` - `expand_ui` - `inspect_ui` - `act_ui` - `read_text` - `wait_for` - `launch_browser` - `navigate_browser` - `evaluate_browser` The included `omp-computer-use` skill teaches OMP the state/ref workflow automatically. ## Configuration User configuration: ```text ~/.omp/agent/extensions/omp-computer-use.json ``` Project configuration: ```text <project>/.omp/computer-use.json ``` Example: ```json { "browser_use": true, "headless": false, "cursor_overlay": true } ``` Project settings override user settings. `PI_COMPUTER_USE_BROWSER_USE`, `PI_COMPUTER_USE_HEADLESS`, and `PI_COMPUTER_USE_CURSOR_OVERLAY` override both. See [Configuration](./docs/configuration.md) for details. ## Development ```bash bun install bun run test ``` Additional platform checks: ```bash bun run test:platform bun run test:windows-scripts ``` See: - [Usage](./docs/usage.md) - [Architecture](./docs/architecture.md) - [Configuration](./docs/configuration.md) - [Development](./docs/development.md) - [Troubleshooting](./docs/troubleshooting.md) - [Contributing](./CONTRIBUTING.md) ## Upstream This project is an OMP-native port of [`injaneity/pi-computer-use`](https://github.com/injaneity/pi-computer-use). It preserves the upstream native helpers, state model, and MIT license while replacing the host integration, package metadata, paths, approval policy, runtime guidance, and Bun lifecycle for Oh My Pi. ## License MIT

リポジトリの README を読み込んでいます…

このリポジトリの skills

ソフトウェア開発者

1 件の職業カテゴリ · 100% 分類済み

skill

職業分類

説明

更新

omp-computer-use

ソフトウェア開発者

Use OMP's state-scoped desktop and managed-browser tools to observe and control macOS or Windows apps.

原文の言語: 英語

2026-07-13

収集済み skill 1 件中 1 件を表示しています。

AI アシスタントでインストール

このプロンプトをコピーして、使用中の AI アシスタントに送ってください。

Follow https://skillsmp.com/skill-install/prompt.md to install Agent Skills from https://github.com/edlsh/omp-computer-use.