# pi-codex-computer-use
Pi package that exposes [Open Computer Use](https://github.com/iFurySt/open-codex-computer-use) as Pi tools.
It gives Pi Codex-style GUI automation through the open-source `open-computer-use` MCP server:
- `computer_use_list_apps`
- `computer_use_get_app_state`
- `computer_use_click`
- `computer_use_perform_secondary_action`
- `computer_use_scroll`
- `computer_use_drag`
- `computer_use_type_text`
- `computer_use_press_key`
- `computer_use_set_value`
Plus management/fallback tools:
- `computer_use_open`
- `computer_use_close`
- `computer_use_status`
- `computer_use_doctor`
- `computer_use_call`
- `computer_use_sequence`
## Install in Pi
From this checkout:
```bash
pi install /Users/joel/Code/joelhooks/pi-codex-computer-use
```
Or once published:
```bash
pi install git:github.com/joelhooks/pi-codex-computer-use
# or
pi install npm:pi-codex-computer-use
```
The package depends on `open-computer-use`, so Pi package install should install a package-local native runtime. If you prefer a global/runtime override, set:
```bash
export PI_CODEX_COMPUTER_USE_COMMAND=/path/to/open-computer-use
```
## First run
On macOS, run the permission doctor and grant Accessibility + Screen Recording if prompted:
```bash
/ocu-doctor
```
Or through the model tool:
```ts
computer_use_doctor({})
```
Then inspect apps:
```ts
computer_use_list_apps({})
```
Get current state before acting:
```ts
computer_use_get_app_state({ app: "TextEdit" })
```
Then prefer element-targeted actions:
```ts
computer_use_click({ app: "TextEdit", element_index: "3" })
computer_use_type_text({ app: "TextEdit", text: "Hello from Pi" })
```
Use a sequence when a short flow needs to reuse element indexes in one MCP session:
```ts
computer_use_sequence({
calls: [
{ tool: "get_app_state", args: { app: "TextEdit" } },
{ tool: "click", args: { app: "TextEdit", element_index: "3" } },
{ tool: "type_text", args: { app: "TextEdit", text: "Hello from Pi" } }
]
})
```
## Commands
- `/ocu-open [command]` — connect/reconnect the MCP process
- `/ocu-close` — disconnect it
- `/ocu-status` — show resolved command and connection state
- `/ocu-doctor [command]` — run `open-computer-use doctor`
## Safety rules
This touches the real desktop. Don’t be a dingus:
- Call `computer_use_get_app_state` before action tools each turn.
- Prefer element indexes over coordinates.
- Ask before sending, deleting, purchasing, approving prompts, uploading files, or touching sensitive apps.
- Use dedicated APIs/tools before GUI automation when they exist.
## Validation
```bash
npm install
npm run check
npm run doctor
```