// Control the desktop using AI vision (computer-use). Use for GUI automation, clicking buttons, typing in applications, and interacting with desktop software.
| name | lux |
| description | Control the desktop using AI vision (computer-use). Use for GUI automation, clicking buttons, typing in applications, and interacting with desktop software. |
The lux CLI enables AI-powered desktop control. It takes screenshots, sends them to a vision AI, and executes the returned actions (clicks, typing, etc.) on your desktop.
This is for GUI automation - use it when you need to interact with graphical applications that can't be controlled via command line.
# Start a session with a goal
lux start "Open Firefox and navigate to github.com"
# Run until the task is complete (or limit reached)
lux run --max-steps 10
# Or run step-by-step for more control
lux step # Take screenshot, get actions, execute
lux step # Repeat until done
lux end # End session when finished
Start a new computer-use session with a goal description.
lux start "Click the Settings icon and enable dark mode"
lux start "Open the terminal and run htop"
lux start "Fill out the login form with username 'test'"
Options:
--model <model> - AI model: auto (default), lux-actor-1 (fast), or lux-thinker-1 (complex tasks)--max-steps <n> - Maximum steps (default: 20 for actor, 100 for thinker)Model selection in auto mode (the default):
--model lux-actor-1 or --model lux-thinker-1Execute one step: takes a screenshot, sends it to the AI, and executes the returned actions.
lux step
This is useful when you want to observe each step or intervene between actions.
Run steps continuously until the task is complete or the limit is reached.
lux run # Run up to 20 steps
lux run --max-steps 10 # Run up to 10 steps
End the current session early.
lux end
Show your usage and limits.
lux status
The AI returns actions that are executed on your desktop:
| Action | Description | Example Argument |
|---|---|---|
click | Single left click | "512, 384" (normalized 0-1000) |
left_double | Double click | "512, 384" |
right_single | Right click | "512, 384" |
drag | Click and drag | "100, 100, 500, 500" (start to end) |
type | Type text | "Hello World" |
hotkey | Press key combination | "ctrl+c", "alt+tab" |
scroll | Scroll in direction | "up", "down", "left", "right" |
wait | Pause execution | "1000" (milliseconds) |
finish | Task completed | - |
call_user | Need human help | "Please enter password" |
Be specific about what you want to accomplish:
# Good - specific and actionable
lux start "Click the blue 'Submit' button in the form"
# Bad - vague
lux start "Submit the form"
When things don't work as expected, use lux step to see what the AI is doing:
lux start "Navigate to settings"
lux step # See what it clicks
lux step # Continue manually
Always end sessions to free up resources:
lux end
Monitor your usage:
lux status
lux start "Open Firefox and go to https://github.com"
lux run --max-steps 5
lux start "Fill in the username field with 'testuser' and click Login"
lux run
lux start "Open the file manager and navigate to Documents folder"
lux run --max-steps 10
For complex tasks, use lux-thinker-1 model:
lux start "Open VS Code, create a new file called test.py, and write a hello world program" --model lux-thinker-1
lux run --max-steps 20
Start a session first:
lux start "your task here"
Sessions expire after 30 minutes of inactivity. Start a new one:
lux start "your task here"
You've hit the per-session limit. End and start a new session:
lux end
lux start "continue from where I left off"
The AI might misidentify UI elements. Try being more specific:
lux start "Click the green button labeled 'Save' in the bottom right corner"