| name | mai-ui-agents |
| title | MAI-UI: Real-World Centric Foundation GUI Agents |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2512.22047 |
| keywords | ["agents","gui-automation","reinforcement-learning","real-world","multi-modal"] |
| description | Scale GUI agents to real-world complexity via extended action space (user interaction, tool calls) and device-cloud collaboration. Online RL supports 500+ parallel environments with asynchronous handling; local agent monitors trajectory alignment and handoffs to cloud when drift detected—achieving 41.7% MobileWorld success with privacy-preserving delegation. |
Overview
MAI-UI addresses critical limitations in existing GUI agents through pragmatic design choices: extended actions enable richer interactions, device-cloud collaboration preserves privacy, and online RL at scale improves agentic reasoning.
Core Technique
Extended Action Space:
Beyond pure UI operations, agents can request clarification and invoke tools.
class ExtendedActionSpace:
user_ask = "ask_user"
mcp_call = "mcp_call"
Device-Cloud Collaboration:
Local agent monitors alignment; cloud only handles complex cases.
class HybridAgent:
def should_handoff_to_cloud(self, trajectory, instruction):
deviation = self.alignment_monitor.evaluate(trajectory)
if deviation > threshold:
return True
return False
Key Performance
- 73.5% grounding (ScreenSpot-Pro)
- 76.7% mobile navigation (AndroidWorld)
- 41.7% real-world tasks (MobileWorld)
- 500+ parallel environments
References
- Extended action space design
- Device-cloud collaboration architecture
- Online RL with asynchronous parallelism
