Guide AI agents in creating native desktop UI tests with PlatynUI (new_core) using the PlatynUI.BareMetal library — cross-platform desktop automation over Windows UIA and Linux AT-SPI2 with an XPath query model. Use when asked to automate desktop applications (windows, buttons, menus, text fields), locate native UI elements by XPath, or write Robot Framework tests with PlatynUI.
Installer avec Codex ou Claude Copiez ce prompt, collez-le dans Codex, Claude ou un autre assistant, puis laissez-le vérifier la page du skill et l'installer pour vous.
Une commande directe contourne le prompt de vérification. Examinez la source avant de l'exécuter.
Guide AI agents in creating native desktop UI tests with PlatynUI (new_core) using the PlatynUI.BareMetal library — cross-platform desktop automation over Windows UIA and Linux AT-SPI2 with an XPath query model. Use when asked to automate desktop applications (windows, buttons, menus, text fields), locate native UI elements by XPath, or write Robot Framework tests with PlatynUI.
PlatynUI Library Skill (new_core / BareMetal)
Quick Reference
PlatynUI is cross-platform native desktop UI automation for Robot Framework. It normalizes Windows UIA, Linux AT-SPI2 (and planned macOS AX) into one desktop UI model that you query with XPath-2.0-style selectors.
⚠️ Preview / pre-release. This skill targets the new_core rewrite. The only usable keyword library today is PlatynUI.BareMetal; the high-level PlatynUI library is a not-yet-implemented placeholder. Keywords, CLI output, and behavior may change.
Installation
Get the version with the right pin — this is the #1 mistake:
# ✅ new_core (PlatynUI.BareMetal, documented keywords) — pin OR use --pre:
pip install robotframework-PlatynUI==0.12.0.dev330
pip install --pre robotframework-PlatynUI # latest pre-release# ❌ WRONG — installs old 0.9.2 (a different library, NO BareMetal):
pip install robotframework-PlatynUI # no --pre, no pin → 0.9.2
A plain pip install robotframework-PlatynUI resolves the old stable 0.9.2, whose surface is a different, near-undocumented PlatynUI library with no . You only get by opting into the pre-release () or pinning an exact dev version.
BareMetal
new_core
--pre
The wheel is prebuilt — it bundles the compiled platynui-native runtime (_native.abi3.so). No Rust toolchain and no git checkout required. Python 3.12+ only.
Optional CLI / inspector tools (for locator development — see below):
Linux additionally needs a running AT-SPI2 accessibility bus and (recommended) an X11 session — see references/platform-setup.md.
Library Import
*** Settings ***
Library PlatynUI.BareMetal
Import arguments (all keyword-only, all optional):
Argument
Default
Purpose
auto_activate
True
Bring the target element's window to the foreground before pointer/keyboard actions. Override per-call with activate=.
use_mock
False
Use the Rust mock provider (deterministic, no desktop). Note: the published wheel is built without mock support and raises ProviderError; needs a --features mock-provider source build.
Library scope is SUITE. The native runtime is created lazily on first keyword use (so libdoc and import work without a desktop).
Essential Concepts
The normalized desktop UI model
PlatynUI exposes every application's accessibility tree as one queryable model rooted at the Desktop. Each node has a normalized PascalCase role (Window, Button, Edit, ListItem…), a Name, an optional developer Id, and typed attributes (Bounds, IsEnabled, Value…). Native roles are normalized cross-platform (UIA Button / AT-SPI PUSH_BUTTON → Button); the original stays under native:Role.
Critical rule: an unprefixed query matches control:only. You MUST prefix app: and item: explicitly — //ListItem finds nothing; //item:ListItem works.
XPath query language
Elements are referenced by an XPath string passed directly as a keyword argument (auto-converted to an element descriptor), or by a UiNode captured from Query ... only_first=${True}.
Supports the full XPath-2.0 axis/function set: // (descendant), ../parent::, following-sibling::, predicates ([@Name='OK'], [1]), and functions (contains(), matches(), count(), starts-with()). See references/locators-and-queries.md.
Lazy resolution with built-in wait
There is no explicit "wait for element" keyword. When a descriptor is first used, PlatynUI polls the query (default ~30 s, retrying ~0.1 s) until the element resolves or it raises ElementNotFoundError. Write the correct selector and let the runtime wait — don't add Sleep.
Coordinate semantics (pointer keywords)
You pass
Behavior
descriptor, no x/y
acts at the element's ActivationPoint (else bounds center)
descriptor + x/y
x/y are offsets from the element's top-left bounds
no descriptor, x/y
x/y are absolute screen coordinates
only one of x/y
ValueError — pass both together
Window keywords need a window node
Activate/Minimize/Maximize/Restore/Close/Move/Resize Window operate on a node that supports the window surface — target a Window node, not an arbitrary control, or they raise PatternError.
Core Keywords Quick Reference
Query & scope
${node}= Query //control:Button[@Name="OK"] only_first=${True}
${all}= Query //item:ListItem # list of nodes
${n}= Query count(//control:Button) # computed value
Set Root app:Application[@Name="Calculator"]//control:Window # scope later queries
Set Root ${None} # reset to desktop
Pointer / mouse
Pointer Click //control:Button[@Name="OK"]
Pointer Click //control:Button[@Name="OK"] button=RIGHT activate=${False}
Pointer Multi Click //item:ListItem[@Name="file.txt"] clicks=${2}
Pointer Move To ${None} x=${100} y=${200} # absolute screen
Pointer Press //control:Slider
Pointer Release ${None}
${pos}= Get Pointer Position
Keyboard
Keyboard Type //control:Edit[@Name="Search"] Hello <Ctrl+A> world${\n}
Keyboard Type ${None} <Ctrl+A><Delete> # type into whatever has focus
Keyboard Press ${None} <Shift>
Keyboard Release ${None} <Shift>
Key syntax: <Ctrl+A>, <Return>, <ESC>, <Shift+Tab>, plain text, ${\n}. Pass ${None} as the descriptor to skip focusing. Discover valid key names with platynui-cli keyboard list.
Focus, attributes, windows
Focus //control:Button[@Name="New File"]
${bounds}= Get Attribute //control:Window[@Name="Calc"] Bounds
Get Attribute //control:Button[@Name="OK"] Name == OK # built-in assertion
Bring To Front //control:Window[@Name="Calc"]
Activate Window //control:Window[@Name="Calc"]
Maximize Window //control:Window[@Name="Calc"]
Move And Resize Window //control:Window[@Name="Calc"] ${0} ${0} ${800} ${600}
Diagnostics
Take Screenshot # full screen → ${OUTPUT DIR}
Take Screenshot //control:Window[@Name="Calc"] # just the element
Take Screenshot filename=EMBED # inline base64 in the log
Highlight //control:Button[@Name="OK"] duration=${2} # draw an overlay (debug)
Locator Strategy
Default namespace is control — //Button ≡ //control:Button. Add control: only for clarity.
Always prefix app: and item: — applications and container items are NOT in the default namespace. Forgetting this is the most common cause of "element not found."
Prefer @Id over @Name — [@Id="num5Button"] is language-independent and stable (UIA AutomationId / AT-SPI accessible_id / AX AXIdentifier). Use @Name only when no Id exists.
Anchor on the application, then descend — app:Application[@Name='…']//control:Window//control:Button[@Id='…']. Or Set Root once on the app/window and use short relative queries (.//Button[@Id='…']).
Handle localization with or — [@Name='Calculator' or @Name='Rechner'].
Fuzzy match with functions — contains(@Name,'Save'), matches(@Name,'.*Save.*'), [contains(@*,'Close')] (search across all attributes).
Step axes when structure beats identity — …/Label[@Name='5']/parent::Button.
Let resolution wait — don't Sleep; the descriptor retries for ~30 s.
Common Patterns
Launch an app, scope to it, act
*** Settings ***
Library PlatynUI.BareMetal
Library Process
Test Setup Start Calculator
*** Keywords ***
Start Calculator
Start Process calc.exe
# Prefer letting the next Query wait over a fixed Sleep.
Enter Digit
[Arguments] ${d}
Set Root app:Application[@Name="ApplicationFrameHost"]/control:Window[@Name="Calculator" or @Name="Rechner"]
Pointer Click .//control:Button[@Id="num${d}Button"]
Type into a field and assert
Keyboard Type //control:Edit[@Name="Search"] robot framework${\n}
Get Attribute //control:Edit[@Name="Search"] Value == robot framework
Verify targeting before automating
${node}= Query //control:Button[@Name="Submit"] only_first=${True}
Highlight ${node} duration=${2} # visually confirm you matched the right node
Pointer Click ${node}
CLI Tooling (locator development loop)
Develop and debug selectors with platynui-cli before baking them into tests:
platynui-cli list-providers # confirm a provider is active
platynui-cli query "//control:Button[@Name='OK']"# test a selector live
platynui-cli snapshot "//control:Window" --pretty # dump a UI subtree
platynui-cli highlight "//control:Button[@Name='OK']"# visually confirm on screen
platynui-inspector is a GUI that shows the live UI tree, lets you test XPath interactively, and highlights matches. See references/cli-and-inspector.md.
Troubleshooting
pip install robotframework-PlatynUI gives a library with no BareMetal → you got 0.9.2; reinstall with --pre or pin ==0.12.0.dev330.
Linux: queries return nothing / empty tree → AT-SPI2 not running, or accessibility not enabled. See references/platform-setup.md (platynui-cli list-providers should show the atspi provider active).
Wayland: clicks land wrong / screenshots fail → use an X11 (or XWayland) session; Wayland is degraded for input injection and screen coordinates.
ElementNotFoundError → check the namespace prefix (item:/app:), prefer @Id, and verify the selector with platynui-cli query.
ProviderError: requires building with feature 'mock-provider' → use_mock=${True} needs a source build; the published wheel has no mock provider.
When to Load Additional References
Need
Reference File
Full keyword signatures, args, return types
references/keywords-reference.md
XPath syntax, namespaces, roles, axes, locator best practices
references/locators-and-queries.md
platynui-cli subcommands + the inspector
references/cli-and-inspector.md
Install matrix, Linux AT-SPI/X11/Wayland, mock, Docker
references/platform-setup.md
BareMetal-vs-high-level status, version footgun, migration
