Reading an interface

When an application exposes its structure, the agent can read its elements instead of interpreting an image: it is more precise and more economical.

What you ask

"List the buttons of the foreground window."

"What is under the cursor?"

What the agent does

• touche_tree returns the accessibility tree of the foreground window: each element with its role, its label and its position. The tree is bounded in depth (4 by default) and stripped of sensitive content. The agent can filter by roles to keep only, for example, the buttons.
• touche_probe returns the element located under a point on the screen.

Both tools are read-only and are not subject to the Mains ceiling.

What you get

A list of elements, with their positions: these are then used to act (see Acting).

When the tree is empty

Some applications (legacy Win32 applications, custom-drawn interfaces, Electron) expose nothing. The agent then falls back on vue_find, which locates a text on screen by OCR.

If the call is refused

Code	What the agent should do
accessibility_permission_denied	The Accessibilité (Accessibility) permission is missing: Permissions
excluded	The target belongs to an application you have protected: the agent does not insist
snapshot_failed	The read failed: retry, then call daimon_doctor

Full detail: Touché.