Looking at the screen

This is the starting point of almost every task: the agent sees what you see.

What you ask

"Look at my screen and tell me which window is in the foreground."

"Find the Save button on screen."

What the agent does

1. vue_displays lists the screens, with their index. With two screens, the agent knows which one to look at.
2. vue_snapshot captures a screen. It returns a coordinate contract (coord_space) and then the image. By default the image is downscaled to 720 pixels wide: the agent asks for more for fine detail, or limits the capture to a region.
3. vue_find locates a visible label by on-device OCR and returns clickable coordinates. It is mostly used when Touché sees nothing.
4. vue_resolve converts a pixel of the image into a global point, and the reverse, without capturing: the agent does not have to compute offset and scale.

What you get

A description, a position, or a clickable point. Daimon does not interpret the image: your agent looks at it and decides.

What Daimon masks

The windows of the applications you protect, the declared areas and the secret fields are never refused: they are censored, replaced by an opaque purple flat tint #3A2A5C. Your agent knows that an area of this color is masked content, not an empty screen. OCR reads the censored image: it returns no word that touches a censored area. See Privacy.

If the call is refused

Code	What the agent should do
permission_denied	The Enregistrement de l'écran (Screen Recording) permission is missing: Permissions
invalid_arguments	Fix the argument
index_out_of_range	The requested screen does not exist: re-read vue_displays
capture_failed	Retry, then call daimon_doctor
exclusions_unreadable	Your exclusions file is broken: the message names the file and the line

Full detail: Vue.

Next: Reading an interface, or Acting.