Skip to content

Acting ​

Mains (Hands) lets the agent click, type text, press a button, drag and drive windows. This is what you bound, with the ceiling.

What you ask ​

"Open the File menu."

"Fill the Name field with "Dupont", then confirm."

Before acting: the ceiling ​

The ceiling is set to L0 by default: the agent can only observe. For it to act, raise the panel's slider (L1 to L3): see Ceiling and exclusions.

A well-driven agent first calls main_ceiling. The tool returns the active ceiling, and gated_above lists the tools refused at this level: the agent announces up front what it will not be able to do, instead of being refused along the way. It cannot raise the ceiling: that is for you to do.

LevelWhat it allowsTools
L1Reversible actions: focus, windows, navigationmain_navigate, main_hover, main_activate, main_window_*
L2Clicks, typing, dragging, keysmain_click, main_type, main_drag, main_key
L3Buttons that commitmain_press
L4Everything, without per-action confirmationmain_mouse_*, main_key_down, main_key_up

What the agent does ​

  1. It looks: Looking at the screen or Reading an interface, to obtain a reliable position.
  2. It acts, declaring its intent: a sentence saying what it wants to do, recorded in the audit log. Tools that commit also carry reversible, which the agent must fill in honestly.
  3. It verifies: a new look confirms the result.

To click, the agent preferably passes role and label (read with touche_probe): Daimon can then check the target's reversibility. To type, it first clicks in the field, because main_type does not choose the target. The typed text is never written to the audit log.

To aim, the agent gives either global points (those returned by vue_find), or, with space="image" and a display, pixels of a capture that Daimon converts itself.

What you get ​

Each action tool answers in one of these three forms:

ResponseMeaning
{"status": "done", …}The action was executed
{"status": "refused", "reason": "…"}The action is refused: ceiling, guard, human confirmation not obtained
{"status": "error", "reason": "…"}The action could not run

Confirmations ​

Up to L3, an action that Daimon judges irreversible, or that the agent declares as such, stops on a confirmation dialog. You answer; without an answer, it declines by itself after 30 seconds, and the agent receives refused.

At L4, there is no more per-action confirmation. This level requires your explicit consent, from the panel or the terminal; no tool lets the agent engage it.

If the call is refused or fails ​

CaseWhat the agent should do
refused above the ceilingSay that the action exceeds the ceiling, and ask the person to raise it if they wish
refused, confirmation not obtainedDo not insist: the person declined or did not answer
done with no effect at allWithout the Accessibilité (Accessibility) permission, main_type, main_key, main_hover and main_navigate answer done without doing anything: call daimon_doctor
30-second waitWithout Accessibility, main_click, main_press and main_drag wait for a confirmation: call daimon_doctor
invalid_argumentsFor example space="image" without display: fix the argument
motor_unavailableMains is not available on this host

Full detail: Mains.

Next: Showing what is happening.