Skip to main content

Module computer_use

Module computer_use 

Source
Expand description

X11 backend for the Computer Use actions: XTEST injection and one-shot root screenshots over a private x11rb connection, so the agent can drive an X session with no active capture and no shared state with any streaming capture thread (the X server itself serializes).

Keysyms the active layout cannot produce (after base+AltGr resolution, which stays the preferred path) are typed through a transient remap: under XGrabServer the keymap is re-fetched, the needed keysyms are bound onto all-NoSymbol spare keycodes with ONE XChangeKeyboardMapping, and the key sequence is injected; after a settle window (which lets the focused client re-fetch the bound map — see type_with_transient_binds) one conditional XChangeKeyboardMapping under a second grab puts the spares back — two MappingNotify broadcasts per action. This coexists with a selkies session’s own spare-keycode overlay allocator on the same server: spares here are chosen DESCENDING from the top of the keycode range while selkies allocates ASCENDING, each grab re-fetches the keymap so every binding selkies already made is respected, the restore clears only keycodes still carrying our content, and selkies’ foreign-change invalidation (fired by each MappingNotify) is self-healing because its held keys live on BOUND (non-NoSymbol) keycodes an all-NoSymbol spare can never steal.

Structs§

CuX11Backend
ServerGrabGuard 🔒
RAII server grab release: the grab is dropped (and the request flushed) on every exit path, early error returns and panics included — a leaked server grab freezes every client on the display.
ServerKeymap 🔒
Reverse view of the server’s current keymap.

Constants§

KEYSYM_ISO_LEVEL3_SHIFT 🔒
KEYSYM_MODE_SWITCH 🔒
MAX_SCROLL_CLICKS 🔒
One wheel “click” of scroll per unit of CU scroll_amount, capped so a hostile amount cannot flood the server with press/release pairs.
TRANSIENT_BIND_SETTLE 🔒
How long transient binds outlive the injected key events before being restored: the focused client has to wake up, see the bind’s MappingNotify, and re-fetch the keymap while the bindings are still live, or the presses translate against the restored map and type nothing.