Skip to main content

handle_action_inner

Function handle_action_inner 

Source
fn handle_action_inner(
    req: CuActionRequest,
    b: &dyn CuBackend,
) -> Result<String, String>
Expand description

Turn one parsed Computer Use request into a real action on the captured desktop — the bridge that lets an external AI agent see and drive the session.

Every action ultimately becomes a backend input primitive or a framebuffer read, so this is where the API’s vocabulary meets the running desktop. Coordinates are clamped to the current framebuffer so a mistaken agent click cannot address off-screen pixels; keyboard and pointer actions translate into input events; and actions that must look at the screen (screenshot, zoom) request a fresh capture first, so the agent reasons about the current frame rather than a stale one. Returns the JSON response, or an error string explaining why the action could not run.