LLM desktop automation in April 2026: every model release, plus the one-line replacement that decides whether their clicks actually land

Two distinct things, and most writeups only cover the first. The model side: Microsoft Fara-7B (April 19) targeted a 7B agentic small language model with the CUAVerifierBench benchmark; Google released the Gemma 4 family under Apache 2.0; Anthropic previewed Claude Mythos to partners; Manus continued shipping incremental updates to its general autonomous agent. The framework side: Terminator 0.24.31 went out on April 2 with PR #473, fixing multi-monitor click validation. The first set is what every model-news roundup covered. The second is what determines whether any of those models can actually click a button on a second monitor on Windows.

Windows UI Automation exposes a method called IsOffscreen on every UIElement. It returns true when the element is not visible. The problem is the implementation: IsOffscreen reports based on the primary monitor's bounds, not the virtual desktop. An element living entirely on a second monitor has bounds outside the primary monitor's rectangle, so IsOffscreen returns true even when the element is fully visible to the user. Terminator 0.24.30 trusted this method as the first gate in validate_clickable(), so any LLM driving a multi-monitor Windows workflow saw clicks rejected with ElementNotVisible errors on perfectly valid targets.

A function called is_visible_on_any_monitor at line 316 of crates/terminator/src/platforms/windows/element.rs. It enumerates every monitor via xcap::Monitor::all(), reads each monitor's x, y, width, and height, then runs a rectangle-intersection test against the element's bounds. If the element's bounds intersect any monitor, the element is visible. The function returns Ok(true) on first match and logs which monitor it intersected with. Both validate_clickable() and is_visible() now use this helper. The old IsOffscreen calls are deleted in both places.

One file. crates/terminator/src/platforms/windows/element.rs. 71 insertions and 31 deletions. The commit hash is e36b9785, dated 2026-04-02, and it closes GitHub issue #473 (which had been opened on March 30 by a user reporting the exact symptom). Released as Terminator 0.24.31 the same day.

Because the LLM never sees the monitor coordinates. Whether the LLM is Claude, Fara-7B, Gemini, Gemma, or Manus, the model returns a selector like role:Button && name:Save. The MCP server resolves that selector to an element, then asks the OS whether it is clickable. If the OS-side answer is false because of a buggy Microsoft API, the click never fires and the LLM gets a generic 'not visible' error back. No prompt-engineering on the model side fixes that. The fix has to be in the validation layer where the OS coordinates actually live.

Yes, at the tail. The vision loop returns coordinates in 0-999 normalized space, which Terminator converts to screen pixels using window offset, DPI, and resize scale. Once that conversion produces an absolute (x, y) on the virtual desktop, the click is dispatched the same way every selector-driven click is dispatched, through the same validate_clickable() gate. So a Gemini Computer Use turn that picked a coordinate on a secondary monitor was hitting the same false negative. The fix is in the shared validation code path, not in either branch.

Two reasons. First, IsOffscreen looks like the right method. The Microsoft documentation does not advertise its primary-monitor bias; the method just says 'the element is not on the screen.' Second, the dominant computer-use development setup is a single laptop monitor, often the primary on a docking station. The bug only surfaces when you click into a secondary monitor that is geometrically outside the primary monitor's bounds. The reporter on issue #473 had a wide horizontal layout where the secondary monitor lived at negative x coordinates relative to primary; that is exactly the configuration where IsOffscreen lies most reliably.

macOS goes through a different code path. The Windows-specific bug was in crates/terminator/src/platforms/windows/element.rs. The macOS adapter under crates/terminator/src/platforms/macos talks to AXUIElement and uses the system's own multi-display geometry, which has not exhibited the same misreporting. If you are driving macOS apps with an LLM, you do not need this April patch; if you are driving Windows apps, you do.

Run `terminator --version` if you have the CLI installed, or check `crates/terminator-cli/Cargo.toml` for the workspace version. Anything 0.24.31 or newer has the fix. As of May 2026, the workspace version is 0.24.32. If you are pulling from npm, `npx -y terminator-mcp-agent@latest` will resolve to the most recent published agent. The relevant comment block ('This replaces the old is_offscreen() check which incorrectly returned true for elements on secondary monitors') is the in-source signature of the patch.

No. The 31 MCP tools (click_element, type_into_element, get_window_tree, and the rest) accept the same arguments and return the same shape. The fix is entirely below the MCP boundary. From the LLM's point of view, the only observable difference is that clicks on secondary monitors now succeed instead of returning ElementNotVisible. That is the kind of fix that does not need a prompt change in any agent that uses Terminator.