MCP dev tools for desktop accessibility: the inspect/highlight/state loop, by tool name

It means the set of tools an MCP server exposes to a coding agent (Claude, Cursor, Windsurf, VS Code) so the agent can inspect, identify, and operate on the OS-level accessibility tree the way a frontend developer uses Chrome DevTools to inspect, identify, and operate on the DOM. Concretely, that is five primitives: read the tree (get_window_tree), draw labeled rectangles over every element on the actual screen (get_window_tree with show_overlay:"ui_tree"), highlight one specific match (highlight_element), check element state (validate_element, wait_for_element), and clear the overlay (stop_highlighting). The Terminator MCP server has all five as named tools in one dispatch table at crates/terminator-mcp-agent/src/server.rs near line 9953.

Two reasons. First, when get_window_tree returns 800 elements with similar role and name attributes, an agent that has only the JSON cannot tell which one a human user means by "the Save button." Drawing rectangles labeled with [index] or [index:role:name] over the actual screen lets the agent ask the user to confirm by index, or pass an annotated screenshot back into a vision model to ground the reference. Second, when a workflow misclicks, the developer reading the run log wants to see what the agent saw at that step. The overlay is the visual artifact that closes the gap between the JSON tree and the pixels on the user's monitor. Implementation is at crates/terminator/src/platforms/windows/inspect_overlay.rs, show_inspect_overlay() at line 103 builds a layered Win32 window with WS_EX_LAYERED | WS_EX_TRANSPARENT | WS_EX_TOPMOST and paints rectangles plus labels with GDI Rectangle and DrawTextW.

They live in OverlayDisplayMode at crates/terminator/src/platforms/mod.rs lines 37-53. Rectangles draws the boxes only, no labels, useful when you just want to see the partition of the window. Index labels each rectangle with [N], the same N you can pass to nth: in a selector. Role labels each with the accessibility role (Button, Edit, Window, etc.). IndexRole combines the two as [N:Role] and is the default for inspector workflows. Name uses the accessible name, which is what a screen reader would read aloud; this is what to use when triaging localization issues. IndexName is [N:Name]. Full is [N:Role:Name] and is the densest label, useful when you are screenshotting the overlay to feed to a vision model. The mode is chosen by the overlay_display_mode argument on get_window_tree.

Inspect.exe is a standalone GUI you launch with a mouse. It is fine for one-off debugging. The MCP overlay is the same idea but driven by tool calls from an LLM in your editor. You write "show me the accessibility tree of Notepad with index labels," the agent calls get_window_tree with process:notepad and show_overlay:"ui_tree" and overlay_display_mode:"index_role", and the rectangles appear on your monitor in under a second. The same agent can then pick an index, call click_element with selector:nth:14 (or with the resolved role+name), and verify the result with validate_element. Inspect.exe cannot do that loop without a human at the keyboard. The selectors that the overlay surfaces are also the exact strings the rest of the MCP tool surface accepts: there is one grammar across read, inspect, click, type, and validate.

The overlay shows every element. highlight_element shows one. Implementation at server.rs line 5600. After the agent has resolved a selector to a specific match, it calls highlight_element with that selector and an optional color (passed as a u32 ARGB at server.rs line 5640) and an optional text label that prints next to the rectangle. The default duration is 1000ms; a tokio::spawn at line 5717 schedules cleanup after that window so highlights do not leak. On Windows you can also pass font_size, font_bold, and font_color to control the label rendering (server.rs lines 5651-5658). This is the same affordance Chrome DevTools gives you when you hover an element in the Elements panel and a colored box flashes around it in the viewport.

It runs find_and_execute_with_retry_with_fallback against the live accessibility tree (server.rs line 5477) with a no-op action, so the only outcome that matters is whether the selector resolved. It returns the matched element's attributes (role, name, AutomationId, bounds, process_id), the selector that won (selector_used), and every selector it tried (selectors_tried), with timestamps. If you also pass include_tree_after_action:true, it attaches the resulting subtree under the matched element, which is the cheapest way to walk into a control without re-issuing get_window_tree. This is the equivalent of opening the Elements panel and clicking on a node to see its computed properties, but as a single tool call.

validate_element resolves the selector once. wait_for_element polls the selector until a named state condition holds, with a timeout. The valid conditions are exists, visible, enabled, focused (server.rs line 5990). exists short-circuits to the standard locator wait at line 5838. The other three poll, because the property they are watching can change without the tree shape changing (a button can become enabled in place after a network call resolves). Use exists when you are blocked on a window appearing, visible when you are blocked on a popup that is not yet on screen, enabled when you are blocked on a Submit button that just turned interactable, and focused when you are scripting a text field that has to be the focused control before keystrokes will land in it.

It is the cleanup tool. After get_window_tree drew the overlay or highlight_element flashed a box, the visual artifacts persist by design (the InspectOverlayHandle::Drop impl at inspect_overlay.rs line 60 explicitly does not auto-close). stop_highlighting at server.rs line 7458 removes the active overlay window and drains the active highlight handles. Call it before the workflow ends, or at the start of the next inspection pass, otherwise the user is staring at the previous run's labels.

The MCP tool surface is identical on both platforms: get_window_tree, validate_element, wait_for_element, highlight_element, stop_highlighting all dispatch to a desktop trait that has Windows and macOS implementations. The on-screen overlay (show_inspect_overlay) is currently Windows-only because it uses Win32 layered windows; the cfg gates are at server.rs line 1717 (#[cfg(target_os = "windows")]). On macOS get_window_tree still returns the JSON tree (read from AXUIElementCopyAttributeValue calls underneath), but the overlay is a no-op. highlight_element on macOS uses the AX-side highlight path. The non-visual subset (validate_element, wait_for_element) is full parity.

Two commands. claude mcp add terminator 'npx -y terminator-mcp-agent@latest' to register the server with Claude Code (Cursor and VS Code take an equivalent MCP entry in their config). Then in a chat ask: "open Notepad, run get_window_tree with show_overlay:'ui_tree' and overlay_display_mode:'index_role', then highlight the File menu, validate that it is enabled, and clear the overlay." The agent will issue open_application, get_window_tree, highlight_element, validate_element, stop_highlighting in that order, and you will see labeled rectangles appear over the Notepad window then disappear. The MCP server logs are written under ~/.terminator/logs by default; tail one to see the actual JSON each tool returned.

Yes. The same tool dispatch is reachable from the Rust crate (terminator-rs on crates.io) and the Node binding (@mediar-ai/terminator on npm) without going through MCP. The MCP server is a thin wrapper that exposes the dispatch_tool match block (server.rs line 9844) over JSON-RPC for LLMs; the underlying functions like Desktop::get_window_tree and Element::highlight are exported from the crate for direct use. So a CI script that wants to take an annotated screenshot of an app's accessibility tree before and after a change can call show_inspect_overlay directly, no LLM in the loop.

Playwright MCP and Chrome DevTools MCP expose the same shape of tools (get tree, highlight, click, wait) but bound to a Page or browser context. Their inspector overlay is the browser's own inspect mode. The desktop accessibility equivalent of Playwright MCP is a server that binds tools to the OS accessibility tree and exposes the same primitives over the OS surface. Terminator is that server. The boundary matters when the workflow leaves the tab (a Save dialog, a desktop authenticator, an Excel paste, a native menu): a browser MCP cannot reach those, an OS-rooted MCP can, and the inspector overlay tool gives the agent the same visual grounding for native windows that Chrome DevTools gives it for the DOM. Source: github.com/mediar-ai/terminator.