AX tree vs screenshot for computer use on Mac. The choice is per-element.

When the target subtree contains at least one element with a named role below AXGroup and a non-empty accessible name. That is roughly every native Cocoa surface: Notes, Mail, Calendar, Reminders, Finder, System Settings, every AppKit and SwiftUI app, plus Xcode and Office for Mac. On those surfaces, AX gives you role, title, bounds, and state in a few kilobytes of structured text per window, and the click happens via AXUIElementPerformAction with kAXPressAction in microseconds. The agent reads structured text instead of grounding from a 2576px PNG every turn.

Three real surfaces. First, Electron apps and Chromium-embedded webviews (Slack, Discord, VS Code, Notion, Cursor, ChatGPT desktop, Claude desktop): macOS Chromium maps the DOM to nested AXGroups with no titles, no AXIdentifier, and no role information. Even with full accessibility permission, the agent gets unactionable subtree shapes. Second, custom-painted surfaces: games rendered through Metal, Figma's canvas, Photoshop's document area, terminal emulators with GPU compositors. AX sees one opaque element with the window bounds and nothing inside. Third, image interpretation: any task where the user asks the agent to read a chart, an image embed, or visually compare two states. The tree does not carry that information.

Per-element. A single task on Mac frequently spans both regimes: open Slack (Electron, vision), copy a message (vision), switch to Notes (AX), paste and save (AX). Routing at the task level forces you to use vision everywhere just because one surface in the task needed it, which is wasteful for the AX-friendly steps and slow for the agent loop. Routing at the call level keeps each tool invocation honest. Terminator exposes this as the vision_type parameter on click_element with five values (Ocr, Omniparser, UiTree, Dom, Gemini) that the agent selects per call, defaulting to UiTree when omitted. Source: crates/terminator-mcp-agent/src/utils.rs lines 1060 to 1073.

The default TreeWalker does not traverse windows. macOS AX returns children of one window when you read kAXChildrenAttribute on an app element, but a real Mac app frequently has several windows (the main window plus inspectors, palettes, modal sheets, preferences). To see them all you have to read kAXWindowsAttribute and walk each window separately. Terminator's repo has this written into its first comment: crates/terminator/src/platforms/tree_search.rs line 1 reads 'TLDR: default TreeWalker does not traverse windows, so we need to traverse windows manually.' The file ships a TreeWalkerWithWindows struct that enumerates kAXWindowsAttribute siblings and walks them all. If your agent uses an off-the-shelf accessibility crate without this workaround, you will silently miss elements in roughly half the apps your users open.

Not yet. The Rust core has cross-platform scaffolding: AccessibilityEngine is shaped as a trait at crates/terminator/src/platforms/mod.rs line 86, the tree_search.rs file uses the accessibility-sys crate to walk AXUIElement trees, and several files (element.rs around line 1883, lib.rs around line 1567) have target_os = 'macos' code paths. But the same mod.rs ends at lines 318 to 320 with #[cfg(not(target_os = 'windows'))] compile_error!('Terminator only supports Windows. Linux and macOS are not supported.'). The published binaries on npm (terminator-mcp-agent) and pip (terminator-py) are Windows-only as of this writing. If you need this routing shape on Mac today, the architecture is here to copy: use atomacos or a pyobjc shim around AXUIElement for the AX path, wire OmniParser or Gemini for the vision path, and route per-call.

Two reasons. First, Retina. A Mac on a 16-inch built-in display is 3456 by 2234 logical pixels, which is roughly 7.7 megapixels at native resolution and around 3 megapixels after Anthropic's 2576px long-edge downscale. Every screenshot is large, every encode round-trips through PNG, every upload spends real bandwidth. Second, the round-trip shape. Native Anthropic computer use is screenshot, act, screenshot, where the post-action screenshot is mandatory because the tool result for left_click does not echo state. Every action costs two round-trips and one vision pass. The AX path on Mac is one Mach call, one InvokePattern equivalent (AXUIElementPerformAction), and one optional tree re-read. The first runs in microseconds; the second runs in hundreds of milliseconds plus vision pricing.

Not for free, and the absences matter. The tree gives you role, name, bounds, value, and state. It does not give you color, font, image content, custom-drawn icons, or anything rendered into a canvas. If your agent task involves 'click the red button' or 'find the chart that is trending down' the tree is silent on the discriminator. Production agents handle this in two ways. The first is a small per-element vision call on demand: when the agent has a target named element in the AX tree but needs to disambiguate among several matches, it takes a cropped screenshot of the bounds and asks the vision model to pick. The second is the explicit vision_type fallback per call when no usable AX subtree exists. Both are cheaper than 'screenshot every turn' but recover the visual signal where the task needs it.

Three modes, in roughly the order users hit them. One: silent permission failure. AXUIElementCopyAttributeValue against another process returns kAXErrorAPIDisabled until your binary is in System Settings, Privacy and Security, Accessibility. A first-run agent that does not detect this with AXIsProcessTrustedWithOptions tells the model 'the app has no UI' and the loop dies confused. Two: window blindness. The default child walk misses non-main windows, so the agent thinks Calendar has no inspector or that Mail has no compose window. Three: Electron silence. The agent enters Slack and the tree is six nested AXGroups with no titles. The model emits 'click the Send button' and the resolver returns no matches, then synthesizes a click at coordinates the model invented from nothing. The fix in all three cases is the same: detect each failure shape and route to the vision_type fallback before the agent thinks the surface is empty.

The opposite shape. Every turn costs a screenshot, an encode, a vision pass, and a coordinate decision the model has to ground from raw pixels. Most native Mac surfaces are stable AX-friendly apps where the model spends its entire vision budget re-finding a button it could have looked up by role plus title for zero tokens. Latency stacks up too: a 100-action task at 2 seconds per action is 200 seconds of vision; the same task with AX routing on the native surfaces and vision routing on the two Electron surfaces is around 30 seconds. The model's accuracy also dips: a 5-pixel tooltip shift, a theme animation mid-capture, a transient hover state, all of these turn into 'click missed' on the screenshot path but never reach the AX path because the tree is theme-independent.

AX-first with vision-fallback per call. Default every click and read to the AX tree. Detect the three failure shapes (permission denied, AXGroup soup, single opaque AXGroup) at the tool-router layer and switch to vision_type Omniparser or Gemini on those specific calls. Keep the same selector grammar on top so the agent code reads the same on both branches. The Terminator MCP server's click_element tool is shaped this way: it takes a selector, an optional vision_type, and runs the tree path by default. The agent gets to pick the grounding mechanism without restructuring the prompt or the tool surface.