macOS AX vs Windows UIA agent: what one trait can hide, and where it leaks

Same job, different shapes. Both expose the live UI tree of running apps and let you act on elements without screenshots. Windows UI Automation is a COM-based API rooted at IUIAutomationElement, with strongly typed control patterns (InvokePattern, ValuePattern, TogglePattern, ExpandCollapsePattern) you query and call. macOS Accessibility uses an opaque AXUIElement reference and a string-keyed attribute model: AXUIElementCopyAttributeValue with kAXTitleAttribute, kAXValueAttribute, kAXRoleAttribute, and so on, plus AXUIElementPerformAction with kAXPressAction for clicks. UIA gives you a typed dispatch table; AX gives you a string-keyed bag of attributes and actions. The shapes converge once you wrap them in a higher-level trait, but the call sites underneath look nothing alike.

At the signature level, yes. Terminator's pub trait AccessibilityEngine at crates/terminator/src/platforms/mod.rs line 86 declares find_element, find_elements, get_focused_element, get_window_tree, click_at_coordinates, and a dozen more, all with platform-neutral types (UIElement, Selector, AutomationError). On Windows the implementation walks the IUIAutomation tree via the uiautomation Rust crate, on macOS it would walk the AX tree via accessibility-sys or a similar shim. The trait says nothing about COM or Mach. Below the trait, the implementations diverge: the Windows engine resolves IUIAutomationCondition trees and calls IUIAutomationInvokePattern.Invoke, the macOS engine would build kAXAttributedStringForRangeParameterizedAttribute queries and call AXUIElementPerformAction. The leaks are not in the trait shape; they are in what each method has to do underneath to return the same UIElement.

Three places, in increasing order of pain. First, role names. macOS prefixes most roles with AX (AXButton, AXTextField, AXWindow), Windows uses bare control type strings (Button, Edit, Window). Terminator's repo has the leak written into element.rs around line 1885: the macOS-only branch matches role == 'axwindow' || role == 'window', the Windows branch matches role == 'window'. Second, action invocation. UIA exposes a typed pattern: GetCurrentPattern(InvokePattern) then Invoke. AX uses a string action name: AXUIElementPerformAction(element, kAXPressAction). The trait method click() papers over both, but the fallback story when the action is not supported is different on each. Third, focused element. UIA's GetFocusedElement returns a process-scoped element instantly. AX requires you to walk from the system-wide AXUIElement (AXUIElementCreateSystemWide) and ask for kAXFocusedUIElementAttribute, with permission gates that can fail at runtime. Same trait method, same return type, very different failure modes.

Nothing, on the main branch. The trait is shaped to be cross-platform, the locator grammar in selector.rs is platform-neutral, and several files (element.rs around line 1883, lib.rs around line 1567, health.rs around line 145) carry #[cfg(target_os = 'macos')] code paths. But crates/terminator/src/platforms/mod.rs at lines 319 and 320 ends the file with: #[cfg(not(target_os = 'windows'))] compile_error!('Terminator only supports Windows. Linux and macOS are not supported.'). On any non-Windows host the workspace will not build. The published binaries on npm (terminator-mcp-agent), pip (terminator-py), and crates.io (terminator-rs) are Windows binaries only. If you read llms.txt in the repo it states this directly: 'The Node.js, Python, and MCP packages currently ship Windows binaries only.' macOS support is a trait shape and some scaffolding, not a working engine.

Three honest paths. One: use a separate engine per OS. atomacos for macOS Python, FlaUI or pywinauto for Windows Python; or AXSwift for native macOS and the C# UIAutomationCore for Windows. You write two driver modules and merge them at a higher layer. Two: pick a screenshot-and-vision engine like Anthropic Computer Use or OpenAI Operator that does not depend on OS accessibility at all; you trade speed and determinism for portability. Three: use a Playwright-shaped trait library (Terminator on Windows is one) and assume one binary per OS, with the trait keeping your higher-level code identical. None of these gives you 'one binary, two platforms'. The trait can promise that signature; the underlying COM and Mach worlds will not let you ship a single dlopen-able shared library that talks to both.

Because the trait method is doing the heavy lifting of picking the right action. Inside element.click(), the Windows implementation tries patterns in order: InvokePattern.Invoke for buttons, TogglePattern.Toggle for checkboxes, SelectionItemPattern.Select for radios, then falls back to a synthetic mouse event at the element's bounding box if no pattern matches. The macOS implementation would try AXUIElementPerformAction with kAXPressAction first, then kAXShowMenuAction or kAXIncrementAction depending on the role, then fall back to a CGEventCreateMouseEvent at the element's AXFrame. The trait promise is 'click does the right thing for this element'. The implementation per platform owns the dispatch table for what 'right' means. The reason this works is that both APIs expose enough role information to pick a sensible default action; the leak is only that the default actions are named differently and have different fallbacks.

Not exactly, and this is one of the largest practical differences. UIA defines an AutomationId attribute that an app developer can set and that survives localization, theme changes, and most refactors. WinAppSDK apps (Calculator, Notepad, Settings) populate it for nearly every interactive element, and a selector like id:NumberPadFiveButton works on every Windows machine in any language. macOS AX has no native equivalent. The closest thing is kAXIdentifierAttribute which AppKit sets sporadically, mostly for SwiftUI-built controls that opted in. In practice, agents on macOS rely heavily on (role, title, parent) tuples and positional walks (the third button in the toolbar of the focused window), which are inherently more localization-sensitive than UIA AutomationId-based selectors. A trait abstraction can hide this with a single Selector grammar, but the macOS resolver will silently fall back to title and position more often, which is something you observe in production but cannot fix in the trait.

Right, and this is a runtime leak the trait cannot fully hide. On Windows, UIA calls just work; the COM API is not gated. On macOS, AXUIElementCopyAttributeValue against another process returns kAXErrorAPIDisabled or kAXErrorNotImplemented unless your binary has been added to System Settings > Privacy & Security > Accessibility, and the user has toggled it on. The first call from a fresh install will silently fail and the agent has no signal to recover. Production macOS agents prompt with AXIsProcessTrustedWithOptions during init, surface a permission setup screen, and re-check on every cold start. None of this maps to anything UIA-side. The trait method get_root_element() succeeds on Windows from any process; on macOS it fails until the user clicks a checkbox in System Settings. The trait can return the same Result type, but the failure modes need different runbooks.

Yes for both, for different reasons. On macOS, AX silently no-ops on AXPress for many web views in Chrome and Safari, and Electron apps inconsistently expose role and title. The honest production path on macOS uses AX where it works and falls back to OCR or vision (OmniParser, Gemini) on Electron and web-rendered surfaces. On Windows, UIA covers WinUI, WPF, WinForms, MFC, and most major IDEs natively, but games, custom-painted line-of-business apps, and anything rendered via DirectX or canvas show up as one opaque element. The fallback is the same: OCR or vision for grounding, then synthetic input for the click. Terminator's MCP server exposes this as vision_type with five values (UiTree, Ocr, Omniparser, Gemini, Dom), defaulting to UiTree. The same router would apply on macOS, with AX as the default and screenshot+vision as the labelled fallback for AX-empty surfaces.

Match the platform your users are on, and assume one engine per OS. If your agent runs on Windows desktops, a UIA-backed engine like Terminator (terminator-rs on crates.io, terminator-mcp-agent on npm) or a UIA wrapper like FlaUI is the production path; UIA is mature, AutomationId-based selectors are stable, and synthetic input is the fallback for AX-empty surfaces. If your agent runs on macOS desktops, atomacos or a pyobjc shim around AXUIElement is the production path today, paired with a vision fallback for Electron and Chromium. If you need both, write a thin layer that picks the engine per OS and exposes a Playwright-shaped Locator API to your agent code; do not expect any single library to ship one binary that works on both. The Playwright-shape is the right abstraction, but each leg of it is a separate ship.