The automation desktop application framework that does not pick one view of the screen

One fewer failure mode. A pure accessibility-tree tool misses anything the app draws with a custom renderer (SAP, Qt apps, games, some Electron menus). A pure vision tool misses elements obscured behind a modal that the accessibility tree can still name. A DOM-only tool cannot see the Windows file dialog that covers the browser. Terminator runs all five in parallel, merges overlapping detections into a single cluster, and gives the model prefixed indexes like u1, d2, o3, p4, g5. When the UIA entry for a button is missing a name, the OCR entry in the same cluster still has 'Submit' on it. The script targets the cluster, not one specific source, so a single step survives one or two sources failing at once.

crates/terminator/src/tree_formatter.rs line 42. pub enum ElementSource { Uia, Dom, Ocr, Omniparser, Gemini }. The prefix() method on the next line maps each variant to a single character: 'u' for Uia, 'd' for Dom, 'o' for Ocr, 'p' for Omniparser, 'g' for Gemini. parse_prefixed_index(s) does the inverse: hand it 'u1' or 'd23' and it returns the source and the numeric index. Every element in the clustered output is addressable by its prefixed index, which is how the AI references what to click.

cluster_elements at tree_formatter.rs line 460 runs union-find. For every pair of detections across all five sources, should_cluster() at line 452 computes the minimum edge distance between the two bounding boxes, then compares it to 1.5 times the smaller element's shorter dimension. If the distance is under that threshold, the two indexes are unioned. The result is a Vec<Vec<UnifiedElement>> where each inner vec is one visual cluster that might contain a UIA button plus its DOM counterpart plus the OCR word the user actually sees. Within a cluster the elements are sorted in reading order (Y then X), and clusters themselves are sorted the same way. The 1.5x multiplier is relative to element size, not pixels, which means a 12px icon clusters tightly and a 200px card clusters generously.

Only the sources the caller asked for. The clustered tree is assembled in format_clustered_tree_from_caches at line 557, which takes five separate cache maps as arguments (uia_bounds, dom_bounds, ocr_bounds, omniparser_items, vision_items). An empty map for a source just skips it. A typical Windows script runs UIA + OCR. A browser-heavy script runs UIA + DOM + OCR. When the app is a Citrix window or a game, the caller drops UIA and runs OCR + Omniparser + Gemini. The ElementSource abstraction means the downstream code never changes, only the set of populated caches does.

Legacy robotic process automation tools make the author pick one selector type per step. A UI-selector step fails if the UI is not exposed. An image step fails if the button got two pixels wider. An OCR step fails if the font changed. The author then writes a ladder of fallbacks by hand. Terminator inverts it: every step targets a cluster, and the cluster is populated by whichever sources actually resolved an element near those bounds on the last detection pass. When the click fires, the action routes through the most reliable source in the cluster for that element type (UIA for native controls, DOM for browser forms, vision fallback only if the first two did not see it). The author wrote one step, the framework chose the source.

A prefixed index is a string like u17 or d3 or g42 that uniquely identifies one detected element in one detection source on one screenshot. The AI reads the clustered YAML tree, decides which element to interact with, and emits that prefixed index. Under the hood, parse_prefixed_index() splits it into (ElementSource, u32) and the executor looks the element up in the right cache. Without prefixes, two sources might both return index 1 for different elements and the AI would send ambiguous references. With prefixes, one namespace per source, guaranteed by a single-character key the model writes reliably.

Yes. The comment block at tree_formatter.rs line 546 documents it exactly. Each cluster is a block starting with '# Cluster @(x,y)' followed by lines like '- [Button] #u1 "Submit" (bounds: [100,200,80,30])' or '- [button] #d1 "Submit" (bounds: [100,200,80,30])' or '- [OcrWord] #o1 "Submit" (bounds: [102,205,76,25])'. When UIA, DOM, and OCR all see the same Submit button, the AI gets one cluster with three lines. It can click u1 and the framework will route through accessibility APIs, or click o1 and it will route through a coordinate click at the OCR bounds, depending on how the step was authored.

The ElementSource enum is platform-agnostic and lives in the core terminator crate. The UIA source is Windows-specific and resolves through UI Automation COM APIs under crates/terminator/src/platforms/windows. The macOS adapter uses the AX accessibility APIs under the same tree_formatter abstraction. DOM comes from the Chrome extension over the WebSocket bridge, same on every OS. OCR, Omniparser, and Gemini operate on screenshots and are OS-independent. On a machine where UIA returns nothing (Linux, or a Windows app that draws its own UI), the other four sources still populate the clustered tree and the script keeps working.