Test automation tools for desktop applications, gated by tsc before any pixel moves

Browser tests have a typecheck story by accident: most teams write Playwright in TypeScript, run tsc as a CI step, and the IDE catches the rest. Desktop test tools, on the other hand, sit on three discontinuities. The test process drives Win32 UI through UIAutomation, the assertions live in a vendor scripting language (TestComplete, Ranorex, Squish), and the AI assistant on top of the stack speaks an entirely different protocol. Shift-left typically means 'add a tsc step in CI'. That does not help an AI coding assistant that is editing a workflow file at 2am between two execute_sequence calls. typecheck_workflow makes the gate a tool, not a CI job, which is why it can sit inside the agent loop instead of outside it.

The pattern is ^(.+?)\((\d+),(\d+)\):\s*error\s+(TS\d+):\s*(.+)$. Five capture groups: the relative file path (non-greedy), the 1-indexed line number, the 1-indexed column number, the TS error code (TS2345, TS2304, etc.), and the human-readable message. The anchor on each end and the explicit 'error' keyword keep warnings, info notes, and unrelated tooling output out of the parse result. parse_tsc_output trims each line first, so trailing whitespace from terminal pipes does not break the match. Lines that do not match are silently dropped, which is the right default: tsc emits banners, file lists, and progress lines that have no business becoming TypeError records.

Three lines before, the error line, three lines after. That is the standard rg, fd, and modern compiler output shape, and it is what models like Claude Sonnet, GPT-4o, and Gemini 2.0 are best at consuming. Less context and the model loses scope (which function is this in, what variable was just declared); more context and the prompt budget bleeds. The arrow marker (' -> ' on the error line, four spaces on the others) is intentionally a four-character gutter so the JSON output is column-aligned regardless of line-number digit width. get_error_context formats it with format!("{}{:>4}: {}", marker, line_num, line) at line 111. The model can render it inline in the chat without any additional parsing.

Two reasons. First, bun spawns a TypeScript runtime in single-digit milliseconds where node + tsx adds 200 to 500ms of cold start, and a typecheck loop the assistant runs three or four times per repair cycle compounds that delay. Second, bun's bundled tsc is fully API-compatible with the npm tsc, so 'bun tsc --noEmit' produces identical output to 'npx tsc --noEmit'; there is no behavior difference for the regex parser. The fallback exists because not every customer environment has bun on PATH (macOS dev machines vs Windows CI runners differ), and silent failures at the tool level would block the agent loop. The match returns a clean error string when both are missing; the model handles the message rather than the agent.

Two early-return checks. workflow_path absent returns Err(format!("Workflow path does not exist: {}", workflow_path)). tsconfig.json absent returns Err(format!("No tsconfig.json found in: {}", workflow_path)). Both surface as a CallToolResult::error to the MCP client, which the assistant reads as a tool error rather than a TypecheckResult. That distinction matters: an Err means the gate could not run, so the assistant should fix the workspace; a TypecheckResult { success: false } means the gate ran and the workflow has type errors, so the assistant should fix the source. The two failure modes do not get conflated.

The comparison points are different in kind. Those tools are GUI-first products with a recorder, a script editor, and a runner. The 'tool surface' an AI coding assistant can call is at best a CLI to start the recorder, sometimes a REST endpoint to launch a saved test. None of them ship an MCP server or expose 35 named tools as JSON-RPC, and none ship a typecheck-before-execute primitive that returns structured errors. In practice a team using one of them and an AI assistant ends up writing a wrapper script that scrapes the vendor reporter's stdout, which is exactly the loop typecheck_workflow eliminates. The Terminator answer is to make the model the editor and the agent the runner, with TypeScript as the only language between them.

No, it sits before it. typecheck_workflow only addresses static type errors in the workflow source. Runtime failures (an element does not appear, an assertion fails, a network call hangs) are still reported through the event pipe as StepFailed events and aggregated into whatever final reporter the team wants. A common pattern is to keep a thin sink on top of the event stream that emits JUnit XML at the end of a run; the typecheck step is a separate gate the assistant can invoke standalone. Picture two layers: type errors caught by tsc before any UI runs, runtime errors reported through the event stream while UI runs.

Yes. The MCP tool surface is open. Adding a tool follows the rmcp #[tool] macro pattern in server.rs (35 examples to copy), so a 'lint_workflow' or 'eslint_workflow' or even 'biome_workflow' gate is roughly the same 200 lines of Rust as typecheck.rs: a parser, an enrichment pass, a JSON-shaped result. The workflow rules in the assistant's system prompt then call them in sequence: lint, typecheck, execute_sequence. Because the MCP transport is JSON-RPC, the tools are addressable by name; the model decides the order. The agent does not enforce an order beyond what individual tool descriptions suggest.

Three conditions, all required. output.status.success() (the tsc process exited with code 0). error_count == 0 (the regex parser found zero matching lines). The function returns TypecheckResult { success: output.status.success() && error_count == 0, ... }. A non-zero exit but zero parsed errors is treated as a failure, which guards against tsc emitting an unexpected error format that the regex misses. A zero exit with parsed errors is also treated as a failure, which guards against a regression in tsc's exit-code semantics. Both edges close. The raw_output field is None on success, Some(combined) on failure, so a green run is small and a red run is debuggable.

crates/terminator-mcp-agent/src/tools/typecheck.rs, 278 lines, MIT licensed. The unit tests at the bottom (#[cfg(test)] mod tests, lines 206 to 277) cover parse_tsc_output for single, multiple, no, and noisy outputs, and get_error_context against a temp file with a known type error. The integration into the MCP server is at crates/terminator-mcp-agent/src/server.rs lines 9521 to 9545: a #[tool(description = ...)] block that wraps typecheck::typecheck_workflow into a CallToolResult. Run cargo test -p terminator-mcp-agent typecheck to see the parser tests pass. The full file is short enough to read in one sitting.