What changed
GitHub made computer use available in public preview for the GitHub Copilot CLI and Copilot app on macOS and Windows. Copilot can read accessible application content and visual context, click controls, enter and edit text, press keys, scroll, drag items, and navigate workflows across applications.
The feature is aimed at the long tail of software that does not expose a reliable API, command-line interface, or MCP integration: legacy desktop tools, GUI-only business systems, and cross-application handoffs. It can be enabled with /computer on, inspected with /computer show, and disabled with /computer off.
Why it matters
The product shift is an expansion of the agent integration boundary. Until now, a builder usually had to wrap a system in an API, terminal command, browser tool, or MCP server before an agent could use it. Computer use gives the agent a fallback path through the same interface a person sees, which makes previously stranded workflows automatable.
That flexibility comes with a much higher safety bar. Computer use is disabled by default, asks for approval before controlling an application, and lets users review or reset always-allowed apps. On macOS, it requires Accessibility permission to control application elements and Screen Recording permission when visual context is needed. GitHub's documentation also warns that changing window state or ambiguous instructions can cause the agent to select the wrong control or affect sensitive data.
The product takeaway
For agent builders, computer use is best treated as a last-mile adapter rather than a replacement for structured integrations. Prefer an API, MCP server, filesystem tool, terminal command, or dedicated browser tool when one exists; use desktop control for the GUI-only gaps.
A robust product should make that fallback explicit: show which application is in scope, require approval for risky actions, keep permissions narrow and revocable, and give users a clear stop path. The interesting lesson is not that agents can click buttons—it is that the quality of an agent product increasingly depends on how carefully it controls the boundary between reasoning and real-world side effects.