GitHub Copilot adds Computer Use to operate desktop applications
IA en un minuto newsroom · Editor: Jon Elgezabal

In 30 seconds
GitHub's Copilot can now run desktop programs the way a person would: it reads the screen, clicks buttons, types and carries data from one app to another. The feature, called computer use, is available as a preview in Copilot CLI and the Copilot app for macOS and Windows, and it works with legacy software that has no API. Before taking control of each app it asks for permission, and organizations can switch the feature off.
GitHub announced on October 1 that Copilot, its AI assistant for coding, is now able to operate desktop applications on behalf of the user. The feature goes by the name computer use (the assistant works the computer much as a person would) and has entered public preview, on macOS and Windows, in two places: Copilot CLI, the version for the terminal, and the GitHub Copilot app.
Copilot reads what applications contain and what appears on screen, clicks on controls, types and edits text, presses keys, scrolls, drags, and carries a workflow across from one application to the next. GitHub says this opens the way to automating tasks in legacy programs, or in programs that offer only a graphical interface and have no API, no command line and no MCP integration (MCP being the standard for connecting AI agents to other tools). The examples it gives are summarizing browser notifications, updating the content of a presentation and taking information through a workflow inside a desktop application; in GitHub's video, Copilot processes an expense report in Safari.
Before it takes control of an application, Copilot asks for approval, and the list of apps set to always be allowed can be reviewed or reset. On macOS it walks the user through granting the Accessibility and Screen Recording permissions, and organizations can turn the feature off in their managed settings. The feature is switched on with /computer on in Copilot CLI or, in the app, from Settings, Computer Use.
Why it matters · analysis and opinion
Old management software, with no API and usable only with mouse and keyboard: until now that was the hardest corner to automate, and that is exactly where this feature is aimed. Copilot does not need connectors, because it looks at the screen and acts like a person, even jumping from one application to another. That same power is its risk: a wrong click in an invoicing or customer program is not easily undone. GitHub has put sensible brakes in place, such as asking for permission per app and letting the organization turn it off. Since it is in preview, the prudent approach is to try it first on a test computer with a repetitive, supervised task, and to go carefully through the list of always-allowed apps.
Official source: GitHub · Written with the help of AI: how we make the news


