Anthropic Computer Use

Every piece of software has an interface a person can use. Only some of it has an API. Anthropic Computer Use closes that gap by letting a Claude model operate a computer the way a person does — by looking at the screen and moving the mouse and keyboard.

The mechanics
It is a tool definition rather than a separate product. The developer runs a virtual desktop and exposes a computer tool to the model; from there the loop is ordinary agent machinery:

• The model requests a screenshot and receives the current screen as an image.
• It reasons about what it sees and emits an action — a click at specific coordinates, a string to type, a key combination, a scroll.
• The runtime performs that action on the virtual desktop and returns a fresh screenshot as the next observation.

Text-editor and shell tools sit alongside it for the cases where touching files directly is simpler than clicking through a window. Anthropic publishes a reference container so the whole arrangement runs sandboxed rather than on your own desktop.

What it unlocks, and what it costs
The appeal is reach: legacy desktop applications, internal tools and sites that were never given an integration all become addressable. The costs are just as real. Every step needs a screenshot, so runs are slow and token-hungry; clicking by coordinate is brittle the moment a layout shifts; and the screen is untrusted input, which makes prompt injection through a web page or document a genuine attack path. Sandboxing and narrowly scoped credentials are not optional here. Progress on this class of task is tracked by benchmarks such as OSWorld.