Browser Agent

Almost everything a person does online happens through a browser, and almost none of it has an API. A browser agent closes that gap: it drives a real browser — clicking, typing, scrolling, navigating — to complete a task you described in a sentence.

Seeing and acting on a page
Each turn of the loop is a perception step followed by an action:

• Perception — the page arrives as an accessibility tree or simplified DOM, as a screenshot with the interactive elements numbered, or both. A raw page is far too large for a context window.
• Action — the agent emits something concrete: click element 14, type into the search box, scroll, go back, extract this value.
• Observation — the page changes, a new view comes back, and the plan is revised against it.

Underneath, the browser is usually controlled through Playwright or the Chrome DevTools Protocol.

Why it is harder than it looks
The web was built for human eyes, and that is the whole difficulty. Pages shift beneath the agent, modals steal focus, logins and CAPTCHAs block the path, and one mis-click on a confirm button can be irreversible. Progress is tracked on WebArena and Mind2Web, where success rates stay well below a competent person's. The security problem is sharper still: every page the agent reads is untrusted input, so text planted on a site can act as an instruction — which is why serious deployments scope credentials tightly, keep the browser sandboxed, and require approval before a purchase or a submission.