Webpages Can Trick Your Agent Too
From Nenad Tomasev on the DeepMind podcast: the open web is the agent’s environment—hidden instructions, cloaking, and why defense in depth matters.
As more agents shop, research, and click their way across the internet, someone will try to profit from their weaknesses. Tomasev calls this both frightening and fascinating—and one of the main reasons large-scale agent deployments are hard. Agents do not float in a clean sandbox. Their environment is the web. If that environment is polluted with traps, agents will walk into them.
What humans see is not what agents read
Fry offers a wedding-wine scenario: an agent visits a merchant site, and something on the page quietly rewrites its goal. Tomasev confirms that kind of attack is possible. Pages can hide elements that never appear visually. A person scrolling the pretty storefront never notices them. An agent that parses raw HTML or accessibility trees may treat those hidden strings as instructions—and suddenly behave off-script.
That is not the only trick. Malicious sites can cloak content: show one page to humans and another to agents. Behavior on the page—timing, mouse patterns, how content is fetched—can reveal whether the visitor is a person or a program. Once the site knows, it can tailor the trap.
An old security story in a new costume
Fry asks the hard question: if you cannot control the whole web, how do you stop your agent from going rogue after a bad encounter? Tomasev’s answer is that the problem is not brand new. Opening a malicious email attachment, clicking a shady link, trusting unreliable inputs—these patterns already exist. Machine learning has long fought adversarial examples: tiny image changes people miss that still break a model.
What changes with agents is the blast radius. A chat reply that is wrong is annoying. An agent with inbox, wallet, or calendar access that follows a poisoned page can send money, leak data, or act in your name. The environment and the agent have to be hardened together.
Defense in depth, not one magic fix
Tomasev argues for defense in depth: the problem is too complex for a single silver bullet. You stack layers so that when one fails, another still limits damage. The interview returns again and again to permissions. Give an agent only what it needs for the job. Even if it is jailbroken while talking to a hostile site, the harm should stay as small as possible.
- Least privilege: shopping agents should not also hold unrestricted send rights on your whole mailbox.
- Human gates on irreversible steps: payments, contracts, messages that leave your account.
- Treat untrusted pages like untrusted attachments—assume they may try to rewrite goals.
None of this means agents should stay offline forever. It means the open web is a contested space, and agents are high-value targets. Combining tight permissions, verification, and layered defenses is how Tomasev sketches a security posture people might actually trust—not because traps disappear, but because walking into one no longer has to be catastrophic.