The More Capable the Agent, the Easier It Is to Lower Your Guard

Not long ago, an AI assistant was mostly a large language model: you asked a question, it wrote a reply, and that was the end of the loop. Tomasev draws a sharper line. An agent observes a situation, chooses actions, and carries them out in an environment. The model may still draft the plan, but it is “wearing a seatbelt”—tools, permissions, and steps that touch the real world.

That shift changes your job. You still talk to the system in natural language, but you become more of a decision-maker: review, approve, then let it book tickets or message a friend while you do something else. The product feels magical precisely when you stop watching every click—and that is also when risk quietly grows.

Failure rates rise with complexity

Tomasev is blunt: nothing an agent can do is guaranteed to be done correctly every time. Like people, every action carries some chance of failure, and the more steps you chain together, the higher that expected failure rate. Mistakes can be obvious—a wrong restaurant—or subtle, buried in a long email you only skimmed.

That is the door to automation bias. When an agent has been right several times, you start trusting the next run without checking. You skip verification, miss an important detail, and treat silence as safety. Tomasev’s point is not “never automate.” It is that staying in the loop only helps if you stay alert.

Trust is earned—and still not blind

Fry notes that we still live in a world where language models hallucinate, so asking people to trust agents with real tasks is a big ask. Tomasev’s reply is crisp: trying is fair, but trust has to be earned. An agent that keeps failing should not be trusted. An agent that usually works still should not be trusted blindly—you still need ways to verify what it did.

• Treat approval as a real gate, not a reflex tap on “OK.”
• Separate reversible chores from actions that spend money or cannot be undone.
• Expect some hallucination; design the workflow so rare errors do not break the whole job.

What to watch when you hand work over

A wedding planner example in the interview makes the difference concrete. A chat model can list caterers and venues; you still send every email. An agent with Gmail access can draft and send—unless you catch a wrong recipient first. The gain is real. So is the need to confirm the sensitive step before it leaves your account.

Tomasev also stresses that agents shine at chaining many steps—and that those multi-step paths are exactly where human approval matters most, because the settings are sensitive and easy to get wrong. Coding agents are popular for the same reason: lots of work can be turned into software, yet people still need to supervise creative judgment and check results.

The practical takeaway is simple. Agents move you from “ask and read” to “authorize and verify.” The more often they succeed, the more carefully you should design the moments where you still look. Trust grows with evidence—not with the hope that the next complex task will somehow be risk-free.