Bots Don’t Click Ads: Why the Web’s Business Model Must Be Rewritten
Matthew Prince’s blunt constraint: if machines dominate traffic, the ad model that funded free content for nearly thirty years starts to fail—and pay-per-crawl is one proposed path out.
For nearly thirty years, the open web mostly ran on a simple bargain: sites published content for free, and advertisers paid for human attention. That bargain funded blogs, newsrooms, tools, and entire product categories. Prince’s warning is almost cruel in how simple it is: bots do not click ads.
If automated traffic already rivals or exceeds human traffic—and may someday dwarf it—then the commercial logic that paid for the free internet is loosening from the foundation up. Building more servers is not enough. Someone still has to pay for content and for the infrastructure that delivers it.
From pageviews to a new scoreboard
Matt Turck asked whether the industry needs a new value system: instead of clicks and pageviews, perhaps actions. Prince’s reply was that the sector first has to discover what the new yardstick even is—and that the answer may overturn habits people treat as common sense.
In a world where agents research on a user’s behalf, attention is no longer the scarce resource in the same way. A site can be fetched a thousand times and still generate zero ad impressions that matter. Volume without human eyes is not the old business.
Two steps Prince sketches: control, then micropayments
Under the banner of a “Content Independence Day” idea he floated about a year earlier, Prince framed a path in two moves. First, give creators real access control: tools that can either make AI crawling more efficient or restrict it—so publishers choose, rather than being silently harvested. Second, build pay-per-crawl micropayments: charge for each automated fetch in a way small enough to scale.
That second step is technically demanding. He argued it needs transaction capacity roughly two orders of magnitude beyond a classic card network like Visa. Cloudflare, he said, has been working with partners such as Coinbase and Stripe to activate the long-dormant HTTP 402 status code—Payment Required—as plumbing for that future.
Why he still sounds optimistic
Prince compared the shift to music: from Napster-era free-for-all, through iTunes teaching people to pay, to Spotify sending tens of billions a year to creators. The lesson he draws is that content markets can be rebuilt after a free-for-all era. He also argued that models ultimately hunger for net-new knowledge—local, specific, hard-to-duplicate facts—more than for endless recycled pages.
- Outrage headlines optimized for human clicks lose leverage when agents, not people, do the browsing.
- Unique local knowledge may regain value under a pay-for-access regime.
- The open question remains unsettled: what final shape the post-ad web takes.
The traffic numbers make the pressure obvious. The harder work is redesigning who pays whom when the visitor is a machine acting for a person. Until that settles, creators, platforms, and AI companies are negotiating over the same scarce thing: useful content that is worth fetching at all.