The Absorption Economy
Generative AI is destroying the hyperlink by absorbing content without returning traffic, forcing creators to lock the open web behind paywalls.
By Jonah Reyes
Sparked by Cloudflare's new AI traffic options for customers · discussion
Cloudflare recently introduced a one-click AI bot blocker that allows all customers, including those on a free tier, to instantly stop artificial intelligence from scraping their content for model training. This deceptively simple software toggle is designed to let website owners effortlessly block incoming crawlers from their domains.
[Aside: B2B security dashboards are perhaps the least cinematic places on earth, rendering them the perfect hiding spots for existential industry shifts. The truly tectonic realignments in tech rarely arrive via a grand keynote; they arrive as a gray radio button in a settings menu you check while waiting for your coffee to brew.]
Flipping a toggle on a screen to block an incoming request from an OpenAI crawler shuts off a digital valve that previously allowed the uninhibited exchange of text across the open web. This represents something fundamental. It is the architectural death of the hyperlink.
When you are the one paying the server bill, the romanticism of the open web fades quickly. Small site operators are acting as rational economic actors. You allowed the Googlebot to read your site because the bot was guaranteed to send human eyeballs back to your site. The economic operating cycle was positive. It was an ecosystem of continuous, oxygenated flow. Let us call this historical norm the Routing Economy.
Since the dawn of PageRank, the open web operated structurally as a global circulatory system. Search crawlers functioned as the heart, pumping attention, daily active users, and subsequent ad revenue back to the original host via the hyperlink. Yes, Google extracted your text and metadata, and yes, they built a massive monopolistic empire on the back of your uncompensated labor, but they did so to generate a map that facilitated a mutualistic exchange of value. The hyperlink provided positive optionality, driving network effects that allowed independent blogs, niche forums, and massive newsrooms alike to monetize their traffic.
That flow has been surgically severed. Consider how Google themselves phrased the transition when they decoupled traditional search from AI, announcing Google-Extended as a new control for web publishers. They framed the tool as a choice for administrators to manage whether their sites help improve Bard and Vertex AI models over time.
This corporate phrasing masks a profound architectural divorce. The infrastructure of discovery has been formally decoupled from the infrastructure of knowledge generation. This bifurcation marks the genesis of the Absorption Economy.
In this new paradigm, artificial intelligence bypasses the act of routing entirely. It just eats. The underlying architecture of the internet has shifted from a circulatory system to a digestive tract. Large language models metabolize independent data permanently into an opaque black box of neural weights, entirely severing the path dependence that previously rewarded the original creator for their effort.
To visualize this, look to Hayao Miyazaki’s animated masterpiece Spirited Away and the character of No Face. The spirit initially appears benign, offering tokens of gold to the workers in the bathhouse, but soon begins a terrifying rampage of consuming everything in its path. No Face absorbs the voices, personalities, and traits of his victims, projecting a disturbing amalgamation of the things he has eaten, while leaving an empty, starved wake behind him.
Generative LLMs function with the same ravenous, terminal architecture. They offer the initial magic of instantaneous answers and conversational interfaces, but they swallow the source material whole, ensuring the end-user never actually needs to click through to the origin. Total metabolic absorption. Zero outbound traffic. Extremely bad vibes.
So, faced with a digestive system that fundamentally starves the host, the only rational response for a creator is to build a wall. Erecting a defensive boundary is merely the initial phase; the natural evolution of that barrier is the installation of a tollbooth. Cloudflare is already preparing the necessary tooling for the next phase of this architectural shift, openly teasing a marketplace to charge AI developers directly for scraping access. They envision a future where creators set a strict price for their content and AI models purchase access, ensuring explicit financial compensation for the human labor that models rely on to generate value.
This is the rapid financialization of the void. We have already watched this macro validation play out at scale. Reddit made the highly controversial decision to place its application programming interface behind a steep, enterprise-tier paywall, explicitly citing the urgent need to stop tech giants from freely downloading their massive, irreplaceable corpus of human conversation to train models.
Because human conversation is the ultimate training data—the absolute rarest, most fungible asset in the race to build artificial general intelligence—the open web is now violently fracturing into sovereign, gated domains. Publishers must now navigate an environment where their content, once a reliable engine for discovery, has been structurally downgraded into raw biomass meant only to sustain a machine learning model. They must choose between strict data sovereignty via a block toggle or licensed extraction via a tollbooth. The era of permissionless, frictionless discovery—the very foundation of the web’s initial explosive growth and the enabler of countless online subcultures—is functionally over. It is being rapidly replaced by walled gardens attempting to extract rent before they are entirely digested by the machine.
We are, at our core, status-seeking primates obsessively tracking our standing within the tribe. For three decades, the hyperlink served as the undeniable currency of digital status. It conferred intellectual authority, drove parasocial validation, and proved that you existed in the vast expanse of the web. Publishing online was always, on some level, an act of ego. If the underlying algorithm of the internet refuses to grant publishers that status—if it refuses to route a single human reader back to their domain—they will inevitably demand hard cash instead. The hoarding of data and the erection of paywalls is an entirely rational response when the alternative is feeding an opaque intelligence that strips away your identity and offers nothing in return.
If I were an executive at a generative AI startup, I would regard this seemingly minor Cloudflare UI update as a terrifying signal. The temporary arbitrage of the open internet is closing. The foundational layer of the web is arming itself, rapidly converting the free pasture of human knowledge into a fiercely guarded private marketplace.
Remove the hyperlink, and the web stops being a circulatory system. It becomes a buffet. And the structural problem with being the meal is that, eventually, the digestive tract runs out of food.