Epoch AI estimates that frontier models could fully use up the available stock of useful public human-written text sometime between 2026 and 2032. After that, developers may increasingly turn to private datasets and enormous quantities of AI-generated synthetic data — raising a strange possibility: a growing share of what future AI learns may ultimately have been written not by humans, but by earlier AI systems.

Artificial intelligence has often been described as compute-hungry, but the largest language models need another input that cannot be manufactured by a chip factory: useful examples of human language. Epoch AI estimates that the effective stock of quality- and repetition-adjusted public human text is about 300 trillion tokens. If recent scaling trends continue, frontier developers could fully use that stock between 2026 and 2032.

The prospect sounds like a hard deadline for the internet. It is better understood as a forecast about diminishing novelty. A training run can revisit the same books, websites and code, but more passes do not create new human observations. Once datasets approach the effective stock, developers must get more learning from each token, secure text that is not publicly available, move into other data types or generate new examples with models.

The forecast is about useful data, not the last sentence online

Epoch AI’s June 2024 analysis estimates a central stock of roughly 300 trillion effective tokens, with a wide 90 per cent uncertainty interval from 100 trillion to 1,000 trillion. “Effective” does a great deal of work. The estimate adjusts for quality and for the value of training on the same material more than once. It is not a raw count of every word reachable by a crawler.

The 2026 to 2032 window is also a modelled 80 per cent confidence interval, not an expiry date stamped on the web. Researchers combined two approaches: one extrapolated the historical growth of language-model datasets, while the other projected compute growth and calculated the corresponding amount of training data under scaling laws.

“Fully use” therefore does not mean people stop writing, nor that one company must have copied every useful page. It means projected frontier dataset demand becomes comparable with the usable public stock. Fresh human text will continue to appear, but the flow of new writing grows much more slowly than training datasets have grown. Access, duplication, language coverage, copyright, privacy and quality filters also determine how much of the theoretical stock a developer can actually use.

The date depends on how developers spend compute

A compute-optimal model balances its number of parameters against its quantity of training data. Yet a developer may deliberately train a smaller model on far more tokens. That costs more during training but can make the finished model cheaper to run, an attractive trade when inference will be repeated across millions of queries.

Epoch describes this as overtraining. In its simplified scenarios, a modest fivefold overtraining factor could bring full use of the stock forward to 2027. More aggressive assumptions moved the constraint earlier still. Changes in hardware supply, training budgets, algorithmic efficiency and market demand can move the date in either direction.

The underlying paper by Pablo Villalobos and colleagues is correspondingly conditional: if current development trends continue, datasets reach the scale of the available public human text between 2026 and 2032. It points to synthetic generation, transfer learning from data-rich domains and better data efficiency as possible responses. The forecast does not say progress ends when the public stock is fully used.

Private text becomes an economic and legal asset

If another open-web crawl mostly returns material already seen, collections outside the crawlable web gain strategic value. Publishers, libraries, research databases, enterprises and consumer platforms hold books, articles, technical documents, support conversations and specialist records that are private, paywalled, archived or contractually restricted.

That does not make every private archive good training data. Much of it will be repetitive, narrow, sensitive or impossible to use lawfully. But original material with clear provenance and permissions can provide something a larger public scrape cannot: genuinely new examples and domains.

The change is already visible at a smaller linguistic scale. Silicon Canals reported how Dutch publishers supplied legally obtained news text to GPT-NL, expanding the high-quality Dutch data available to the project. The collaboration illustrates a possible market structure for scarce training material: rights holders provide access, developers gain traceable data, and the terms of use become part of the model’s technical foundation.

This is also why data provenance may become as important as volume. A corpus labelled by source, licence, creation date and whether it was written by a person or generated by a model is easier to audit and rebalance. A vast undifferentiated scrape may contain more tokens while offering less certainty about what those tokens represent.

Synthetic data is not one uniform substance

The phrase “AI-generated data” covers very different practices. A model can produce mathematics problems with checkable answers, executable code, simulated conversations, critiques, translations, alternative proofs or examples targeted at a known weakness. In some domains, a compiler, calculator, simulator or formal verifier can reject wrong answers cheaply. In open-ended writing, quality and factual accuracy are much harder to establish.

Synthetic data is most useful when generation is part of a controlled curriculum. A stronger model may create examples for a smaller one. Multiple models can propose and criticise answers. Human reviewers can concentrate on uncertain cases instead of writing every sample from scratch. The model is not creating truth merely by producing more text; the value comes from selection, verification and a connection to an external objective.

A 2025 Google Research paper on curating synthetic training data formalised part of this problem. Its experiments found that dynamically focusing labelling resources on harder examples could improve performance. In that setting, the scarce input is not token generation. Models can produce tokens almost without limit. The scarce input is a reliable signal about which examples are correct, useful, diverse and worth learning from.

Recursive training can narrow the world a model represents

The alarming version of synthetic training resembles a photocopy of a photocopy. A model generates plausible text, the output enters the next model’s training set, and small distortions become the raw material for another generation. If that loop replaces the original distribution rather than extending it, information can disappear.

A 2024 paper in Nature found that indiscriminate recursive training can produce model collapse. In the researchers’ experiments and mathematical treatment, low-probability parts of the original distribution were lost first. Later generations became narrower and accumulated errors from their predecessors.

That finding is a warning about a training regime, not proof that all synthetic data poisons a model. The mixture matters. Another study found that retaining the original real data while accumulating synthetic generations avoided collapse across the language and generative-model settings it tested. The two results fit together: replacing reality with recursively generated approximations is dangerous, while preserving the human anchor and adding selected synthetic examples can behave differently.

Real-world frontier systems are more complicated than either experiment. Developers filter data, change model architectures, use human feedback and train on mixtures that are rarely disclosed in full. “Model collapse” should not become a slogan that substitutes for inspecting those choices.

The future corpus may be recursive without becoming autonomous

Future models may learn from answers, exercises and simulations written by earlier systems. In a literal sense, a growing portion of their training tokens could be machine-produced. But the lineage will still lead back to human-written corpora, human-designed environments and human decisions about what counts as a successful output.

Synthetic data can recombine knowledge, create practice problems and explore combinations that no person wrote down individually. It cannot retroactively add the lived experience, minority language usage, local knowledge or unusual failure cases missing from its source models. Without deliberate preservation, those lower-frequency parts of the distribution are exactly what recursive systems risk smoothing away.

The more consequential change may therefore be one of ownership and control. Public text once looked abundant enough to be treated as background infrastructure. If high-quality human data becomes a binding input, access to authentic archives and interactions becomes a competitive advantage, while provenance and licensing become engineering concerns rather than paperwork at the edge.

The strange possibility in the title is real, but it is not a closed loop yet. The central question after the public stock is fully used will not simply be whether AI trains on AI. It will be who controls the human anchor, how carefully it is preserved, and whether synthetic expansion broadens what models can learn or merely repeats their ancestors more fluently.