Generative AI is becoming the main door through which people reach information, and increasingly the only gatekeeper at that door. In an essay published by the Guardian on November 18, researcher Deepak Varuvel Dennison argues that this shift carries a risk he and others call knowledge collapse: a future in which people no longer know what they do not know, because the systems they rely on were never exposed to large parts of human knowledge in the first place.
The web is not a mirror of human knowledge
The essay’s central point is that the internet, and therefore the data used to train large language models, is a partial and skewed record. Vast bodies of traditional, local and oral knowledge were never digitized. Because the models cannot see what was never put online, they tend to reinforce the languages, institutions and traditions that are already dominant, above all English and Western academic frameworks.
Dennison cites figures from Common Crawl, the open web archive widely used for AI training. English accounts for roughly 45 percent of its pages, he writes, while English speakers make up about 19 percent of the world’s population. Hindi, one of the world’s most spoken languages, represents only about 0.2 percent of the data, and Tamil, spoken by tens of millions of people, about 0.04 percent. For languages at that level, he argues, there is effectively no room in the model’s view of the world.
Languages carry more than words
The essay stresses that a language is also a container for generations of knowledge about agriculture, weather, medicinal plants, water management, building methods and the local environment. When a language is absent from the digital record, that knowledge becomes invisible too. Dennison gives the example of Bengaluru, where the traditional lake systems and the knowledge of the Neeruganti community that managed them could help the city with its water crises, yet most of that knowledge lives with elders rather than in any database.
Mode amplification
A second mechanism compounds the first. Language models favor the most common words and ideas in their training data, a tendency the essay describes as mode amplification. If pizza appears far more often than biryani in the data, the model will reach for pizza. Popular concepts get amplified; less common ones are pushed further down. Dennison argues that training techniques such as reinforcement learning from human feedback, along with the preferences of corporate customers, push models further in the same direction, making them fluent in the priorities of powerful institutions and often silent or wrong on local and traditional knowledge.
The feedback loop
The danger, as the essay frames it, is circular. The more people depend on AI, the less they discover how much knowledge is missing from it. New generations learn the concepts the models know and repeat, and the diversity of human knowledge shrinks accordingly. Farmers receiving AI-generated advice, for instance, may get recommendations drawn from published international research that do not fit local conditions, while effective local practices go unmentioned because they were never written up.
Dennison does not argue that traditional knowledge is always right. His claim is narrower: that its absence from the systems now mediating knowledge is a loss, and that the question of whether to carry it forward into the digital era is being decided by default, as billions of dollars flow into models that cannot see it. The essay’s closing question is whether societies want to bring that knowledge with them, or settle for knowing only what the models show them.
