A large language model does not look up the truth when it answers a question. It calculates which words are most likely to follow the ones it has been given. That technical fact is the starting point for an argument by Carissa Véliz, a philosopher at the University of Oxford, whose book Prophecy: Prediction, Power, and the Fight for the Future, from Ancient Oracles to AI places chatbots in a long line of tools that people have used to foretell the future, and warns about what happens when society mistakes prediction for knowledge.
Fortune tellers, not truth tellers
Véliz’s core claim, which she has also made in a widely shared essay, is that chatbots are best understood as forecasting systems. Given the phrase “once upon,” a model completes it with “a time” because that is the most common continuation in its training data, not because it understands anything about stories. The same mechanism produces fluent answers to factual questions. When the training data is good and the question is ordinary, the prediction is usually right. When it is not, the system produces a confident guess, which is why these models can state falsehoods with the same tone as facts.
Véliz uses a familiar image to describe the risk of trusting such predictions too much: a farm bird that is fed every day comes to expect that the pattern will continue indefinitely, right up to the day it does not. Extrapolating from the past is useful, she argues, but it creates a false sense of security about what comes next.
Prediction as power
The book’s larger argument is about who benefits. Prediction improves with data, and the organizations that hold the most data, including the companies that build today’s AI systems, gain the most accurate forecasts and, with them, the most influence. Véliz contends that the real danger of AI is not the science-fiction scenario of machines seizing control but something quieter: predictive systems becoming the infrastructure for surveillance, control and inequality.
Her examples include algorithms that decide who qualifies for credit, who is flagged as a risk, and what price a particular customer is shown. These systems act on predictions about individuals, and the people affected often cannot see or contest the forecast. Those with resources can largely avoid the consequences; those without cannot. Véliz argues that in the long run no one is fully insulated from a society organized around prediction.
Averages and sameness
A further concern is cultural. Because predictive models favor the most probable output, they tend toward the average. Véliz suggests that heavy reliance on them flattens variety in language, ideas and even commerce, in the same way that cities around the world have come to share the same chain stores. Creativity depends on the improbable, which is precisely what a prediction engine is designed to avoid.
Living with uncertainty
Véliz’s prescription is less about technology than about attitude. Rather than chasing ever more precise forecasts, she argues, people and institutions should accept that the future is uncertain and treat that uncertainty as the space in which free choices are made. A prediction is a claim about probability, not a fact about the world, and the mistake is to act as though the two are the same.
The argument is a useful corrective at a moment when chatbots are being built into search engines, office software and phones. The systems are getting more capable, but their basic nature has not changed. They estimate what a good answer would look like. Whether that estimate is true is a question the user still has to ask.
