LLMs Are Reaching a Plateau
(originally published August 27, 2025)*
Not a Revolution, but an Incremental Improvement
After more than two years of anticipation, media hype, promises of general AI (AGI), and even "superintelligence," the recent GPT-5 is a damp squib. This probably explains its release in the middle of summer, on August 7, 2025.
Yet expectations were high and fully matched by the wild promises. Recall that on May 21, 2024, at Microsoft Build 2024, Kevin Scott, Microsoft's CTO, along with Sam Altman of OpenAI, stated: "We are nowhere near the point of diminishing marginal returns on how powerful we can make AI models as we increase the scale of compute" — in front of a giant screen with a beautiful exponential curve. Sam Altman also claimed that the improvement from GPT-4 to GPT-5 would be as significant as that from GPT-3 to GPT-4.
The Exponential Curve Is Actually a Sigmoid
The quasi-religious hypothesis of increasing artificial intelligence through scaling up (more data, more computation) has hit a wall. For large language models (LLMs), bigger and taller do not mean smarter.
The evolution curve of LLMs is not the dreamed-of exponential — it is rather a sigmoid, well known to practitioners of neural networks. What we do know is that the curve started as an exponential and is showing clear signs of slowing down, leveling off on a plateau.
"The most important thing to remember about GPT-5 is that LLMs are reaching a ceiling." — Claude Coulombe
The progress of GPT-5 over GPT-4 is simply not comparable to the performance gains of previous generations. This is consistent with findings from AI expert Gary Marcus, a notable skeptic of the large language model approach to general AI.
"Nobody with intellectual integrity should still believe that pure scaling will get us to AGI." — Gary Marcus
LLMs Don't Reason — They Pretend
A study from Apple's AI research group, "The Illusion of Thinking," challenged the idea that LLMs can reason by generalizing at any depth. They showed that LLMs fail on moderately complex versions of basic algorithms like the Tower of Hanoi. Beyond a certain level of problem complexity, LLMs fail to follow necessary basic logical rules.
A further study, "Is Chain-of-Thought Reasoning of LLMs a Mirage?" by researchers at Arizona State University, shows that Chain of Thought — the supposed trace of reasoning in LLM-based chatbots — is "largely a brittle mirage." As soon as a problem deviates even slightly from training data, the pseudo-reasoning breaks down completely.
Researchers at Anthropic discovered that the steps in the chain of thought do not actually match the outcome: a model may have correct problem-solving steps but produce an incorrect result, or get a correct result from incorrect steps. LLMs are sophisticated simulators of reasoning text, not full-fledged reasoning engines.
Confabulations Remain Unresolved
The problem of confabulations (misnamed hallucinations) persists and has not been resolved after nearly three years and many promises. Within days of GPT-5's launch, the web was overflowing with errors identical to those made by its predecessors.
This model, supposedly at "expert with a Ph.D. level" according to Sam Altman, is still not reliably able to count the number of "R" letters in a word or perform basic arithmetic. The erratic behavior of GPT-5's model-switching module at launch speaks volumes about quality control.
What This Means for Organizations
This plateau does not make AI useless — LLMs remain genuinely valuable for:
- Writing assistance and drafting
- Code completion and review
- Summarization and information retrieval
- Tasks where humans provide quality control
But for critical applications, the human must remain in the loop. By the very nature of LLM-based systems, we cannot guarantee their reliability. Confabulations can be reduced using RAG and other techniques but never completely eliminated.
Organizations that treat LLMs as infallible decision engines — in courts, medical applications, financial systems — are taking on unacceptable risk. Sound AI governance frameworks that acknowledge these fundamental limitations are not optional: they are essential.
The Path Forward
The hypothesis that scaling alone would yield AGI has been falsified. We now need to return to the scientific method: rigorous evaluation, honest benchmarking, and governance frameworks that match actual capabilities rather than vendor promises.
The future of productive AI adoption lies not in chasing the next GPT release, but in deploying current capabilities responsibly — with clear scope, robust governance, and realistic expectations.