Consumers of artificial intelligence currently navigate an ecosystem characterised by artificial scarcity. Platform providers frequently impose severe rate limits. Context windows degrade without warning.
The outputs from major platforms often feel progressively worse before each new product release. Platforms restrict access to protect their margins, a process Doctorow (2023) describes as enshittification (Doctorow, C., 2023. The ‘Enshittification’ of TikTok. [wired]).
This consumer experience contradicts the underlying technical reality. The mathematics dictate that these models should be becoming virtually free.
The Collapse of Production Costs
The cost to train and maintain a large language model is falling rapidly. This decline results from four independent variables compounding over time.
First, hardware improves. Next-generation chips and increased memory bandwidth reduce compute costs. Second, algorithmic efficiency increases. Techniques like sparse routing and quantisation allow models to achieve the same performance with fewer parameters. Third, data efficiency improves. Better filtering and synthetic data generation extract more signal per token. Finally, system optimisation reduces the overhead of distributed training.
If we project these factors over a five-year period, the individual reductions appear modest. Hardware might improve by a factor of 0.75 annually. Algorithmic efficiency might improve by a factor of 0.70. However, the compound effect of these four variables creates a steep curve. The total cost to produce a baseline model drops by orders of magnitude. The reasoning engine itself is becoming a commodity.
The graph above projects these factors over a five-year period. While the solid trend lines, representing isolated improvements in hardware, algorithms, data, and systems, yield steady, incremental savings, their interplay is multiplicative. The dashed line captures this compound effect, revealing a remarkably steep decline in relative cost. Ultimately, the total expenditure required to produce a baseline model falls by orders of magnitude, cementing the fact that the reasoning engine itself is rapidly becoming a commodity.
The assumptions are the following:
Hardware improvements (about 1.2–1.5× / year)
Algorithmic efficiency (about 1.3–2× / year historically)
Data efficiency (about 1.1–1.3× / year)
Systems + infra optimisation (about 1.1–1.2× / year)
See also our article about capability densing about algorithmic effects.
The Knowledge Paradox
A counterargument to this cost collapse is the sheer volume of new information. Software developers produce new libraries constantly. Media generation has exploded. If a model must memorise all human output, the training costs would scale infinitely. Furthermore, the supply of high-quality human text has largely saturated.
However, a model does not need to memorise every programming language. It needs to grasp the core invariants of computer science. When presented with a novel library, a competent model can read the documentation at runtime.
The system relies on retrieval augmented generation, which severs the link between model size and the total volume of global data. Because the system is now trained primarily on core logic, a dataset that remains relatively stable, the compounding efficiency gains are finally allowed to drive the pretraining cost down.
As illustrated above, we are witnessing a profound structural break. The upper curve represents the exponential growth of global knowledge. Conversely, the lower curve demonstrates the simultaneous collapse in effective pretraining costs as driven by hardware, algorithms, and the reduced burden of memorisation. This widening gap proves that the mathematical cost of reasoning falls even as the world’s data explodes.
With effects from scaling naturally diminishing, the advantage shifts to laboratories capable of inventing entirely new approaches including algorithmic approaches to handle memory, continuous learning, and long-term planning.
The Fracture of Access
We arrive at a market fracture. The ability to process logic and parse language has commoditised. Open-source releases prove that capable reasoning engines can run on consumer hardware. This occurs because the computational requirements for inference are decreasing. An individual with a standard computer can run a model comparable to the centralised systems from a couple of years ago. As researchers optimise architectures and reduce parameter counts without sacrificing logic, this trend will accelerate.
Furthermore, pairing these local models with personal knowledge systems is cheap. Developers can host local indexes and execute rapid retrieval on basic servers. Therefore, basic reasoning and local knowledge extraction will be universally accessible. You will be able to download a competent model, connect it to your documents, and pay very little for the inference. The logic processing itself is the commodity.
The Premium for Infrastructure
Scarcity and high costs apply to the centralised platforms. A model running on a laptop lacks specific capabilities. While a local system can quickly query a personal hard drive, it lacks real-time retrieval across proprietary databases. It lacks the computational budget to maintain extended context windows. It lacks the execution environments required for autonomous enterprise tasks.
The major providers will charge a premium for this surrounding infrastructure. They must recoup the capital expenditure of their data centres. Consequently, they will restrict the consumer tiers to manage their compute costs. They will reserve the unthrottled access, the memory capacity, and the live data pipelines for corporate contracts.
The Diverging Future
The future holds a bifurcated market. You will possess a capable reasoning engine on your own device for local tasks. Simultaneously, you will face steep paywalls or severe rate limits to access the centralised platforms operated by technology companies. The mathematical model becomes free. The infrastructure scale remains expensive.
Chelsea AI Ventures is providing consultancy services for ML and AI integrations from local recommendation systems to marketing optimisation and automations.