The industry is spending a trillion dollars betting that bigger data centers will stop AI from making things up. The bet is half right. Scale gives crowded concepts more elbow room, at a price that climbs with every error removed. But no amount of gigawatts teaches a model to say “I don’t know.”
A trillion dollars walks into a data center
Here's a number that should make you sit down and count the zeros in the air, or at least put the coffee down. The world (US, China, India and others) is on track to spend more than a trillion dollars on data centers this year, according to Dell'Oro's forecast. In one quarter, the four biggest US cloud providers spent 78% more than they had a year earlier. Nobody says it out loud, but there's a bet hiding in that number: build the thing big enough and it'll stop making stuff up.
I think the bet is half right. Maybe a bit more than half, I go back and forth on it. And the half that's right gets pricier with every error it removes. The half that's wrong? You can't buy your way out of that one at any price.
The elbow room problem
Picture a giant map. A model doesn't store words as text. It gives every concept a location, and things that are alike get placed near each other. "Dog" lives close to "puppy," and both live far from "tax return." That location is what engineers call a vector. Fancy word, it's really just an address.
Models don't keep facts in filing cabinets. They store concepts as directions in a space with a few thousand dimensions, and here's the awkward part: there are way more concepts than dimensions. Researchers have a neat sci-fi word for this: superposition. The model crams in more than it has room for and lives with some interference as the cost of doing business.
Interference is a polite word for collision. Think of two researchers with nearly the same name. Two court cases with the same shape. Two libraries whose APIs look like cousins. They sit close together in that space, and every so often the model reaches for one and comes back with the other.
Imagine a street where two men named John Smith live next door to each other. The mailman is fast and usually right, but every so often one John's package lands on the other John's porch. It's an honest mistake, the addresses were just too close. I'd probably mix them up too.
So here's my theory: a real share of hallucinations are collisions. The model isn't lying, and it isn't being creative either. Two vectors needed more elbow room and didn't get any.
Attention is all you need, plus some personal space
In 2017 a paper called "Attention Is All You Need" handed us the transformer, which is the thing under every chatbot you've ever argued with (and you have, don't pretend). Its base model carried each token as a vector of 512 numbers. Attention runs on similarity: each token asks a question, and whichever tokens look like the best match get to answer.
Call it retrieval by resemblance. It's the reason transformers generalize so well, because similar things are parked next to each other on purpose. It's also the reason they mix things up. If two entities look alike in that space, attention will happily pull from the wrong one and keep a straight face about it.
So collisions aren't a bug somebody forgot to fix. They come bundled with the design that made all of this work.
What a trillion dollars buys
More compute buys bigger models. Wider vectors, more parameters, more training. Wider vectors give concepts more elbow room, and more elbow room means fewer collisions. On this half of the problem, credit where it's due, the data center bet pays.
In mailbox terms, a bigger model is a bigger town with more space between the houses. Fewer packages on the wrong porch.
Then you look at the price curve. The scaling-law research from 2020 found that model error falls as a power law in compute. Translation, roughly: every equal step of improvement costs a multiple of the last one. You don't pay the same again, you pay several times over.
The first collisions are cheap to get rid of. Each one after that costs more than the one before, and the curve never actually hits zero. The gains keep coming, sure, they just keep shrinking while the power bill does the opposite.
What it cannot buy
Ok so now for the bad half. A lot of hallucinations have nothing to do with collisions. The model simply never learned the fact, and it answers anyway.
This is mail addressed to a house that doesn't exist. A sensible mailman would send it back. Ours picks a porch and delivers it anyway.
Research from OpenAI explains why, and it's almost embarrassing how simple it is. We grade models like students on a multiple-choice test. A blank gets zero and a guess might get a point. Guess a stranger's birthday and you've got a 1-in-365 shot, which still beats admitting you don't know.
Their own numbers tell the story. On one benchmark, an older model abstained 1% of the time and was wrong 75% of the time. A newer one abstained 52% of the time and was wrong 26% of the time. Accuracy barely moved: 24% for the old one, 22% for the new.
So what changed between those two models? Not the size of the data center. The newer one is simply far more willing to say "I don't know" instead of guessing. And here's the part I love: it takes much less computing power for a model to notice it's unsure than to actually know the answer.
The ceiling
Put the two halves side by side. Collision errors shrink with scale, but the price climbs faster than the benefit. And it doesn't matter how big your data center is if your model is afraid to say "I don't know."
So here's where that leaves us. Bigger data centers will keep reducing mistakes, but they will never get the number to zero. There's a limit to how reliable today's AI can get just by making it bigger.
To be fair, people have predicted that AI would stop improving before, and they were wrong. The models kept getting better. That could happen again.
But a better AI is not the same as an AI you can trust. To get there, I think we need two things that bigger buildings can't give us. First, AI that admits when it doesn't know. Second, a separate system that checks the AI's answers. Banks figured this out long ago. In a bank, nobody trusts a ledger just because the accountant is smart. They trust it because someone else checks the books.
No data center will ever be big enough to remove the last mistake. Someone, or something, has to catch it.
Which brings me back to that 2017 idea. Attention was the ladder that got AI this far, and it was a great one. But we're near the top of it now. Every extra step costs more and gives less. The juice is no longer worth the squeeze.
Pouring more concrete won't change that. I think it's time to go looking for the next ladder: a new breakthrough as big as attention was.

![Physical AI Foundations and Definitions — Ch.00: Introduction [KR]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fzz2y7pn0xi5qj511e402.png)
![Physical AI Foundations and Definitions — Ch.09: Standards and Definitions [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fw4mzb4ntt5lhj7vgf333.png)
![Physical AI Foundations and Definitions — Ch.06: Relation to Embodied AI and Robotics [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fshqcmgwjgqqejta2saya.png)
![Physical AI Foundations and Definitions — Ch.03: History and Origins [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Frrm68fq83d8whnovwor0.png)
![Physical AI Foundations and Definitions — Ch.02: Physical AI vs Digital AI [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fsgutmwrbqzyn5a22lilm.png)