One question, followed from the keyboard to a GPU in America and back: about 13,600 km, one second, and a model that never knows its answer in advance.
Your question, as light, crossing the Atlantic.
You type “What is the capital of France?” and press Enter. About a second later, ChatGPT answers: “The capital of France is Paris.”
We do this dozens of times a day, and the answer feels instant, almost local. It isn’t. From my desk in France, that one question went to the bottom of the Atlantic, into a hall full of GPUs in America, through every layer of a neural network, and back, in about one second.
I wanted to see all of it, not just “the model” or just “the internet”, so I followed one question from start to finish and turned it into a 14-minute animated video. This article is the written tour. The video shows every step.
I’ve always learned best by teaching: having to explain something is the fastest way to find out what you don’t understand yet. This video is the first one on my channel, The Next Token Lab.
1. Leaving your laptop (0 ms)
Before anything moves, your browser does three things. It packs the question into a small structured message (JSON). It looks up where chatgpt.com lives in the internet’s phone book (DNS). And it locks the message (TLS), so nobody on the way can read it.
Pack the message, find the address, lock it: three steps before your question even leaves the laptop.
Then it leaves as pulses of light in a glass fibre 0.125 mm across, barely thicker than a human hair.
Your question, as light in a glass fibre.
2. Paris, two milliseconds later
From my connection, the question needed 9 hops and about 2 milliseconds to reach the nearest edge server, in Paris. That server, run by Cloudflare for ChatGPT, is the front door. It checks that you’re not a bot or an attack, and it’s where the encrypted connection is unlocked.
3. Under the Atlantic
OpenAI doesn’t say which data centre answers a given question. But for a question from Europe, it’s most likely in the United States.
So the message heads to the Atlantic coast, to a small building by the beach where a transatlantic cable goes into the sea. On the ocean floor, kilometres down, that cable is about as thin as a garden hose. Inside are 12 pairs of glass fibres, wrapped in steel wires for strength and copper that carries power to repeaters every 60 to 100 km. The repeaters boost the light along the way.
Inside a subsea cable: 24 glass fibres (12 pairs), steel wires, and copper carrying power from the shore.
In glass, light travels at about 200,000 km per second. The cable is about 6,400 km long, so that’s at least 32 milliseconds: the longest stretch of the whole trip. About 45 milliseconds after you pressed Enter, your question comes ashore in Virginia.
4. OpenAI’s front door
Now it reaches OpenAI’s servers. They receive tens of thousands of messages every second, so a load balancer sends yours to a server that isn’t too busy. Those servers run in huge clusters managed by a system called Kubernetes. (In December 2024, one bad update overloaded those clusters and ChatGPT was down for hours.)
Before anything else, OpenAI checks who you are and that you haven’t hit your limit. Your message is saved to your history: ChatGPT’s main database keeps one master copy and about fifty copies around the world. Automatic safety classifiers look for harmful requests. A question about capitals goes straight through.
Then comes something most people don’t know: the model never sees your question alone. ChatGPT wraps it in a longer text, with instructions about who it is and how to behave, what it remembers about you if memory is on, and the conversation so far.
And now, the first step that’s really about AI: tokenization. The model can’t read letters. It only works with numbers. The text is cut into tokens, and each token gets its number from a fixed list of about 200,000. Common words are one token, usually with the space in front. Rare words are cut into pieces: “strawberry” becomes st · raw · berry.
Three pieces, not ten letters: why models stumble when you ask them to count the R’s in “strawberry”.
Your question becomes 7 numbers. With a minimal set of instructions and the little markers that say who’s talking, the whole text is already 74 tokens. (ChatGPT’s real instructions are longer and not public.)
5. A room full of GPUs
The machine that writes the answer may be in another building, even another state, like the new AI campuses in Texas: eight buildings, about a gigawatt of power. Inside, there are rows and rows of racks. One rack holds 72 GPUs, chips built to do huge numbers of multiplications at the same time. Together, a rack draws about as much power as a hundred homes, so it’s cooled with liquid, not air.
Liquid-cooled racks: about 120 kW each, the power of roughly 100 homes.
Your question doesn’t get the chips to itself. It rides along with thousands of other conversations, and at every step, the machine works on all of them at once.
6. Inside the model
Nobody outside OpenAI can open ChatGPT’s model. But almost every chatbot today is built on the same architecture, called the Transformer. So in the video I opened a small public one, GPT-2 from 2019, and every number shown is real: I ran GPT-2 on our sentence.
A model is a long list of numbers, called weights, learned during training. GPT-2 has about 124 million of them (half a gigabyte), loaded into the GPU’s own memory before any answer.
Each token becomes an address. A token’s number picks one row in a table of about 50,000 rows, and that row is a list of 768 numbers: an address in a space of meaning. Training placed related words close together. Next to “France” sit “French”, “Germany”, “Spain”, “Italy” and “Europe”.
Attention lets words look at each other. To guess what comes after “is”, the last word has to look back at the words before it. It’s a bit like searching a library: each word makes a query, compares it with every earlier word’s key, and takes in a blend of their values. GPT-2 has 144 of these attention heads, each with its own job.
One of GPT-2’s 144 attention heads, measured on our sentence: every word looks at the word just before it.
Then each word thinks on its own, in a block of 3,072 detectors, each tuned to a pattern. For “is”, only about 3 in 100 fire strongly. This block holds two thirds of each layer’s weights, and researchers have found that facts, like the capital of a country, are largely stored there.
Attention, then the detectors: that’s one layer. GPT-2 stacks twelve of them, like a draft going through twelve editors. At the start, “Paris” is ranked around 28,000th among possible next words. Layer by layer, it climbs.
At the top, the model votes. The last word’s list is compared with every token GPT-2 knows, and the scores become percentages. Small GPT-2’s leaders are “the” at 16%, “Paris” at 11% and “France” at 6%. That’s not a confident answer, because this model is tiny. The largest GPT-2 already gives Paris 78%.
Decoding: instead of always taking the top word, chatbots spin a weighted wheel. Paris comes up about one time in nine.
Always taking the top word is called greedy decoding, and here it would write “the capital of France” and go round in circles. So chatbots usually spin a weighted wheel instead. A setting called temperature shapes that wheel: low, and the favourite becomes even more likely; high, and the text gets creative, then strange.
And here’s the most important idea of all: the model never writes the answer in one go. It only ever guesses the next token. That token is added to the text, and the whole model runs again: “Paris”, then a full stop, then a special token that means “end of answer”. Two tricks make this fast. The model keeps earlier words’ keys and values in memory (the KV cache), and it reads your whole prompt in a single pass.
7. The way back
Each token is sent back as soon as it’s chosen, back under the ocean, to your screen. That’s why the answer appears word by word. About one second, start to finish.
The receipt for one easy question.
- Distance: about 13,600 km, there and back, if the answer is written in Virginia
- To the edge server: 9 hops, about 2 ms (measured)
- Tokens in: 74. Tokens out: 7.
- Energy: about 0.34 Wh, OpenAI’s average per query
What’s measured, and what isn’t
OpenAI’s internals aren’t public. The network numbers come from my own connection in France. The model numbers were measured on GPT-2, an open model with the same design. The steps inside OpenAI come from what OpenAI has said publicly. Wherever something is likely or illustrative, the video says so.
If you’d like to see all of this move, the full video is here: https://youtu.be/GWjA5eHIXXk
My channel, The Next Token Lab, is where I’ll keep explaining how the tech we use every day actually works: https://www.youtube.com/@TheNextTokenLab
What should I explain next?
