Originally published on wrightfunctions.com, where the demos are live.

Every team using AI ends up asking this. You can pay per token to use someone else's model through an API. Or you can download an open-weights model and run it yourself, on GPUs you rent or buy.

Either way, a question goes in and an answer comes out. Which is cheaper, faster or safer?

The answer is "it depends", and this post is about what it depends on. Prices are as of September 2026 and they move fast. Treat the numbers as a snapshot and the reasoning as the point.

The cost: a meter or a lease

An API is a meter. Anthropic charges $2 per million input tokens and $10 per million output for Claude Sonnet 5. Hosted open models are much cheaper: Together AI serves Meta's Llama 3.3 70B for $1.04 per million, in or out. You pay for exactly what you use, and nothing when you don't.

Self-hosting is a lease. Lambda rents H100s from $4.19 an hour each in a two-GPU machine, and a 70-billion-weight model at 16 bits needs two of them just to hold its 140 GB of weights. Leave them on all month and that's about $6,100, whether they answer ten questions or ten million. So what decides the cost per token is GPU utilization: a GPU earns its keep only while it's busy.

▶ Interactive demo: try it on the original post

That break-even point is sobering. Hosted open-model prices are so low that two rented GPUs have to stay busy around the clock to compete. Big, steady workloads can get there, especially with heavy batching and quantization. Many teams can't, and they'd be paying for idle GPUs overnight.

Buying the box

Owning changes the math: you pay once, plus power. NVIDIA's RTX 5090 launched at $1,999 with 32 GB of memory, enough for a 30-billion-weight model at 4 bits. At its full 575 watts around the clock, and the US average home rate of about 18 cents per kWh in July 2026 (EIA), the card alone would burn roughly $77 a month in electricity.

For bigger models, Apple's Mac Studio now takes up to 512 GB of unified memory, and NVIDIA's DGX Spark packs 128 GB into a desktop box. None of these match a data-center GPU's throughput, but for one person or a small team, they don't have to.

Nobody else's queue

An API's speed is out of your hands. You share the provider's servers, and every account has a rate limit that rises as you use the service more (Anthropic's tiers are one example). Your own GPU has no rate limit, only a capacity limit, and at low load it can be very quick: in NVIDIA's benchmarks, a small model on one H100 has a time to first token of about 10 milliseconds for a short prompt with a single user. Load it up and each user waits longer, which is the latency-versus-throughput trade that runs through all of serving.

It never left the building

This is often the real reason teams self-host. With your own hardware, prompts never leave your network, which may be the only acceptable answer for some data (the government cloud post is all about that).

APIs have closed some of the gap. OpenAI says API data isn't used to train its models by default, and Anthropic says the same for its commercial products. Some providers also offer zero data retention on request. But "they promised" and "it never left the building" aren't the same thing to an auditor.

The best models aren't for sale

As of September 2026, the top of the LMArena text leaderboard is all closed models: their weights aren't published, so you can only reach them through someone else's service. Open models have closed much of the distance. OpenAI's own open-weights gpt-oss-120b runs on a single 80 GB GPU under the Apache 2.0 license.

For well-defined tasks like classification, extraction and summarizing your own documents, an open model is often enough. For the hardest reasoning, the frontier APIs usually still lead. Either way, test both on a few hundred of your own real questions before deciding. A leaderboard can't tell you that.

Who gets paged?

Price comparisons leave out one line: with an API, when the model is down, the provider's engineers are on it. With your own GPUs, it's you. Drivers, inference-server upgrades, security patches, new model versions, failed hardware and capacity planning are all yours now. That's a real cost, and it's paid in engineers' time rather than in the hourly GPU price.

So, rent or run?

Rent by default. Per-token pricing is cheap, elastic and someone else's pager. Run your own when your data can't leave, when your volume is big and steady enough to keep GPUs busy, or when you need control over the exact model and its latency. Start on the meter; sign the lease when the meter runs all night.

Many teams end up with both: an API for the hard questions and a small self-hosted model for the steady, sensitive, repetitive work. Whichever you pick, re-run the numbers every few months, because they won't hold still.

References & further reading

  1. Claude pricing (Anthropic · as of September 2026). Claude Sonnet 5 at $2 per million input tokens and $10 per million output.
  2. Together AI pricing (Together AI · as of September 2026). Llama 3.3 70B at $1.04 per million tokens, input or output.
  3. Lambda pricing (Lambda · as of September 2026). On-demand H100 SXM at $4.19 per GPU-hour in a two-GPU instance ($3.99 in an eight-GPU one), plus tax.
  4. NVIDIA Blackwell GeForce RTX 50 Series Opens New World of AI Computer Graphics (NVIDIA · January 2025). The RTX 5090 at $1,999, available January 30, 2025.
  5. GeForce RTX 5090 (NVIDIA · specs). 32 GB of GDDR7 and 575 W total graphics power.
  6. Electric Power Monthly, Table 5.6.A: Average price of electricity to ultimate customers (US Energy Information Administration · preliminary, released September 2026). US average residential price of 18.31 cents per kWh in July 2026.
  7. Mac Studio technical specifications (Apple · as of September 2026). M5 Ultra with up to 512 GB of unified memory at 1.2 TB/s.
  8. NVIDIA DGX Spark (NVIDIA). 128 GB of unified memory in a desktop box, for models up to 200 billion parameters.
  9. Rate limits (Anthropic · docs). Usage tiers that rise automatically with usage history and account standing.
  10. NVIDIA NIM for LLMs performance benchmarks (NVIDIA · docs). With 200-token prompts, Llama 3.1 8B on one H100 at about 10 ms to first token with one user, and much higher total throughput, with slower answers, at 250.
  11. Your data (OpenAI · docs). API data isn't used to train OpenAI's models unless you opt in. Zero data retention needs prior approval.
  12. Is my data used for model training? (Anthropic Privacy Center). Inputs and outputs from Anthropic's commercial products aren't used for training by default.
  13. Text Arena leaderboard (LMArena · as of September 2026). Every model in the top 16 is proprietary; the best open-weights model ranks 17th.
  14. Introducing gpt-oss (OpenAI · August 5, 2025). Open-weights models under Apache 2.0; the 120B one runs on a single 80 GB GPU.

AI-assisted: I write these with help from AI tools.