Run AI on Your Own Laptop: Free, Private and Offline with Ollama

What if you could run an AI model on your own laptop? No API key, no monthly bill, and your prompts never leave your machine. With a free tool called Ollama, it takes two commands. After that, you can call the model from your own code like any other web API.

Everything below was run for real on a Mac (Ollama 0.40, llama3.2:3b).

Video version:

Step 1: install Ollama

brew install ollama # macOS (or download the desktop app) ollama serve # starts the local server # Listening on 127.0.0.1:11434 

Linux and Windows installers are on ollama.com. The desktop app starts the server for you, so you can skip ollama serve if you use it.

Step 2: run a model

ollama run llama3.2:3b "Explain recursion in one sentence." 

Recursion is a programming concept where a function calls itself repeatedly until it reaches a base case that stops the recursion… 

The first run downloads the model, about 2 GB for Llama 3.2 with 3 billion parameters. After that, answers take a few seconds. On my Mac it generates about 44–48 tokens per second, completely offline.

Call it from JavaScript

Ollama is just a local HTTP API, so you don't need an SDK or a key. Send a POST to /api/chat:

// chat.mjs (Node 18+ has fetch built in) const res = await fetch("http://localhost:11434/api/chat", { method: "POST", body: JSON.stringify({ model: "llama3.2:3b", messages: [{ role: "user", content: "One tip for naming variables." }], stream: false, }), }); const data = await res.json(); console.log(data.message.content); 

My run answered: use meaningful, descriptive names. Instead of x, use something like temperatureInCelsius. Good advice from a 2 GB model.

Set stream: true (the default) if you want the reply token by token, like a chat UI. You then read the response as newline-delimited JSON.

Bonus: it speaks the OpenAI API format

Ollama also exposes an OpenAI-compatible endpoint at /v1/chat/completions. Here it is with nothing but Python's standard library:

import json, urllib.request req = urllib.request.Request( "http://localhost:11434/v1/chat/completions", data=json.dumps({"model": "llama3.2:3b", "messages": [{"role": "user", "content": "What is 2+2?"}]}).encode(), headers={"Content-Type": "application/json"}) reply = json.load(urllib.request.urlopen(req)) print(reply["choices"][0]["message"]["content"]) # 2 + 2 = 4 

This means code and tools written for the OpenAI format can often switch to your local model just by changing the base URL. That's handy for prototypes, tests and CI runs where you don't want to pay for tokens.

Local AI vs cloud AI

✅ Local is great for:

• Cost: no API bill and no rate limits
• Privacy: sensitive data never leaves your machine
• Offline work, experiments and learning

❌ The trade-offs:

• Small local models are less capable than the top cloud models, especially at complex reasoning and long code changes.
• Memory: small models need about 8 GB of RAM; big ones need far more.

Many teams use both: a local model for cheap, simple tasks (classification, summaries, test data) and a cloud model for the hard ones.

Cheat sheet

What would you build with a free, private AI? 👇

I post one tested coding tip every day on YouTube at Refactorly. The 30-second version of this one is here.