this is my second progress update for the Decentralize AI Hackathon by HackerNoon and Nosana.
In my first article, I Built a Local AI Linux Assistant That Doesn't Rely on the Cloud, I introduced zkzkAgent: an AI agent that manages a Linux machine through plain language and runs entirely on my own hardware with Ollama.
That is still how it works by default. But running everything locally has a ceiling, and I kept hitting it.
The Problem: The Model That Fits on a Laptop
- zkzkAgent doesn't push every request through one big prompt. A small classifier node first decides what kind of request it is: a question to answer, a single action to run, or a task that needs a plan. Plans are where the agent is most useful, and they are also where the model works hardest. To scaffold a project, the planner writes every step, including the full content of every file, before anything runs.
- On my laptop, I run a 2B model (
qwen3.5:2b). It fits in memory and answers quickly. A 27B model does not fit. Until now, anyone on modest hardware had two options: accept a small model, or send everything to a centralized AI provider. - Nosana adds a third option: rent a GPU from a decentralized network, only for the model, and keep everything else at home.
zkzkAgent in 30 Seconds
- A multi-node LangGraph graph. Separate nodes classify the request, answer questions, write plans, and execute tools. The router logs every decision with the reason for it.
- 25 tools for files, apps, networking, processes, packages, and deployment.
- Human-in-the-loop safety. Destructive tools (removing files, emptying the trash, installing or removing packages) are blocked in code until I type
yes. - A chat CLI with live streaming, history and slash commands, plus Docker support and an optional voice mode.
The Design Rule: Only the Model Moves
- Before writing any code, I set one rule for the integration: Nosana gets the model calls and nothing else. The graph, the routing, the confirmation gates, and all 25 tools still run on my machine. A model on a remote GPU can suggest
remove_file. Only my machine can run it, and only after I typeyes. - That rule kept the change small. It lives in one file,
models/LLM.py, and it is controlled by one setting in `.env`:
MODEL_TYPE="NOSANA" NOSANA_API_KEY="nos_..." NOSANA_BASE_URL="https://inference.nosana.com/v1" NOSANA_MODEL="qwen/qwen3.8-27b" - Nosana's inference API uses the OpenAI chat format, so the agent talks to it through LangChain's `ChatOpenAI` client. The rest of the graph only sees an `llm` object and doesn't know which backend is behind it:
if MODEL_TYPE == "NOSANA": API_KEY = _env("NOSANA_API_KEY") if not API_KEY.startswith("nos_"): raise RuntimeError("NOSANA_API_KEY missing or invalid (must start with 'nos_').") BASE_URL = _env("NOSANA_BASE_URL", NOSANA_URL).rstrip("/") MODEL_NAME = _env("NOSANA_MODEL") or _pick_nosana_model(BASE_URL, API_KEY) llm = ChatOpenAI( model=MODEL_NAME, base_url=BASE_URL, api_key=API_KEY, timeout=120, max_retries=2, stream_usage=True, # final chunk carries token counts ) elif MODEL_TYPE == "OLLAMA": pass #the other implement of the code Three details are worth explaining:
- Picking a live model. On Nosana, a model only appears in
GET /modelswhile a GPU is actually serving it. If `NOSANA_MODEL` is empty, the agent fetches that list and uses the first available model, so it never starts against a model that nobody is serving. - Token counts. With
stream_usage=True, the last streamed chunk includes token usage. That is what usage is billed on, and it is the basis for the cost numbers I plan to publish next. - Bring your own GPU job. The repo also includes
nosana-ollama.json, a Nosana job definition that starts the official Ollama container on a GPU node and pullsqwen3:14b. It is for people who want their own model server on Nosana instead of the shared API.
Case Study: One Request, One Working Web App
To test the integration in real work, I switched to the 27B model on Nosana and gave zkzkAgent a task: create a new project called zkzk_ai, a calculator web app with a FastAPI backend and an HTML, CSS, and JavaScript frontend, and open it in VS Code.
Step 1: Route and Plan
The classifier sent the request to the planner (`NEEDS_PLANNING`). The planner wrote a 9-step plan, including the full content of every file, and then stopped and waited for me:
Reply 'ok' (or 'yes') to run this plan, or tell me what to change. Enter your request: ok INFO:core.agent:[ENTRY ROUTER] Pending plan approved → routing directly to execute Step 2: Execute
Execution: 10 Tool Calls, One Step at a Time
Once I approved the plan, the executor got to work. It made 10 tool calls, one at a time, across 10 graph iterations (with a maximum of 15).
Here's what it actually did:
| # | Tool | Action |
|---|---|---|
| 1 |
| Created |
| 2 |
| Wrote |
| 3 |
| Created the |
| 4 |
| Wrote |
| 5 |
| Wrote |
| 6 |
| Wrote |
| 7 |
| Wrote |
| 8 |
| Wrote |
| 9 |
| Wrote |
| 10 |
| Opened the project in VS Code |
Nothing complicated happens behind the scenes. The model running on Nosana picks a tool; that tool executes on my machine, and the result is sent back to the model.
- Then it looks at the result and decides what to do next.
- It repeats this cycle until the task is done or it hits the iteration limit.
To make this clearer, here's an actual example from the execution log (trimmed for readability):
Each call follows the same loop. The model on Nosana picks the next tool, the tool runs on my machine, and the result goes back into the conversation. In the log, one turn of that loop looks like this (trimmed):
INFO:httpx:HTTP Request: POST https://inference.nosana.com/v1/chat/completions "HTTP/1.1 200 OK" Tools in last message: [{'name': 'create_project_folder', 'args': {'working_directory': '/home/aloha-zkaria', 'project_name': 'zkzk_ai'}}] iteration: 0/15 INFO: tools_module.files_tools.createProjectFolder. TOOL: Successfully created project folder /home/aloha-zkaria/zkzk_ai Step 3: A Follow-Up Edit
Next, I asked: "Can you change the zkzk_ai Calculator to be zkzk?" The classifier sent this to the planner as well, which wrote a 7-step plan. After I approved it, the agent read static/index.html and then changed the name in three places: the page title and heading, the FastAPI app title, and the README. It did not touch the calculator logic.
Step 4: Does It Actually Run?
I started the app with `uvicorn app:app`. The page loaded, and the calculator API answered:
INFO: 127.0.0.1:52166 - "GET / HTTP/1.1" 200 OK INFO: 127.0.0.1:52166 - "GET /static/style.css HTTP/1.1" 200 OK INFO: 127.0.0.1:52172 - "GET /static/script.js HTTP/1.1" 200 OK INFO: 127.0.0.1:58438 - "POST /api/calculate HTTP/1.1" 200 OK 
Across both requests, every model call in the saved log, 18 in total, went to `inference.nosana.com` and returned `200 OK`. Every file write, command, and app launch happened locally.
What the Generated Code Got Right
I reviewed the code the agent wrote, and tested the backend directly:
- No eval. Expressions are parsed with Python's ast module, and only numbers, parentheses, and `+ - * / // % **` are allowed. When I sent __import__("os"), it was rejected with "Unsupported expression element: Call".
- Clear errors. Division by zero and complex results (such as (-1)**0.5) return a readable message instead of crashing.
- Input limits. Expressions are capped at 500 characters in both the browser and the API.
- Keyboard support. Enter evaluates, Backspace deletes, Esc clears and `^` types a power.
The code is on GitHub at [zkzkGamal/calculator], and the full run log is in the [zkzkAgent repo]
Local-First, Not Local-Only
My first article was about an agent that doesn't rely on the cloud. This update doesn't change that. It adds a choice. Private or sensitive work stays on local Ollama, where nothing leaves the machine. Heavy planning can use a much larger model on Nosana's network instead of a centralized provider. In both modes, the tools, the files, and the final yes stay with me.
Next Steps
1. A real benchmark. Run the same five tasks on the local 2B model and on the 27B model on Nosana, and measure success rate, tool calls, time, tokens, and cost. Numbers instead of impressions.
2. Hybrid routing. Because the graph has separate nodes, each node can use its own model. The classifier and chat replies are short and could stay on a small local model, while the planner and executor use Nosana. Plain conversations would then never leave the machine.
3. Automatic fallback. If Nosana can't be reached, switch to local Ollama automatically, the same way the agent already turns Wi-Fi back on by itself.
4. Close the safety gaps. Enforce plan-level confirmations. Also put `run_command` behind a confirmation or an allow-list: today it can run any shell command, and with a remote model choosing the arguments, that matters more than before.
5. Fix the calculator bug with the agent doing the fix.
How to use zkzkAgent
git clone https://github.com/zkzkGamal/zkzkAgent.git cd zkzkAgent pip install -r requirements.txt cp .env.example .env # set MODEL_TYPE="NOSANA" and your NOSANA_API_KEY ./cli.sh Full links:
zkzkAgent: https://github.com/zkzkGamal/zkzkAgent The calculator it built: https://github.com/zkzkGamal/calculator
Full Nosana run log: https://github.com/zkzkGamal/zkzkAgent/blob/main/docs/nosana-run-2026-10-25.log
First article: https://hackernoon.com/i-built-a-local-ai-linux-assistant-that-doesnt-rely-on-the-cloud
