I got a RAG app running on a 512 MB free server. The trick was not running the embedding model at all.

Most "build a RAG app" tutorials work fine in a notebook. Then you try to put one online on a free host, and it falls over.

I wanted a document Q&A app that answers from my own files, cites its sources, refuses when the answer isn't there, and runs entirely on free tiers with no credit card. Here's what broke along the way, and the setup that finally worked.

Live demo: https://rag-0.onrender.com (free tier, so the first request after it's been idle takes about a minute to wake up)
Code (free, 50 lines): https://github.com/Ryan140-cloud/zero-cost-rag

Problem 1: the embedding model doesn't fit

Render's free web service gives you 512 MB of RAM and 0.1 CPU. The usual tutorial setup embeds text with sentence-transformers, which pulls in PyTorch. PyTorch plus a model is a lot to fit in 512 MB, and on a host that small it typically either runs out of memory or crawls.

The fix: don't run an embedding model on your server at all.

Qdrant Cloud has a free cluster, and its Cloud Inference feature embeds text on Qdrant's side. sentence-transformers/all-minilm-l6-v2 (384 dimensions) is listed as free. Your server just sends plain text:

from qdrant_client import QdrantClient, models client = QdrantClient(url=QDRANT_URL, api_key=QDRANT_API_KEY, cloud_inference=True) # Qdrant embeds, not your server MODEL = "sentence-transformers/all-minilm-l6-v2" # storing a chunk: send text, not a vector client.upsert("docs", points=[models.PointStruct( id=chunk_id, vector=models.Document(text=chunk_text, model=MODEL), payload={"text": chunk_text, "source": "guide.pdf"})]) # searching: send the question as text too hits = client.query_points("docs", query=models.Document(text=question, model=MODEL), limit=3, with_payload=True).points 

With no PyTorch in the dependencies, my full Flask app used about 141 MB of RAM when I measured it locally. Plenty of headroom.

One catch: LangChain's QdrantVectorStore expects a local embeddings object, which defeats the point. So I wrote a ~25-line custom LangChain retriever that passes raw text to query_points instead.

Problem 2: the Groq model in every tutorial is gone

Groq is the go-to free LLM API for these projects, and most tutorials hard-code llama-3.1-8b-instant. Groq shut that model down for free and developer-tier accounts on August 16, 2026. Their recommended replacement is openai/gpt-oss-20b. Old code now just returns 404 model_not_found.

Two lessons:

1. Never hard-code the model name. Read it from an environment variable, so the next retirement is a config change, not a code change.
2. gpt-oss is a reasoning model. It "thinks" before answering, and those thinking tokens count toward max_tokens. A low cap like 512 can give you cut-off or empty answers. What worked for me:

from langchain_groq import ChatGroq llm = ChatGroq( model=os.getenv("GROQ_MODEL", "openai/gpt-oss-20b"), temperature=0, reasoning_effort="low", # faster and cheaper; plenty for answering from context max_tokens=1024, # thinking tokens count here too max_retries=4, # backs off and retries on HTTP 429 ) 

Problem 3: the limit that actually runs out

Groq's free tier for openai/gpt-oss-20b (check their rate-limits page, these change):

In RAG, every question carries the retrieved chunks with it. In my real test runs (3 chunks of ~800 characters, short answers), one question cost about 600–700 tokens. That's roughly 280 questions a day before the daily token cap, well before the 1,000-request cap.

That's plenty for a portfolio project or demo. It's not enough for real public traffic, and it's worth being honest about that.

Problem 4: the two sleep timers

Free apps rarely die from traffic. They die from inactivity:

The Render one is annoying. The Qdrant one is dangerous: a portfolio link you shared a month ago can quietly lose all its data.

My fix is a free weekly GitHub Actions job that hits a "deep" health endpoint. That endpoint does one cheap Qdrant read (a point count) and never calls the LLM, so it can't burn your token budget:

on: schedule: - cron: "17 6 * * 1" # Mondays jobs: ping: runs-on: ubuntu-latest steps: - run: curl --fail --max-time 120 --retry 3 "${{ secrets.APP_URL }}/health?deep=1" 

Problem 5: the bug my own first deploy hit

When I pasted my Qdrant API key into Render's environment settings, a trailing newline came along with it. Every Qdrant call failed, and the error message printed the full key into my logs. I had to rotate it.

Now every setting is .strip()ped when it's read:

def env(name, default=""): return os.getenv(name, default).strip() 

Grounding: make it refuse

The prompt tells the model to answer only from the retrieved context, or reply with one exact sentence:

Answer ONLY from the context below. If the context does not contain the answer, reply exactly: "I can't answer that from the provided documents." 

Because the refusal is one exact string, you can test it. I keep 10 questions (6 my documents answer, 4 they don't, including a prompt-injection attempt) and check every answer automatically before deploying.

The stack, summarized

• LLM: Groq openai/gpt-oss-20b (free tier)
• Embeddings + vectors: Qdrant Cloud free cluster with Cloud Inference
• Glue: LangChain 1.x (langchain-core, langchain-groq)
• API + hosting: Flask + Gunicorn on Render's free web service
• Cost: $0, no credit card anywhere

The 50-line version is on GitHub: https://github.com/Ryan140-cloud/zero-cost-rag

Disclosure: I also turned the full build (Flask API, Render deploy, keep-alive, tests and evaluation) into a paid guide and starter kit. If you'd rather not piece it together yourself, it's here. There's also a free 3-page cheat sheet of all the limits above.

Free tiers change. Everything here was checked in November 2026.