Small businesses get the same ten questions all day: opening hours, delivery areas, how to book. Sending each one to a language model is wasted money. A small cache is one of the simplest ways to control AI automation costs for a small business.

Normalise, then hash 

import hashlib, re, sqlite3, time db = sqlite3.connect("answers.db") db.execute("""CREATE TABLE IF NOT EXISTS cache( key TEXT PRIMARY KEY, question TEXT, answer TEXT, created REAL, hits INTEGER DEFAULT 0)""") def normalise(q: str) -> str: q = q.lower() q = re.sub(r"[^a-z0-9 ]+", " ", q) return " ".join(q.split()) def key(q: str) -> str: return hashlib.sha256(normalise(q).encode()).hexdigest()

Read through the cache 

MAX_AGE = 7 * 24 * 3600 # refresh weekly so answers do not go stale def answer(q, call_model): k = key(q) row = db.execute("SELECT answer, created FROM cache WHERE key=?", (k,)).fetchone() if row and time.time() - row[1] < MAX_AGE: db.execute("UPDATE cache SET hits = hits + 1 WHERE key=?", (k,)) db.commit() return row[0], "cache" a = call_model(q) db.execute("REPLACE INTO cache(key, question, answer, created) VALUES (?,?,?,?)", (k, q, a, time.time())) db.commit() return a, "model"

Rules that keep it safe

  • Cache only general questions. Never cache answers that include a customer's name, order or account details.
  • Expire entries when prices, hours or policies change (clear the table on any website update).
  • Log cache versus model so you can see what the cache actually saves.

Going further

Exact matching catches more than you expect once text is normalised. Fuzzy or embedding matching adds complexity; measure first.

Why we often prefer small custom pieces like this over packaged bots: why small custom pieces often beat packaged bots.

Disclosure: I co-founded NxFlowAI, an automation agency.