Your Free Server Forgets Everything: Five Ephemeral-Deployment Myths and a State Probe

I keep reading the same three lines in deploy channels. "It worked yesterday." "The file was there." "The model knew my name." None of those are evidence of memory. They are evidence of a lucky restart window.

Free compute and free model endpoints are usually shared and disposable. That is the deal. The trap is treating them as stateful by default. So here is a myth-busting FAQ, plus a probe you can run in five minutes.

Myth 1: "The disk survives a restart"

Sometimes it does. Sometimes you land on the same machine by chance. Your test is confirming nothing.

Start with a repeatable probe. It writes one small JSON file, prints the previous contents, and exits.

# state_probe.py — stdlib only, safe to run anywhere import json, os, socket, time, uuid STATE = os.environ.get("PROBE_STATE", "/tmp/probe/state.json") BOOT = "/proc/sys/kernel/random/boot_id" def read(): try: with open(STATE) as f: return json.load(f) except FileNotFoundError: return None except json.JSONDecodeError: return {"corrupt": True} def write(rec): os.makedirs(os.path.dirname(STATE), exist_ok=True) tmp = f"{STATE}.tmp" with open(tmp, "w") as f: json.dump(rec, f, sort_keys=True) os.replace(tmp, STATE) # atomic on POSIX  def boot_id(): try: with open(BOOT) as f: return f.read().strip() except OSError: return None previous = read() current = { "run_id": str(uuid.uuid4()), "seen_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()), "host": socket.gethostname(), "boot_id": boot_id(), } write(current) print(json.dumps({"previous": previous, "current": current}, indent=2)) 

Run it twice. Then trigger a real restart, redeploy, or cold start. Run it again. Compare.

How to read the output

That third row is the interesting one. Even when state persists, you still need write-then-read verification. os.replace is atomic on POSIX filesystems, which is why the probe uses it instead of writing in place (Python docs: https://docs.python.org/3/library/os.html#os.replace).

Myth 2: "A sleeping instance keeps its memory"

Idle instances get suspended, rescheduled, or rebuilt. In-process dictionaries vanish. Global counters reset to zero.

Ask yourself one question. If this process dies mid-run, what do I need to finish the job? Whatever the answer is, it cannot live in RAM.

That is why attempt counters, cursors, and "already sent" markers belong in an external store. Not in a module-level variable. Not in memory inside the worker.

Myth 3: "The model endpoint remembers our last call"

It usually does not. Chat endpoints are request-scoped. Context you saw earlier was resent by your client, not stored by the server.

Here is a cheap way to check. Send two calls and see if the second one can echo something from the first.

# Shape only. Field names depend on your endpoint; check its docs. curl -sS "$MODEL_URL" -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \ -d '{"messages":[{"role":"user","content":"My run id is 7f3a."}]}' curl -sS "$MODEL_URL" -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \ -d '{"messages":[{"role":"user","content":"What is my run id?"}]}' 

I have not run this against your endpoint. Treat it as a shape, not a result. But if the second answer guesses, it guessed. It was not recalled.

Corrected mental model: the transcript is yours. You own trimming, resending, and storing it. That is also your chance to cut tokens.

Myth 4: "The logs are my audit trail"

Logs are for debugging. Retention windows are short and search is lossy. Platform logs also disappear with the service.

If you need a record, write the record. One immutable line per attempt, containing run id, input hash, decision, and timestamp. Ship it somewhere you control.

Do not confuse volume with proof. A thousand log lines prove you were busy. One verified checkpoint proves where you stopped.

Myth 5: "Retries after a cold start are free"

They are not. A cold start costs latency. A blind retry costs duplicated work and duplicated spend.

So gate retries on a checkpoint, not on hope. Pull state first, then decide.

# checkpoint.py — resume instead of restart import json, urllib.request def pull(url, token): req = urllib.request.Request(url, headers={"Authorization": f"Bearer {token}"}) with urllib.request.urlopen(req, timeout=10) as r: return json.load(r) def push(url, token, payload): body = json.dumps(payload).encode() req = urllib.request.Request( url, data=body, method="PUT", headers={"Authorization": f"Bearer {token}", "Content-Type": "application/json"}) with urllib.request.urlopen(req, timeout=10) as r: return r.status state = pull(STATE_URL, TOKEN) or {"done": []} step = next((s for s in STEPS if s not in state["done"]), None) if step is None: print("nothing to do") else: result = run(step) # your work  state["done"].append(step) print(push(STATE_URL, TOKEN, state)) # 200 or 2xx = durable  print(result) 

Note the ordering. Work, then push, then confirm the status code. If the push fails, the step is unfinished, and the next run repeats it on purpose.

The corrected mental model

Compute is disposable. State is rented. Design as if the process will die every few minutes, because one day it will.

Where a free tier actually helps

This is the part where a free server option is genuinely useful. Not as production. As a rehearsal space for the probe.

I used MonkeyCode's free server option and free model access to rehearse this loop: run the probe, restart, compare output, then push a checkpoint. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The two supplied facts I relied on are simple. There is free model access, and there is a free server option. Everything else in this post is yours to verify. I did not test quotas, hardware, uptime, or permanence, and I would not repeat secondhand numbers about them.

Why does the free tier fit? Because the probe is cheap and the lesson is expensive. Learn it on throwaway compute, not on the box that runs your billing job.

Limitations and who should skip this

• The probe shows one account, one region, one moment. It is not a guarantee.
• Free tiers change. Re-run the probe monthly, not once forever.
• Do not put personal data in the probe file. It is a marker, not a dataset.
• This is not a compliance control. Auditors want retention contracts, not JSON files.
• Skip this if you need durable queues with delivery SLAs, or sub-100ms steady latency.
• Skip it if your team has no external store at all. Fix that first.

A five-minute checklist

1. Drop state_probe.py in your service and run it twice.
2. Force a real restart, redeploy, or scale-to-zero event.
3. Record host and boot id before and after.
4. Move every counter and cursor out of memory.
5. Re-read after each write, then log the status code.
6. Repeat the probe next month and diff the results.

Does your free server actually remember anything? Run the probe before you answer. If you want a sandbox to try it, MonkeyCode's free server option is one place to start, though the probe is the point.