Audit Every Endpoint Your Retry Chain Can Reach

Your prompt does not go to one endpoint. It goes to every endpoint your retry policy, SDK defaults, and fallback configuration can resolve to, and most teams have classified data for exactly one of them. The endpoint you reviewed in the design doc is the happy path, while the ones that actually receive your payload at 3am are the fallback nobody owns. This post builds a client-side gate that classifies fields instead of prompts, so the decision survives a base-URL swap you never reviewed.

Start with the fan-out, not the prompt

Before you write a single policy rule, list the routes. A route is a triple of base URL, credential, and retry behaviour, and you can usually find most of them with a broad grep across the repo.

rg -n --no-heading -g '!node_modules' -g '!vendor' \ -e 'base_url' -e 'api_base' -e 'OPENAI_BASE_URL' -e 'ANTHROPIC_BASE_URL' \ -e 'fallback' -e 'retry' -e 'preview' 

That grep finds the honest routes, the ones written down in a config file or a constructor call. The dishonest ones live in environment variables and in SDK defaults that quietly read a *_BASE_URL value when your config is empty. That is why the second command prints variable names and deliberately never prints their values.

python - <<'PY' import os print(sorted(k for k in os.environ if k.endswith(("_URL", "_HOST", "_BASE")))) PY 

Write the result into a config file as named routes, not as raw URLs. Naming them is what lets policy refer to a route class instead of a hostname, and it means adding a hostname cannot silently widen what your service is allowed to send.

Classify fields, then gate the transport

A prompt is a string, but a request is a set of fields, and fields are what you can actually reason about. Tag each field with a data class at the call site, and keep the classification next to the value so review sees both together.

# egress/policy.py from dataclasses import dataclass from enum import Enum class DataClass(str, Enum): PUBLIC = "public" INTERNAL = "internal" CONFIDENTIAL = "confidential" SECRET = "secret" class EgressBlocked(RuntimeError): """Raised before serialization; the payload never reaches the transport.""" ALLOWED = { "local_dev": {DataClass.PUBLIC, DataClass.INTERNAL, DataClass.CONFIDENTIAL}, "free_hosted": {DataClass.PUBLIC}, "vendor_paid": {DataClass.PUBLIC, DataClass.INTERNAL, DataClass.CONFIDENTIAL}, } @dataclass(frozen=True) class Field: name: str value: str cls: DataClass def gate(endpoint_class: str, fields: list[Field]) -> dict[str, str]: allowed = ALLOWED[endpoint_class] blocked = [f.name for f in fields if f.cls is DataClass.SECRET or f.cls not in allowed] if blocked: # Field names and classes only. The value never enters the error path.  raise EgressBlocked( f"endpoint_class={endpoint_class} " f"blocked_fields={blocked} " f"classes={sorted({f.cls.value for f in fields})}" ) return {f.name: f.value for f in fields} 

Notice that SECRET is blocked for every route class, including the local development box. A credential written into a local request still lands in a debug log, a crash report, or a screenshot, so locality buys you less than it feels like it should. Route classes come from your config, which means the gate can be tested without mocking a single HTTP call.

# egress/client.py def send(client, endpoint_class: str, prompt: str, schema: str): payload = gate(endpoint_class, [ Field("prompt", prompt, DataClass.PUBLIC), Field("schema", schema, DataClass.INTERNAL), ]) return client.complete(payload["prompt"], schema_text=payload["schema"]) 

A canary that proves the gate holds

Policy files rot quietly, so pair the gate with a canary value that is syntactically real and functionally dead. The canary below is a valid-looking connection string that cannot connect anywhere, which makes it safe to commit and easy to search for later.

# tests/test_egress.py import pytest from egress.policy import DataClass, EgressBlocked, Field, gate CANARY = "postgres://canary:canary@db.internal:5432/app" def test_canary_secret_blocked_on_every_route(route_classes): for endpoint_class in route_classes: with pytest.raises(EgressBlocked) as err: gate(endpoint_class, [ Field("prompt", "summarize the incident", DataClass.PUBLIC), Field("db_url", CANARY, DataClass.SECRET), ]) assert "db_url" in str(err.value) assert CANARY not in str(err.value) 

Run it with pytest -q tests/test_egress.py -k canary. The second assertion matters more than the first, because it checks that your own defence does not become the leak by quoting the blocked value into a log. If the canary ever shows up in a provider dashboard or a log aggregator, you have a detectable leak instead of a silent one.

Make a new endpoint fail the build

Route classes are only useful if an unknown route is loud. Add a coverage test that resolves the configured routes and refuses anything outside the reviewed set.

def test_no_unclassified_endpoint_in_config(route_classes): known = {"local_dev", "free_hosted", "vendor_paid"} assert set(route_classes) <= known 

Now a teammate who adds a fallback host without adding a class breaks CI, and the failure message names the route instead of dumping a stack trace at review time. That single test is what turns a policy document into a control you can point at during an audit conversation.

Why the free tier makes this worse, and where MonkeyCode fits

Free model access inverts the usual cost signal. When an extra request costs nothing, the cheapest engineering move is to send more context rather than less, and that instinct is exactly backwards for data you have not classified. MonkeyCode's free model access, which the operator describes as a free allowance of 10 million tokens, plus a free server option both lower the cost of standing up a throwaway sandbox route. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Use that kind of sandbox as the route you attack first. Point it at a payload stuffed with canary secrets, run the test suite, and confirm the gate raises before serialization rather than after the socket opens. The gate itself never learns which provider it is protecting, which is the point, because you can swap routes and keep the same policy file.

A decision table you can argue about

Argue about the rows, not the code. A four-row table is far easier to defend in review than an unwritten assumption that the free endpoint is "just for testing".

What not to send, plainly

Connection strings, bearer tokens, and anything that looks like a key never leave, no matter which class the route claims to be. Raw stack traces are next on the list, because they carry DSNs, hostnames, and absolute paths containing usernames without ever announcing themselves as sensitive. Customer text needs a lawful basis and a retention story before it needs a model, and internal hostnames are reconnaissance material when combined with a version string. If you cannot say which class a field belongs to, treat it as confidential and let someone argue it upward deliberately.

Limitations, and who should skip this

The gate runs in-process, so a notebook, a curl invocation, or a colleague's local script bypasses it completely; pair it with network egress rules so the process boundary is not your only boundary. Field classification is manual and drifts as schemas evolve, and the field scanner adds a small per-request cost that matters on a hot path. Skip this approach if the provider contract, rather than your code, controls what data may be sent, because a passing test is not a data processing agreement. Skip it too if your debugging workflow depends on raw payload logs, since you cannot have both, and pretending otherwise just moves the decision to whoever reads the logs. Teams without an inventory of routes should do the grep first and write policy second.

If you want an endpoint that costs nothing to break on purpose, the free model access and free server option are a reasonable place to start.