Every team wants AI. Then three questions arrive: What does it cost? Where are the keys? Can we stop it?

I'm a platform engineer, and I built SanVi to answer all three on Kubernetes.

The gateway

Apps call one OpenAI-compatible endpoint (LiteLLM engine) with their own virtual key. Each key has its own models, budget and rate limit. No key → 401. Over the rate limit → 429. Over budget → blocked. Every call is logged with team, model, tokens and cost in Postgres.

Keys never in code

The provider key lives only in Vault and syncs in through External Secrets. I rotated it: about 70 seconds, no redeploy.

The kill switch

Leaked key, runaway agent loop, spend spike: kill_switch.sh block <team> returns 401 in seconds, and unblock restores it. Nothing is deleted.

AI access as code

Teams, models and budgets live in access/teams.yaml. Changes go by pull request; CI checks policy (a greedy $50 budget was blocked); Argo CD applies on merge.

Evals decide, not opinions

The Right-Size Advisor runs the same eval suite on each model. On my workload Haiku matched Sonnet: 95% cheaper, 40% faster. Any change that drops below 80% fails CI.

Human in the loop

The deployment agent turns one sentence into a validated Dockerfile and Helm chart, but it can only open a pull request.

Audit-ready

A compliance script checks 12 controls (no secrets in Git, least-privilege access, audit trail, kill switch, quality gate...) mapped to ISO 27001, SOC 2 and EU AI Act themes, on every change.

Try it

Repo, Quickstart and Helm chart: https://github.com/sandhilk/sanvi-gateway

It's a personal project, so blunt feedback is welcome. What would you need before running this in your company?