Know Before You Cut Over: Building a Migration Readiness Checker

Turning platform migration guidance into an automated risk report

The problem

Many organisations run their applications on an internal developer platform built on top of a public cloud and Kubernetes. The platform gives product teams a paved road: clusters, networking, ingress, monitoring, and security scanning out of the box.

Sooner or later, that platform needs a new generation. When it arrives, every product has to move, and the old platform gets a fixed shutdown date. Teams have to migrate while still shipping features.

A typical next-generation platform changes how teams own, configure, and secure their workloads:

The platform team usually publishes a good migration guide. Teams read it. Then cutovers start, and things break anyway.

Why cutovers fail

The failures are rarely exotic. They are the same small problems, over and over:

• Ingress rules stop working. An ingress relies on an nginx-specific annotation, such as a URL rewrite. Traefik ignores it without complaint, and users start getting 404s.
• Outgoing calls get blocked. A service calls a payment provider or an external API. The new per-namespace egress policy blocks it because nobody listed that destination.
• Pods get evicted. Containers without resource limits are scheduled unpredictably on the new cluster and are first in line when nodes come under pressure.
• DNS names don't resolve. Hardcoded hostnames point at the old platform.
• Retries make things worse. Clients with no backoff turn a brief dependency blip into a retry storm.
• There's no way back. Nobody wrote down how to roll back.

Every one of these is usually covered somewhere in the migration guide. The guide isn't the problem. The problem is expecting dozens of teams to audit their own config by hand against a long document, under deadline pressure.

If the guidance can be written down, it can be checked automatically.

The solution: a Migration Readiness Checker

The Migration Readiness Checker reads a workload's configuration, checks it against the platform team's migration guidance, and returns a risk report before cutover.

It works like a pre-flight checklist that runs itself. Teams don't each have to interpret the guide on their own. The checker audits the config and reports what will break, what might break, and what is still undecided.

What it reads

• Kubernetes manifests and Helm charts: Deployments, StatefulSets, Services, Ingresses, HPAs, PodDisruptionBudgets
• Legacy platform config: the old YAML configuration file, so it can flag settings that need to move into the self-service portal
• Application config: environment variables and config maps holding endpoints, timeouts, and retry settings
• A migration plan file: a short, team-owned file describing the cutover window, owners, and rollback steps

What it checks

Every check maps to a section of the migration guide and to a real cutover risk.

1. Resource limits
Every container should declare CPU and memory requests and limits. Without them, scheduling on the new cluster is unpredictable and pods risk eviction.

2. Pre-scaling
New clusters and fresh nodes need time to warm up. The checker confirms there's a pre-scaling plan for cutover: either an HPA minimum high enough to absorb traffic, or a planned replica increase during the migration window.

3. DNS names
It scans config for hardcoded hostnames, cluster-internal names, and endpoints tied to the old platform. Anything that won't resolve after the move, or will resolve somewhere unexpected, is flagged.

4. Retry settings
Dependencies blink during cutover. The checker looks for sensible client timeouts, bounded retries with backoff, and idempotency where a retry could duplicate work. It flags both extremes: no retries, and unbounded retries.

5. Rollback plan
A cutover without a rollback plan is a one-way door. The checker requires documented trigger conditions, steps, an owner, and an expected duration.

6. Platform-specific checks

• Ingress: detects ingress-nginx annotations and ingress classes, and lists what needs a Traefik equivalent.
• Egress: collects every outbound destination the workload talks to, so teams can set up namespace egress rules before traffic gets blocked.
• Config ownership: maps each setting in the legacy config file to its equivalent in the portal and flags anything without a clear home.
• Subscription scope: highlights references to resource-group-scoped cloud resources that need to move into the team's new subscription.
• Security scanning: checks that pipelines have Wiz reporting wired up and aren't still depending only on Trivy output.

How it works

flowchart LR A[Workload repo<br/>manifests, Helm, legacy config,<br/>app config, migration plan] --> B[Parsers] B --> C[Normalized workload model] D[Migration rules<br/>from platform guidance] --> E[Rule engine] C --> E E --> F[Risk report<br/>Markdown + JSON] F --> G[PR comment / CI gate] F --> H[Platform dashboard]

The design keeps three concerns separate:

1. Parsers turn many config formats into one normalized model of the workload.
2. Rules live as code and are versioned alongside the migration guide. When the guide changes, the rules change in the same pull request.
3. Reporters produce output for humans (a Markdown report on the pull request) and machines (JSON for dashboards and CI gates).

Rules are short and readable:

- id: RES-001 title: Containers must declare resource limits severity: high applies_to: [Deployment, StatefulSet] check: every container has resources.requests and resources.limits for cpu and memory why: Pods without limits are scheduled unpredictably on new clusters and risk eviction. fix: Add requests and limits based on observed usage from the last 30 days. guide_ref: migration-guide#resources - id: ING-002 title: ingress-nginx annotations need a Traefik equivalent severity: high applies_to: [Ingress] check: no annotations with prefix nginx.ingress.kubernetes.io/ why: The new platform uses Traefik. These annotations will be silently ignored. fix: Replace with a Traefik middleware or IngressRoute. guide_ref: migration-guide#ingress 

Each rule carries a why, a fix, and a link back to the guide. A report that just says "failed" invites arguments. A report that explains the risk and the remedy gets acted on.

Architecture and technology choices

The checker has one firm rule: deterministic checks decide pass or fail, and AI only helps with the fuzzy parts. A readiness gate has to give the same answer twice and be easy to audit. An AI model is useful for explaining findings and handling messy input, but it shouldn't decide whether a cutover is safe.

flowchart TB subgraph Trigger["Trigger"] PR["Pull request"] SCHED["Scheduled fleet scan"] end subgraph Collect["1. Collect and render"] GIT["Fetch workload repo"] RENDER["Render manifests<br/>helm template / kustomize build"] REDACT["Redact secrets and<br/>sensitive values"] end subgraph Normalize["2. Normalize"] PARSE["Parsers<br/>K8s, legacy YAML,<br/>app config, migration plan"] MODEL["Normalized workload model<br/>JSON"] end subgraph Check["3. Deterministic checks"] OPA["Rule engine<br/>OPA / Conftest"] RULES[("Rule catalogue<br/>Rego + YAML metadata<br/>versioned with the guide")] end subgraph AI["4. AI assist (advisory only)"] RAG[("Migration guide index<br/>retrieval")] LLM["LLM<br/>structured JSON output"] VALIDATE["Schema validation"] end subgraph Report["5. Report"] MERGE["Merge findings"] MD["Markdown report<br/>PR comment"] JSON["JSON report"] GATE{"Any HIGH findings?"} end subgraph Consume["Consumers"] CI["CI gate<br/>block or allow cutover booking"] STORE[("Report store")] DASH["Fleet risk dashboard"] end PR --> GIT SCHED --> GIT GIT --> RENDER --> REDACT --> PARSE --> MODEL MODEL --> OPA RULES --> OPA OPA -->|findings| MERGE MODEL -->|unmapped config,<br/>free-text plans| LLM OPA -->|findings to explain| LLM RAG --> LLM LLM --> VALIDATE -->|suggestions| MERGE MERGE --> MD MERGE --> JSON JSON --> GATE --> CI JSON --> STORE --> DASH

1. Collect and render

The checker runs as a container in the CI pipeline, triggered on every pull request and on a nightly scan of all workloads. It fetches the workload repo and renders the final manifests with helm template or kustomize build. Checking rendered output matters: the values teams actually deploy often differ from chart defaults.

Before anything else, a redaction step strips secret values, tokens, and connection strings. Only keys and structure are passed on. This keeps sensitive data out of logs and out of anything sent to an AI model.

2. Normalize

Small parsers, written in Python, read each input format and produce one normalized workload model in JSON: containers, resources, scaling, ingress routes, outbound endpoints, retry settings, and the rollback plan. Every downstream component works from this model, so adding a new input format never touches the rules.

3. Deterministic checks: OPA and Conftest

Rules are written in Rego and run with Open Policy Agent through Conftest, a widely used policy-as-code approach for Kubernetes configuration. Each Rego rule is paired with YAML metadata (severity, why, fix, guide reference) like the examples above.

package migration.ingress import rego.v1 deny contains msg if { input.kind == "Ingress" some key, _ in input.metadata.annotations startswith(key, "nginx.ingress.kubernetes.io/") msg := sprintf("ING-002: %s uses %s", [input.metadata.name, key]) } 

Rules are unit-tested with opa test and versioned in the same repository as the migration guide. A change to the guide and its rule ship in one pull request.

4. AI assist: advisory only

A large language model handles the work that rules can't do well:

Guardrails keep it safe and predictable:

• Grounded in the guide. The migration guide is indexed for retrieval, and every prompt includes the relevant sections. Suggestions must cite a guide section.
• Structured output only. The model returns JSON that is validated against a schema. Anything invalid is dropped, and the deterministic finding still stands.
• Never decides pass or fail. AI output is labelled as a suggestion in the report and can't change a severity or clear a finding.
• Redacted input. The model only sees the redacted workload model, never raw secrets.
• Model-agnostic. Any hosted LLM with structured output works. On an Azure-based platform, a model hosted in the company's own Azure tenant keeps data inside existing boundaries.

The checker still works with the AI step turned off. Teams just get the standard why and fix text instead of tailored suggestions.

5. Report and consume

Deterministic findings and AI suggestions are merged into two outputs:

• A Markdown report, posted as a pull request comment.
• A JSON report, used by the CI gate (any high-severity finding blocks booking a cutover window) and stored for the fleet dashboard.

Technology summary

What the risk report looks like

A report for a typical billing service, a few weeks before cutover:

Migration Readiness Report: billing-service Target: new platform Overall risk: HIGH (2 high, 3 medium, 1 low) HIGH ING-002 Ingress uses nginx.ingress.kubernetes.io/rewrite-target → Recreate as a Traefik middleware before cutover. RBK-001 No rollback plan found in migration-plan.yaml → Document trigger, steps, owner and expected duration. MEDIUM NET-003 Outbound calls to 3 external hosts not covered by an egress rule → Add them to the namespace egress allow-list in the portal. RTY-002 HTTP client for the ledger service retries without backoff (max 10) → Use exponential backoff and cap retries at 3. SCL-001 HPA minReplicas=2; peak traffic suggests at least 6 during cutover → Pre-scale for the migration window. LOW DNS-004 Config references an old-platform hostname in an unused block → Remove to avoid confusion. Passed: 14 checks (resource limits, PDB, liveness/readiness probes, ...) 

A team can read this in two minutes. It shows what blocks the cutover, what to fix this sprint, and what can wait.

Where it fits in the workflow

The checker is most useful early and often, not as a last-minute gate:

1. Discovery: one run across all workloads gives the platform team a heat map of migration risk by team, so support goes where it's needed most.
2. Preparation: the checker runs on every pull request, so teams see their score improve as they fix findings.
3. Before cutover: a report with no high-severity findings is required before a migration window can be booked.
4. After cutover: the final report is archived as part of the migration record.

Design principles

• Guidance as code. Every rule traces back to a section of the migration guide. If it can't, it doesn't belong.
• Explain, don't just judge. Every finding says why it matters and how to fix it.
• Severity means cutover risk. "High" means "this will likely break during migration," not "this isn't best practice."
• Allow justified exceptions. Teams can suppress a finding with a reason and an owner. Exceptions stay visible in the report.
• Read-only by default. The checker reads config and reports. It never changes a workload. Fixes stay with the team that owns the code.

What's next

Natural extensions once the basic checker is in place:

• Runtime signals: combine static config with observed traffic to discover egress destinations and peak load automatically.
• Auto-generated fixes: propose Traefik equivalents for common nginx annotations as draft pull requests.
• Post-cutover verification: run the same checks against live deployments on the new platform to catch drift.

Closing thought

Platform migrations rarely fail because of one big problem. They fail because of a dozen small ones that each looked fine on their own. Platform teams already know what those problems are and have written them down. A Migration Readiness Checker turns that knowledge into something every team can run in seconds, long before the cutover window opens.

If your platform has a shutdown date on the calendar, the best time to find out what will break is now.