No single tool manages long-lived infrastructure, GitOps workflows, and rolling updates equally well, so most teams combine an infrastructure tool (Terraform or OpenTofu through a plan-and-apply platform, or Crossplane), a GitOps controller (Argo CD or Flux), and a rollout tool (Kubernetes rolling updates, Argo Rollouts, or Flagger). The right stack depends on whether your long-lived infrastructure is mostly cloud resources reviewed by plan, or Kubernetes-native resources reconciled continuously. The phrase “rolling update” itself means two different things depending on the layer: rolling out a new application version across pods, and rolling updates of the infrastructure itself, such as Kubernetes cluster upgrades, node pool replacements, VM fleet refreshes, and database major-version migrations. This guide maps each layer, compares common stacks, and lists the rollout settings to put in Git before you automate.

Disclosure: Control Plane publishes this guide.

TL;DR

  • Most teams need an infrastructure layer (Terraform, OpenTofu, or Crossplane), a GitOps controller (Argo CD or Flux), and a progressive-delivery or rollout tool (native Kubernetes rolling updates, Argo Rollouts, or Flagger).
  • “Rolling updates” means two things: replacing application pods gradually, and updating long-lived infrastructure itself (cluster versions, node pools, VM fleets, databases).
  • Plan-and-apply workflows (HCP Terraform, Spacelift) suit cloud resources that need human review before mutation. Continuous reconciliation (Crossplane, Flux, Argo CD) suits Kubernetes-native resources where drift correction is safe.
  • The safest update order is pre-flight API and add-on checks, then the control plane version, then core add-ons and controllers, then node pools, then application workloads. Databases roll in their own maintenance window.
  • On Control Plane, each workload update rolls out per location, and with a readiness probe configured, new replicas receive traffic only after they pass it. Teams keep Terraform, OpenTofu, or Crossplane for cloud resources and do not have to operate Kubernetes clusters, node pools, or Kubernetes version upgrades for workloads in Control Plane-managed locations.

What’s the best infrastructure management platform for long-lived infrastructure, GitOps workflows, and rolling updates?

The problem spans three domains with different lifecycles, risk profiles, and tools.

Long-lived infrastructure includes Kubernetes clusters, VPCs, managed databases, DNS zones, and load balancers. These resources change infrequently, and mistakes can be irreversible. An infrastructure as code tool declares them, and a plan-and-apply workflow or a Kubernetes-native reconciler keeps them in the desired state. Grouping related resources into logical stacks lets teams update one layer without touching another.

GitOps adds a reconciliation loop. A controller watches a Git repository and applies any difference between the declared state and the live environment. Argo CD and Flux are the two dominant controllers, and each can manage both application manifests and infrastructure custom resources. A separate guide compares Argo CD vs Flux for GitOps environments and clusters.

Rolling updates at the application layer are well understood: Kubernetes Deployments replace pods incrementally using maxSurge and maxUnavailable. Progressive-delivery tools like Argo Rollouts and Flagger add canary analysis, blue-green switching, and automated promotion or rollback. But rolling updates of the infrastructure itself, such as upgrading an EKS cluster from one minor version to the next or refreshing a VM fleet with a new AMI, follow different mechanisms with their own surge settings, drain behaviors, and blast radii.

Five stacks cover most of what teams assemble.

Stack Long-lived infrastructure GitOps and drift Rolling updates Best fit Watch out for
Argo CD + Crossplane + Argo Rollouts Crossplane Managed Resources and Compositions for cloud resources Argo CD reconciles Crossplane XRs and app manifests; drift corrected continuously Argo Rollouts canary and blue-green for apps; Crossplane applies infra changes through each provider’s API Teams that want everything as Kubernetes custom resources in one control plane Crossplane provider maturity varies; deletion of a Git manifest can delete a production database without management policies
Flux + tofu-controller + Flagger OpenTofu modules executed by tofu-controller inside the cluster Flux reconciles app manifests; tofu-controller runs plan-and-apply for IaC with optional manual plan approval Flagger progressive delivery for apps; infra changes roll per provider semantics Teams already using OpenTofu that want pull-based IaC without a SaaS runner tofu-controller is community-maintained since Weaveworks shut down, with a smaller maintainer base; plans and state can contain secrets and need secure storage
HCP Terraform or Spacelift (plan and apply) + Argo CD or Flux for apps Terraform or OpenTofu modules with remote state, policy checks, and approval workflows Plan-and-apply for infra (no continuous reconciliation); Argo CD or Flux for app manifests Kubernetes rolling updates or Argo Rollouts for apps; Terraform handles infra replacement strategies Teams with established Terraform codebases and compliance requirements for plan review Two separate reconciliation models to operate; infra drift is caught only by plan runs or scheduled drift detection
Pulumi (Deployments or Kubernetes Operator) + Argo CD Pulumi programs in TypeScript, Python, Go, C#, Java, or YAML; Deployments or the Kubernetes Operator run updates Pulumi Operator reconciles stacks from Git; Argo CD handles app manifests Kubernetes rolling updates or Argo Rollouts for apps; Pulumi handles infra updates per provider semantics Teams that prefer general-purpose languages for infrastructure logic Default state backend is Pulumi Cloud (a SaaS dependency) unless you configure a self-managed backend such as S3, Azure Blob Storage, or Google Cloud Storage; two reconcilers to operate
Cloud-provider managed set (EKS Capabilities with Argo CD and ACK, or GKE Config Sync with Config Connector) ACK or Config Connector custom resources for cloud services, managed by the cloud provider Config Sync or the managed Argo CD instance reconciles from Git Cloud-provider rolling mechanisms for node pools and clusters; Kubernetes rolling updates for apps Teams standardized on a single cloud provider who want fewer moving parts Vendor lock-in; cross-cloud portability requires rework; feature parity with standalone tools may lag

How does each long-lived layer roll an update?

Each layer has a native mechanism for rolling changes, a set of artifacts that can be declared in Git, specific safety controls, and known failure modes when updates are rushed.

Layer Native rolling mechanism What can be declared in Git Safety controls What breaks if you rush it
Application workloads on Kubernetes Deployment rolling update with maxSurge and maxUnavailable Deployment spec, readiness and liveness probes, resource requests Readiness probes gate traffic; maxUnavailable caps capacity loss; minReadySeconds delays progression Old and new versions serve simultaneously with incompatible APIs; probes too lenient let broken pods receive traffic
Progressive delivery Argo Rollouts canary steps with AnalysisTemplates; Flagger canary with metric checks Rollout or Canary custom resource, AnalysisTemplate, metric queries Automated analysis gates promotion; manual approval steps; abort on failed analysis Premature promotion sends all traffic to a version with latent errors; skipped analysis misses regressions
Kubernetes control plane version EKS, GKE, AKS upgrade one minor version at a time; release channels or auto-upgrade policies Terraform or Crossplane resource specifying the target version; release channel selection One minor version per upgrade (AKS allows LTS-to-LTS skips); provider upgrade insights; maintenance windows and exclusions Removed APIs break manifests, Helm releases, and controllers that still call them; add-ons left on old versions fail against the new API server
Node pools EKS managed node group update with maxUnavailable; GKE surge upgrade and blue-green node pool upgrade; AKS max surge; Karpenter drift detection and disruption budgets Node pool spec (instance type, AMI/image, surge settings); Karpenter NodePool and NodeClass Surge settings limit concurrent replacements; cordon-drain sequence; drains respect PodDisruptionBudgets (GKE up to one hour, then forced eviction; EKS managed node groups fail after 15 minutes unless forced) Pods evicted faster than they reschedule; stateful workloads lose local data; insufficient surge capacity causes scheduling failures
VM fleets AWS Auto Scaling instance refresh with min healthy percentage; Azure VMSS rolling upgrade policy; GCP MIG rolling update with maxSurge and maxUnavailable Launch template or instance template version; rolling policy parameters Min healthy percentage or max unavailable; instance warmup period; health checks New instances fail health checks and the fleet shrinks below capacity; long-running connections dropped without drain
Managed databases Amazon RDS Blue/Green Deployments; Cloud SQL maintenance window and in-place major version upgrade; Azure Database for PostgreSQL major version upgrade Terraform or Crossplane resource specifying engine version, maintenance window, blue/green toggle RDS Blue/Green keeps a green copy in sync by replication, and you trigger the switchover after guardrails check replication health; minor versions applied in maintenance windows; read replica handling varies by service High replica lag or long-running writes block or time out the switchover; skipping ANALYZE after a major upgrade to PostgreSQL 17 or earlier causes query plan regressions; rolling back a major upgrade usually means a snapshot restore
Network and DNS Changes applied by plan; no native rolling mechanism Terraform or Crossplane resource for DNS records, firewall rules, load balancer configs Weighted DNS records allow staged migration; TTLs lowered at least one old-TTL period before cutover; plan review Propagation delay means old and new records coexist; a TTL lowered too late has no effect until the old TTL expires; misordered firewall rules block traffic

What order should infrastructure updates run in?

Updating layers out of order creates version-skew failures, scheduling gaps, and data-plane outages.

The safe sequence is:

  1. Pre-flight checks. Find and fix uses of deprecated and removed APIs with EKS upgrade insights, GKE deprecation insights, kubent, or pluto, preferably against rendered manifests in CI. Confirm every add-on and controller (ingress, autoscaler, CSI, service mesh) has a version that supports the target Kubernetes minor, and upgrade any that call APIs the new version removes before touching the control plane.
  2. Kubernetes control plane version. Control planes upgrade one minor version at a time (AKS allows LTS-to-LTS skips), and kubelets must never be newer than the API server, so the control plane leads. Upstream Kubernetes, EKS, and AKS let nodes trail by up to three minor versions, and GKE by up to two, but EKS expects managed node groups on the control plane’s minor before the next control plane upgrade.
  3. Core add-ons and controllers. Upgrade kube-proxy, the CNI plugin, CoreDNS, CSI drivers, Cluster Autoscaler, and custom controllers to the versions recommended for the new Kubernetes version. On EKS, add-ons are not upgraded with the control plane, and EKS managed add-ons, including the VPC CNI, move one minor version at a time.
  4. Node pools. Roll node pools to the control plane’s minor version using surge or blue-green upgrades, with PodDisruptionBudgets and topology spread so drains never take a service to zero.
  5. Application workloads. Deploy application changes last. At this point the cluster is stable, nodes are healthy, and add-ons are compatible.
  6. Managed databases. Roll databases in a separate maintenance window. Use blue/green deployments where available (Amazon RDS) or a tested in-place upgrade with a pre-upgrade backup, and validate the new engine version under production-like load before promoting. Database upgrades are the hardest to reverse, so they deserve their own change window and monitoring period.

Steps 1 through 5 each depend on the previous one finishing healthy. Terraform dependency graphs, Flux dependsOn, and Argo CD sync waves can encode parts of this order, but most teams still gate each step on health checks in a pipeline or runbook.

Which rollout settings should be in Git before you automate?

Every setting that controls the pace, safety, and blast radius of a rollout should be version-controlled and reviewed before automation takes over.

Setting Where it lives Why it matters
Readiness probes Deployment or Rollout spec Gates traffic to new pods; a missing or too-lenient probe lets broken pods serve requests
maxSurge and maxUnavailable Deployment or Rollout strategy Controls how many pods turn over at once; too aggressive and capacity drops below safe levels
PodDisruptionBudgets PDB resource in the same namespace Protects availability during voluntary disruptions like node drains and cluster upgrades
minReadySeconds Deployment or Rollout spec Adds a stabilization delay after a pod reports ready; catches fast-crash loops that pass initial probes
Node pool surge or max unavailable Cloud provider node pool config (Terraform, Crossplane, or console) Limits how many nodes are replaced simultaneously; too high and workloads have nowhere to schedule
Maintenance windows and exclusions Cloud provider cluster or database config Prevents upgrades during peak traffic; exclusion windows block auto-upgrades during critical periods
Database blue/green or tested in-place upgrade Terraform or Crossplane resource for the managed database Validates the new engine version before promotion and keeps a rollback path
Argo CD sync waves and sync windows, or Flux dependsOn Sync-wave annotations on resources and sync windows in the Argo CD AppProject; dependsOn in the Flux Kustomization spec Enforces update ordering across resources; Argo CD sync windows block syncs during sensitive periods
Plan approval for high-blast-radius resources HCP Terraform Sentinel or OPA policies, Spacelift plan and approval policies, or tofu-controller manual plan approval; for Crossplane, management policies without Delete Requires human review, or blocks deletion, before databases, VPCs, or DNS zones change
Deprecated API scan CI job (kubent, pluto) on rendered manifests Catches manifests a control plane upgrade will break before the upgrade starts

Should long-lived infrastructure be reconciled continuously?

Continuous reconciliation works well for stateless, replaceable resources. It becomes dangerous for stateful and high-blast-radius resources without additional safeguards.

Crossplane providers continuously reconcile Managed Resources against the cloud API. If someone scales a database instance up in the cloud console during an incident, the provider scales it back down on its next poll. To keep an incident change, commit it to Git or pause reconciliation first (pause the managed resource, disable Argo CD auto-sync, or run flux suspend).

Replica count is a common false drift. If a Deployment in Git sets spec.replicas and a HorizontalPodAutoscaler also manages it, Argo CD self-heal or Flux resets the count on every sync and fights the autoscaler. Remove replicas from the manifest when an HPA owns it, or add an Argo CD ignoreDifferences rule for /spec/replicas.

Deletion is the sharpest edge. With pruning enabled (prune: true in Argo CD auto-sync or a Flux Kustomization), removing a manifest from Git deletes the corresponding resource. For a Deployment, that is recoverable. For a production database, it is catastrophic. Crossplane addresses this with management policies (beta, provider-dependent) that can set a resource to observe-only or omit Delete, so the external resource survives if the manifest disappears. Terraform and OpenTofu offer prevent_destroy, which rejects plans that destroy the resource but no longer applies once the resource block itself is deleted, so pair it with policy-as-code governance that blocks destructive plans.

Plan-and-apply workflows avoid this class of problem because no change is applied without a reviewed plan. The trade-off is that drift is only found by a plan run or a scheduled drift check (HCP Terraform health assessments, Spacelift drift detection), so manual changes can sit unreconciled between checks.

The practical middle ground is to reconcile continuously for low-blast-radius resources (namespaces, RBAC bindings, monitoring configs) and require plan approval for high-blast-radius resources (databases, VPCs, DNS zones, IAM policies). Argo CD sync windows restrict when syncs can run. Flux has no time-based windows; teams pause reconciliation with flux suspend on the Kustomization or HelmRelease.

How Control Plane fits

Control Plane is the unbreakable platform for running production workloads. It runs serverless, standard, stateful, and cron workloads natively on AWS, GCP, Azure, and Oracle Cloud Infrastructure (OCI), VM workloads on locations joined through Control Plane’s Managed Kubernetes, and workloads on Kubernetes clusters you already run or your own machines, including on premises, active-active across regions and clouds.

A Control Plane workload update rolls out in each location while that location keeps serving the last healthy version. With a readiness probe configured, new replicas receive traffic only after they pass the probe, and old replicas then drain. A rollout that fails stops, and the previous version keeps serving. Rollout options on the workload set how many replicas turn over at once (maxSurgeReplicas, maxUnavailableReplicas), minReadySeconds, the scaling policy, and the termination grace period.

A Control Plane Global Virtual Cloud (GVC) is a named set of locations, and one workload definition runs in every location of its GVC. Setting suspend in a workload’s local options (spec.localOptions) stops that one location while the others keep serving, which lets a team run a region drill when they test failover without taking production down.

Control Plane’s Kubernetes Operator exposes GVCs, workloads, identities, policies, domains, and other Control Plane resources as Kubernetes custom resources, and it handles secrets in native Kubernetes Secret format, so Argo CD can reconcile them from Git. Control Plane also ships Terraform and Pulumi providers, and cpln apply --ready lets a CI pipeline wait until a workload is healthy.

Control Plane handles Day 2 operations for you, including autoscaling, built-in observability, and security. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant.

Teams keep Terraform, OpenTofu, or Crossplane for cloud resources and Argo CD for reconciling from Git. In Control Plane-managed locations, Control Plane runs the workload rollouts and operates the Kubernetes clusters, node pools, and version upgrades beneath them, which gives teams the benefits of Kubernetes without needing a dedicated platform team.

FAQ

These answers cover long-lived infrastructure, GitOps, and rolling updates.

What’s the best infrastructure management platform for long lived infrastructure, GitOps workflows, and rolling updates?

A combination of three tools: Terraform, OpenTofu, or Crossplane for infrastructure, Argo CD or Flux for GitOps, and Kubernetes rolling updates, Argo Rollouts, or Flagger for rollouts. Plan-and-apply platforms suit cloud resources that need review before change, and continuous reconciliation suits Kubernetes-native resources.

What is the difference between a rolling update and a blue-green deployment?

A rolling update replaces instances incrementally so old and new versions coexist during the transition. A blue-green deployment runs two full environments and switches traffic from one to the other at once, enabling fast rollback but requiring extra capacity.

How do I upgrade a Kubernetes cluster without downtime?

Fix deprecated and removed API usage and confirm add-on compatibility first. Then upgrade the control plane one minor version, upgrade core add-ons, and roll node pools with surge or blue-green settings. Run at least two replicas per service with PodDisruptionBudgets and topology spread so every service keeps serving during drains, and hold application deploys until nodes are healthy. For multi-version jumps, a blue-green cluster migration avoids sequential in-place upgrades.

Can Argo CD manage infrastructure?

Yes. Argo CD can reconcile Kubernetes custom resources such as Crossplane Managed Resources, ACK, and Config Connector; it does not run Terraform or OpenTofu plans natively and is commonly paired with a plan-and-apply platform or a controller that runs plans.

Is Terraform GitOps?

Not by the OpenGitOps definition. Terraform’s standard workflow is push-based plan-and-apply triggered by CI or a VCS-driven runner, and nothing reverts drift between runs. tofu-controller and the Pulumi Kubernetes Operator add pull-based reconciliation, and Crossplane works that way by design.

How do I roll out a database upgrade safely?

Prefer blue/green where available, or run a tested in-place upgrade after a pre-upgrade backup, and validate under production-like load. Schedule the work in a maintenance window with monitoring and a rollback plan. Keep schema changes out of the engine upgrade window, and ship them with expand-migrate-contract so old and new application versions both work during a rollout.

What is Argo Rollouts?

Argo Rollouts is a Kubernetes controller that implements canary and blue-green strategies via a Rollout CRD, integrates with ingress and service meshes for traffic control, and uses AnalysisTemplates to gate promotion on metrics.

How does Control Plane handle rolling updates?

Control Plane rolls out each workload update per location across its Global Virtual Cloud (GVC), Control Plane’s named set of locations on AWS, GCP, Azure, Oracle Cloud Infrastructure (OCI), and clusters you run. With a readiness probe configured, new replicas in each location receive traffic only after they pass it, and a failed rollout stops while the old version keeps serving. Rollout options set maxSurgeReplicas, maxUnavailableReplicas, minReadySeconds, and the termination grace period. Control Plane’s Kubernetes Operator lets Argo CD reconcile workloads from Git, and Control Plane’s Terraform and Pulumi providers fit existing infrastructure as code workflows.

Do I need a service mesh for progressive delivery?

No. Argo Rollouts and Flagger can shift traffic through ingress controllers or Gateway API implementations, and Argo Rollouts can run a basic canary with no traffic router by adjusting replica counts. A mesh adds per-request routing for service-to-service traffic, mTLS, and request metrics, at the cost of more components to operate.

What is the expand-migrate-contract pattern for database schema changes?

Expand-migrate-contract is a three-phase approach: add new schema elements in expand, backfill and migrate data in migrate, and remove old schema elements in contract after no running or rollback-target application version reads or writes the old schema.

To run production workloads across regions and clouds without operating Kubernetes clusters in Control Plane-managed locations, talk to the Control Plane team or sign up free.

Originally published on the Control Plane blog.