Two sync results side by side: Namespace prune-test-a "Pruned / pruned" and Namespace prune-test-b "PruneSkipped / ignored (no prune)", above the sentence "Argo CD automated prune deletes a Namespace when its manifest leaves Git." and the lines "Two test Applications, the same commit. The second Namespace carried argocd.argoproj.io/sync-options: Prune=false,Delete=false. Eight of our production Namespaces and 23 CRDs had no such protection."

If you searched "argocd deleted namespace", "argocd prune deleted my namespace" or "argocd prevent namespace deletion", run kubectl logs -n argocd sts/argocd-application-controller | grep "Adding resource result". A Namespace that Argo CD has just deleted shows Pruned / pruned; one that it skipped shows PruneSkipped / ignored (no prune). The cause of the first: an Application with automated prune: true tracks its own Namespace (or a CRD), and that object's manifest dropped out of what Git renders, so the next automated sync deleted it with everything inside. The difference between the two results is one annotation on the object, argocd.argoproj.io/sync-options: Prune=false,Delete=false: Prune=false stops the prune, and Delete=false covers the other path, where an ApplicationSet deletes the Application and resources-finalizer.argocd.argoproj.io deletes the resources it tracks. On our cluster an audit found 8 production Namespaces and 23 CRDs exposed this way. Nothing was deleted in production: we reproduced both deletion paths with throwaway test Applications, then fixed the manifests. Below are the raw output of that test, the Argo CD source lines that decide which annotation blocks which path, why the auto-sync will wipe out all resources protection (allowEmpty) does not cover one removed manifest, read-only checks with the value that means you have the same problem, and what we changed and did not change, including an argocd_app_info{sync_status="OutOfSync"} alert. The cluster is a 2-node k3s cluster (k3s v1.33.5+k3s1) running Argo CD v3.2.0, with the manifests repository on GitHub; all times are UTC, 2026-11-14.

What we saw: Pruned / pruned versus PruneSkipped / ignored (no prune)

The finding

The finding came from a full run of kubegap v0.3.0, an open-source audit of Kubernetes cluster operations, taken at 16:25:29. Its problem 3, translated from Korean: "Argo CD apps with prune enabled manage a Namespace and CRDs." The reason it gave: if one manifest drops out of Git or a path changes, Argo CD can delete a production namespace with the PVCs and Secrets in it, or delete the Istio and Gateway API CRDs and with them every object of those kinds. The report counted 9 of 11 production Applications with automated prune: true that track a Namespace or CRD: 8 track one Namespace each, and istio tracks 23 CRDs. All 9 have selfHeal and the finalizer resources-finalizer.argocd.argoproj.io. The report had not opened the repository; it inferred that the Git directories contain a Namespace manifest.

At 19:11 we ran the check ourselves (names masked, CRD lines shortened):

$ kubectl get applications.argoproj.io -A -o json | jq -r '.items[] | select(.spec.syncPolicy.automated.prune==true) | .metadata.name as $n | .status.resources[]? | select(.kind=="Namespace" or .kind=="CustomResourceDefinition") | $n+" "+.kind+"/"+.name' prod-a Namespace/prod-a prod-j Namespace/prod-j prod-k Namespace/prod-k prod-c Namespace/prod-c prod-g Namespace/prod-g istio CustomResourceDefinition/authorizationpolicies.security.istio.io istio CustomResourceDefinition/backendtlspolicies.gateway.networking.k8s.io istio CustomResourceDefinition/destinationrules.networking.istio.io istio CustomResourceDefinition/envoyfilters.networking.istio.io ... 19 more CustomResourceDefinition lines for istio prod-i Namespace/prod-i prod-l Namespace/prod-l prod-h Namespace/prod-h 

Terminal output of the check before the fix: eight lines of the form "prod-a Namespace/prod-a" and the CustomResourceDefinition lines of Application istio, with the note "31 lines: 8 Namespaces in 8 Applications, 23 CRDs in one."

Before the fix, 19:11 UTC: every Application with automated prune that tracks a Namespace or a CRD. 31 lines.

The rest of what we read at 19:11:

  • The cluster has 22 Applications. 21 have automated prune: true. 15 are generated by one ApplicationSet, prod-apps, and 7 are tracked by an Application named root.
  • None of the 8 Namespaces had an argocd.argoproj.io/sync-options annotation. Each had the tracking annotation argocd.argoproj.io/tracking-id: <app>:/Namespace:/<name>.
  • The inference of the report was right: the manifests repository has a Namespace manifest in each of the 8 directories, 7 as a namespace.yaml listed in kustomization.yaml and one inside a larger manifest file.
  • prod-apps uses a git directory generator on prod/*. Its template has prune: true, selfHeal: true, CreateNamespace=true, no finalizers, and the ApplicationSet has no spec.syncPolicy. The generated Applications carry resources-finalizer.argocd.argoproj.io all the same, and they have no destination.namespace; the namespace comes from namespace: in each kustomization.
  • The 8 namespaces held 51 pods, 16 PVCs and 44 Secrets. Counted later, at 19:38: 8 of the 16 PVCs are not tracked by Argo CD (they come from StatefulSet volumeClaimTemplates), 26 of the 44 Secrets are owned by an ExternalSecret and 18 have no owner. Of the 16 PersistentVolumes bound in those namespaces, 15 have reclaim policy Retain and 1 has Delete (a 5Gi backup volume).
  • The 23 CRDs come from two sources: the Istio base chart 1.30.8, whose CRDs carried helm.sh/resource-policy: keep and no sync-options, and Gateway API v1.5.1 from a kustomize remote, whose CRDs carried neither. In the listing taken at 19:11:07, 15 CRDs show keep and 8 show no annotation. Custom objects existed under 7 of the 23 kinds: gatewayclasses 2, gateways.gateway.networking.k8s.io 1, httproutes 3, destinationrules 1, peerauthentications 1, telemetries 1, virtualservices 1.
  • Prometheus had no argocd_app_info series, no rule mentioning argocd_, and there was no ServiceMonitor for Argo CD.

The test

We did not remove anything from a production directory to find out. We added three throwaway directories under prod/, each a kustomization with a Namespace and one ConfigMap: prune-test-a and prune-test-c without any annotation, prune-test-b with argocd.argoproj.io/sync-options: Prune=false,Delete=false on the Namespace. The commit was pushed at 19:13:54. The ApplicationSet created the three Applications at 19:14:36, and all three were Synced and Healthy by 19:14:47, each tracking ConfigMap/tracked-by-git and its Namespace, each with the finalizer. At 19:14:45 we created a Secret named not-in-git by hand in each of the three namespaces, as a stand-in for objects that exist only in the cluster.

Step 1: the Namespace manifest leaves Git. At 19:15:03 we pushed a commit that removes namespace.yaml, and its line in kustomization.yaml, from prune-test-a and prune-test-b. The application controller log (commit replaced by <commit>):

prune-test-a 19:17:51 Initiated automated sync to [<commit>] 19:17:51 Adding resource result, status: 'Pruned', phase: 'Succeeded', message: 'pruned' 19:18:00 Adding resource result, status: 'SyncFailed', phase: 'Failed', message: 'namespaces "prune-test-a" not found' prune-test-b 19:18:10 Initiated automated sync to [<commit>] 19:18:10 Adding resource result, status: 'PruneSkipped', phase: 'Succeeded', message: 'ignored (no prune)' 19:18:11 Sync operation to <commit> succeeded 

At 19:18:40:

$ kubectl get configmap,secret -n prune-test-a No resources found in prune-test-a namespace. $ kubectl get configmap,secret -n prune-test-b configmap/istio-ca-crl 1 4m3s configmap/istio-ca-root-cert 1 4m3s configmap/kube-root-ca.crt 1 4m3s configmap/tracked-by-git 1 4m3s secret/not-in-git Opaque 1 3m56s 

Terminal with the application controller log lines for prune-test-a ("status: 'Pruned' ... message: 'pruned'", then "namespaces \"prune-test-a\" not found") and prune-test-b ("status: 'PruneSkipped' ... message: 'ignored (no prune)'"), followed by kubectl output: no resources in prune-test-a, and the ConfigMaps and the Secret not-in-git still present in prune-test-b.

The same commit removed the Namespace manifest from both test Applications. Argo CD pruned the first Namespace, and the Secret we had created by hand in it was deleted with the namespace by Kubernetes. The second Namespace was skipped.

Namespace prune-test-a got its deletionTimestamp at 19:17:51, 2 min 48 s after the push, and was gone at the 19:18:05 sample. No webhook is configured (inferred from the controller log line "comparison expired"); the controller picked the commit up on its own refresh interval. The Secret we had made by hand went with the namespace.

The Application prune-test-a did not recover. It still had a ConfigMap to apply into a namespace that no longer existed (status at 19:18:40, trimmed):

"sync": "OutOfSync", "health": "Missing", "message": "one or more objects failed to apply, reason: namespaces \"prune-test-a\" not found. Retrying attempt #3 at 7:18PM." 

It retried 5 times until 19:23:10. The namespace was not recreated although the Application has CreateNamespace=true. My reading is that this is because the generated Applications have no destination.namespace (inference; we did not test it separately).

prune-test-b at 19:18:40: operation Succeeded, application OutOfSync / Healthy, and in status.resources the Namespace is OutOfSync with requiresPruning: true. The Namespace was Active with the same uid and the same creationTimestamp (19:14:37), and both tracked-by-git and not-in-git were present.

Step 2: the directories are removed. At 19:18:55 we pushed a commit that removes all three directories. The result is in the next section, with the code that explains it: prune-test-c was deleted with its Application, and prune-test-b stayed.

Why

A tracked Namespace is an ordinary resource to prune

An Application tracks a Namespace when a Namespace manifest is part of what its source renders. Ours had one in every directory, so each Application listed its own Namespace in status.resources.

The Argo CD documentation for automated sync says: "By default (and as a safety mechanism), automated sync will not delete resources when Argo CD detects the resource is no longer defined in Git." With prune: true in the automated policy, it does. The only built-in protection left is this one: "automated sync with prune have a protection from any automation/human errors when there are no target resources." In the source that check, which reports auto-sync will wipe out all resources, fires only when every resource of the Application needs pruning and allowEmpty is false. If one manifest drops out and the rest still render, it does not fire. That is the case of prune-test-a: the ConfigMap stayed in Git, the Namespace manifest left, and the Namespace was pruned.

What is deleted together with a Namespace or a CRD is decided by Kubernetes, not by Argo CD. The Kubernetes documentation on deleting a namespace: "This deletes everything under the namespace!" On deleting a CRD: "the server will uninstall the RESTful API endpoint and delete all custom objects stored in it." Everything under the namespace includes objects Argo CD never tracked, which is what happened to the hand-made Secret in prune-test-a.

Prune is stopped by one annotation only

This is pruneObject in gitops-engine at the commit that Argo CD v3.2.0 pins (pkg/sync/sync_context.go):

func (sc *syncContext) pruneObject(liveObj *unstructured.Unstructured, prune, dryRun bool) (common.ResultCode, string) { if !prune { return common.ResultCodePruneSkipped, "ignored (requires pruning)" } else if resourceutil.HasAnnotationOption(liveObj, common.AnnotationSyncOptions, common.SyncOptionDisablePrune) { return common.ResultCodePruneSkipped, "ignored (no prune)" } if dryRun { return common.ResultCodePruned, "pruned (dry run)" } ... return common.ResultCodePruned, "pruned" } 

It returns without deleting in two cases: the sync ran without prune (ignored (requires pruning)), or the live object has Prune=false in argocd.argoproj.io/sync-options (ignored (no prune), result code PruneSkipped). helm.sh/resource-policy is not checked there, and neither is the kind of the object.

Deleting the Application is a second path, with different exemptions

The Argo CD documentation on the finalizer: "When deleting an Application with this finalizer, the Argo CD application controller will perform a cascading delete of the Application's resources." For ApplicationSets, the documentation says generated Applications get the finalizer "if .syncPolicy.preserveResourcesOnDeletion is set to false", and that preserveResourcesOnDeletion: true "prevents the finalizer from being added to the Application". The source (applicationset/utils/utils.go lines 272–280) adds the finalizer when the template has none and syncPolicy is absent or preserveResourcesOnDeletion is false. Our ApplicationSet had no syncPolicy, which is why every generated Application carried the finalizer without our having written it anywhere.

With a git directory generator, the Application is deleted when the generator no longer produces an Application of that name: the directory was removed, renamed or moved out of the pattern. What the cascade then deletes is decided by shouldBeDeleted (controller/appcontroller.go lines 1165–1170 in v3.2.0):

// shouldBeDeleted returns whether a given resource obj should be deleted on cascade delete of application app func (ctrl *ApplicationController) shouldBeDeleted(app *appv1.Application, obj *unstructured.Unstructured) bool { return !kube.IsCRD(obj) && !isSelfReferencedApp(app, kube.GetObjectRef(obj)) && !resourceutil.HasAnnotationOption(obj, synccommon.AnnotationSyncOptions, synccommon.SyncOptionDisableDeletion) && !resourceutil.HasAnnotationOption(obj, helm.ResourcePolicyAnnotation, helm.ResourcePolicyKeep) } 

It returns false for a CRD, for the Application's self-reference, for an object with Delete=false, and for an object with helm.sh/resource-policy: keep. Prune=false is not in this function.

Step 2 of the test, at 19:19:17 (ApplicationSet controller first, then the application controller):

19:19:17 prune-test-c Deleted application 19:19:17 prune-test-a Deleted application 19:19:17 prune-test-b Deleted application 19:19:17 prune-test-c Deleting application's resources with Foreground propagation policy 19:19:17 prune-test-c 2 objects remaining for deletion 19:19:17 prune-test-b Deleting application's resources with Foreground propagation policy 19:19:17 prune-test-b 1 objects remaining for deletion 

At 19:23:48 (the prune-test-a lines left out; that namespace had been deleted in step 1):

$ kubectl get applications -n argocd | grep prune-test (none) $ kubectl get ns | grep prune-test prune-test-b Active 9m11s $ kubectl get configmap,secret -n prune-test-c No resources found in prune-test-c namespace. $ kubectl get configmap,secret -n prune-test-b configmap/istio-ca-crl 1 9m11s configmap/istio-ca-root-cert 1 9m11s configmap/kube-root-ca.crt 1 9m11s secret/not-in-git Opaque 1 9m4s 

Terminal with the controller log lines "Deleted application" and "Deleting application's resources with Foreground propagation policy" for prune-test-c (2 objects remaining) and prune-test-b (1 object remaining), followed by kubectl output at 19:23:48: no test Applications, only namespace prune-test-b Active, nothing in prune-test-c, and the Secret not-in-git still in prune-test-b.

Removing the directories made the ApplicationSet delete the Applications. The unprotected Namespace went with its Application; the one with Delete=false stayed, with the hand-made Secret.

Namespace prune-test-c got its deletionTimestamp at 19:19:17 and was gone at the 19:19:27 sample, with its hand-made Secret. For prune-test-b the controller counted 1 object for deletion, not 2: the Namespace kept the same uid, and the Secret not-in-git was still in it. The ConfigMap tracked-by-git, which was tracked and had no annotation, was deleted by the cascade. Delete=false on the Namespace protects the Namespace and whatever Argo CD does not track inside it; it does not protect the other tracked resources of the Application. The ApplicationSet controller logged Deleted application for prune-test-a at 19:19:17 as well, but the Application stayed until 19:23:10: its finalizer ran only after the sync operation failed (5 retries), and it then deleted 0 resources.

The two paths against the four protections

Automated prune (manifest leaves Git) Cascade (Application deleted with the finalizer)
Prune=false on the object Skipped: PruneSkipped / ignored (no prune) Not checked: deleted
Delete=false on the object Not checked: pruned Skipped
helm.sh/resource-policy: keep on the object Not checked: pruned Skipped
The object is a CRD No exemption: pruned unless it has Prune=false Skipped, with or without any annotation

The table is read from the two functions above. Our test confirmed the Namespace cases with both options set together (skipped on both paths) and with neither (deleted on both paths). The conclusions that follow are inferences from the source: a CRD can be deleted by Argo CD through prune and not through the cascade; keep does not stop prune; a Namespace needs Prune=false against prune and Delete=false against the cascade.

One thing I had wrong before reading the source: I took the helm.sh/resource-policy: keep on the Istio CRDs for what protects them when the Application is deleted. CRDs are skipped by the cascade with or without any annotation, so keep adds nothing for a CRD on that path, and it does nothing on the prune path.

What to check in your cluster now

Checks 1 to 4 are read-only and need kubectl and jq. Check 5 changes Git and the cluster.

1. Applications with automated prune that track a Namespace or a CRD, and whether that object is protected

kubectl get applications.argoproj.io -A -o json | jq -r ' .items[] | select(.spec.syncPolicy.automated.prune==true) | .metadata.name as $app | .status.resources[]? | select(.kind=="Namespace" or .kind=="CustomResourceDefinition") | "\($app) \(.kind) \(.name)"' | while read app kind name; do opt=$(kubectl get "$kind" "$name" -o json | jq -r '.metadata.annotations["argocd.argoproj.io/sync-options"] // "-"') echo "$app $kind/$name sync-options=$opt" done 

Same problem if: a line ends in sync-options=-, or the value has no Prune=false. (Prune=confirm makes the sync wait for a confirmation; we did not test it.) That Application will delete that Namespace or CRD on the next automated sync after its manifest drops out of the rendered output. Before the fix we had 31 such lines: 8 Namespaces in 8 Applications and 23 CRDs in one. A Namespace line whose value has Prune=false but not Delete=false is still open to check 2 (CRDs are not: the cascade skips them).

No output at all means no Application with automated prune tracks a Namespace or CRD.

If you do not want to repeat check 1 by hand after every change, kubegap runs the first half of it: it lists every Application that has automated prune and tracks a Namespace or a CRD. Its run at 16:25:29 is where this problem was found. Version 0.3.0 does not read the annotation on the object, so the second half of check 1 stays manual; "kubegap after the fix" below says what that means.

Terminal output of check 1 after the fix: eight lines of the form "prod-a Namespace/prod-a sync-options=Prune=false,Delete=false", with the note that no CRD line is printed because Application istio now has automated prune off.

After the fix, 19:39 UTC: the eight Namespaces carry both options, and no CRD is listed because the Application that tracks them no longer prunes.

2. Applications whose tracked resources are deleted when the Application is deleted

kubectl get applications.argoproj.io -A -o json | jq -r ' .items[] | [.metadata.name, ((.metadata.finalizers // []) | join(",")), ((.metadata.ownerReferences // []) | map(.kind+"/"+.name) | join(","))] | @tsv' kubectl get applicationsets.argoproj.io -A -o json | jq -r ' .items[] | [.metadata.name, (.spec.syncPolicy | tostring)] | @tsv' 

Same problem if: an Application has resources-finalizer.argocd.argoproj.io and is owned by an ApplicationSet whose syncPolicy is null (or has neither preserveResourcesOnDeletion: true nor an applicationsSync policy of create-only or create-update). When the generator stops producing that Application (for a git directory generator: the directory is renamed, moved or removed), the ApplicationSet controller deletes the Application and, because of the finalizer, the application controller deletes every resource the Application tracks. A Namespace it tracks goes too unless it carries Delete=false (or helm.sh/resource-policy: keep). CRDs are skipped by this path. Ours printed prod-apps null at 19:39, after the fix; the ApplicationSet was not changed.

3. What is deleted with a Namespace and cannot be recreated from Git

NS=<namespace>; APP=<application> kubectl get pvc -n "$NS" -o json | jq -r '.items[].metadata.name' | sort > /tmp/pvc-all kubectl get application "$APP" -n argocd -o json | jq -r '.status.resources[] | select(.kind=="PersistentVolumeClaim") | .name' | sort > /tmp/pvc-tracked comm -23 /tmp/pvc-all /tmp/pvc-tracked # PVCs Argo CD does not track kubectl get secret -n "$NS" -o json | jq -r '.items[] | select(.metadata.ownerReferences==null) | .metadata.name' # Secrets nothing would recreate kubectl get pv -o json | jq -r --arg ns "$NS" ' .items[] | select(.spec.claimRef.namespace==$ns) | "\(.spec.claimRef.name) \(.spec.persistentVolumeReclaimPolicy)"' 

Same problem if: the first list is not empty (PVCs made from a StatefulSet's volumeClaimTemplates are the usual case), or the second list has Secrets you created by hand, or a PV shows Delete. A PVC is deleted with its Namespace; with reclaim policy Delete the volume's data is deleted too, with Retain the PersistentVolume stays and has to be re-bound by hand. Ours: 16 PVCs in the eight namespaces, 8 of them not tracked; 44 Secrets, 18 without an owner; 15 volumes Retain, 1 Delete.

4. Would anyone notice an Application that stays OutOfSync?

Prune=false turns "the Namespace is deleted" into "the Application stays OutOfSync". That only helps if someone is told.

kubectl get prometheusrules -A -o json | jq -r ' .items[] | (.metadata.namespace+"/"+.metadata.name) as $r | .spec.groups[].rules[] | select((.expr|tostring)|test("argocd_app_info")) | "\($r) \(.alert // .record)"' 

and in Prometheus: count by (sync_status) (argocd_app_info).

Same problem if: the first command prints nothing, or the query returns no series (the application controller's metrics port, 8082, is not scraped). Both were true on our cluster before the fix.

5. Test it with a throwaway Application (changes Git and the cluster)

In the repository your Applications read, add three directories, each with a Namespace manifest and one ConfigMap; give only b the annotation:

apiVersion: v1 kind: Namespace metadata: name: prune-test-b annotations: argocd.argoproj.io/sync-options: Prune=false,Delete=false 

After the three Applications are Synced, create one object by hand in each namespace (kubectl create secret generic not-in-git --from-literal=note=x -n prune-test-a), then:

  1. Remove namespace.yaml (and its line in kustomization.yaml) from a and b, push, wait for the automated sync. kubectl get ns | grep prune-test and kubectl get application prune-test-b -n argocd -o json | jq '.status.operationState.syncResult.resources'; the Pruned result of prune-test-a is in the controller log (kubectl logs -n argocd sts/argocd-application-controller | grep "Adding resource result"), not in the Application status
  2. Remove all three directories, push, wait for the Applications to disappear. kubectl get ns | grep prune-test and kubectl get secret -n prune-test-b
  3. kubectl delete ns prune-test-b

Protected if: after 1, prune-test-a is gone and prune-test-b is Active with the result PruneSkipped / ignored (no prune); after 2, prune-test-c is gone and prune-test-b is still Active with the hand-made Secret in it.

What we changed, and what we have not

Changed

Three commits, pushed at 19:24:03. (The comment in the second one first gave the CRD split as 13 and 10; a follow-up commit at 20:27:55 corrected it to the 15 and 8 shown below.)

1. Prune=false,Delete=false on the 8 Namespace manifests. The manifests stay in Git. The report offered two ways: take the Namespace manifest out of Git and rely on CreateNamespace=true, or keep it and annotate. I kept it, because taking the manifest out is the very change that triggers the prune, and it would have to be preceded by the annotation in any case.

apiVersion: v1 kind: Namespace metadata: name: prod-a annotations: argocd.argoproj.io/sync-options: Prune=false,Delete=false 

2. prune: false in the automated policy of Application istio. The Istio base chart 1.30.8 has no value that adds annotations to its CRDs; its CRD-related values are base.excludedCRDs, base.enableCRDTemplates and base.enableIstioConfigCRDs. Automated prune is the only path we found in the source by which an automated sync deletes a CRD (a manual sync with force or replace was not tested), so turning prune off for the one Application that tracks them closes that path.

# argocd-apps/istio.yaml spec: syncPolicy: automated: # prune is off: this Application owns 23 CRDs (15 from the istio base chart, 8 Gateway API). A CRD that drops out  # of the render would be pruned together with every object of that kind, and the base chart has no value to put  # Prune=false on its CRDs. After a chart upgrade, delete resources the new version no longer renders by hand.  prune: false selfHeal: true 

3. A scrape of the Argo CD application controller and an alert.

apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: argocd-metrics namespace: monitoring labels: release: kube-prometheus-stack spec: namespaceSelector: matchNames: [argocd] selector: matchLabels: app.kubernetes.io/name: argocd-metrics endpoints: - port: metrics --- apiVersion: monitoring.coreos.com/v1 kind: PrometheusRule metadata: name: argocd-rules namespace: monitoring labels: release: kube-prometheus-stack spec: groups: - name: argocd.rules rules: - alert: ArgoAppOutOfSync expr: argocd_app_info{sync_status="OutOfSync", name!="root"} == 1 for: 30m labels: severity: warning 

root is excluded because it is permanently OutOfSync for an unrelated reason: one Application is defined both in argocd-apps/ and by the ApplicationSet. That is not fixed here.

State after. At 19:31 all 8 live Namespaces carried Prune=false,Delete=false and were Active with their original creationTimestamp; istio had automated: {prune: false, selfHeal: true}; the 8 Applications and istio were Synced. At 19:39:18 check 1 printed 8 lines, each ending sync-options=Prune=false,Delete=false, and no CRD line. At 19:37:38, after the monitoring Application had synced (its first Helm hook pod waited about 9 minutes on an image pull), up{job="argocd-metrics"} was 1, count by (sync_status) (argocd_app_info) returned Synced 21 and OutOfSync 1 (root), and the rule ArgoAppOutOfSync was loaded and inactive.

The alert, tested. At 19:43:12 we pushed prune-test-b again, with the annotated Namespace, and at 19:44:43 removed its namespace.yaml. The alert ArgoAppOutOfSync{name="prune-test-b"} became pending at 19:48:49 and fired at 20:18:49, 30 minutes later. Alertmanager shows it active from 20:18:49 with receiver discord. At 20:19:02 the Application was OutOfSync / Healthy, the Namespace result was PruneSkipped / ignored (no prune), and the Namespace was Active, 34 minutes old. We then removed the directory and deleted the Namespace by hand.

Prometheus Alerts page filtered to ArgoAppOutOfSync: rule group argocd.rules, expression argocd_app_info{name!="root",sync_status="OutOfSync"} == 1, for: 30m, severity warning, one alert in state FIRING with labels name="prune-test-b", sync_status="OutOfSync", health_status="Healthy".

Prometheus, 20:22 UTC: the alert firing for the test Application, which had been OutOfSync for more than 30 minutes because its annotated Namespace was PruneSkipped. Project and repository names are replaced.

What it costs

The Argo CD documentation says of Prune=false: "The app will be out of sync if Argo CD expects a resource to be pruned." We saw two consequences.

  • The Application stays OutOfSync until someone puts the manifest back or deletes the object by hand. A Namespace you do want removed now has to be deleted by hand, as we did with prune-test-b.
  • The controller keeps starting sync operations. In the first test prune-test-b started 5 sync operations in 28 s (19:18:10, :12, :14, :20, :38), and the controller logged growing waits between them ("retrying in 1.25s", "5.65s", "17.66s", "53.60s"). In the alert test it logged "Initiated automated sync" 26 times for the Application between 19:44:20 (the initial sync) and 20:17:16.

kubegap after the fix

A full kubegap v0.3.0 run at 19:37:43: its list prune_with_namespace_or_crd went from 9 Applications to 8 (istio is gone), and the finding argocd.prune-manages-namespace-or-crd.cluster is still fail for the 8 Applications whose Namespace now carries Prune=false,Delete=false. The summary code counts the tracked kinds of Applications with prune and does not read the annotation (lib/summaries/argocdctl.py lines 173–174 and 289). We filed this as a defect of the check. Until it is fixed, a fail from this check on an annotated Namespace has to be confirmed with check 1 above.

Not changed, and what stays exposed

  • preserveResourcesOnDeletion on the ApplicationSet. Deleting a directory still deletes that Application's tracked Deployments, Services and Git-declared PVCs. Only the Namespace and what Argo CD does not track in it survive, as the prune-test-b result shows.
  • Prune=confirm / Delete=confirm. Not adopted and not tested; with these values Argo CD waits for a confirmation before it prunes or deletes the object.
  • The root Application's own prune. Unchanged; root still has automated prune: true. It tracks Applications, and the 19:11 check printed no Namespace or CRD line for it.
  • The other Applications. 20 of the 22 Applications still have automated prune: true. The annotation protects the 8 Namespaces, not the Deployments, Services or PVCs those Applications declare.

Limits

  • A manual sync with force or replace was not tested. A record in the Incident Library, a public collection of incident write-ups (IL-0058, from GitHub issue argoproj/argo-cd#28630, which we did not open; it is the third case under "Public cases where Argo CD deleted resources" below), reports that Prune=false did not stop a force sync from deleting and recreating a Namespace. Our fix does not cover that path.
  • Pruning of a CRD was not reproduced. What we say about CRDs is from the Argo CD source and the Kubernetes documentation.
  • Whether argocd app sync --prune on v3.2.0 exits with resources require pruning on a Prune=false resource was not tested.
  • No Argo CD web UI screen was captured, so the result texts in this article are from the Application status and the controller log, not from the UI panel.
  • Whether the Discord message arrived was not looked at. What is confirmed is that Alertmanager holds the alert as active for that receiver.

Public cases where Argo CD deleted resources

Three public cases from the Incident Library, a public collection of incident write-ups, in which Argo CD deleted resources after a configuration change. The summaries below are the library's records; we did not open the three linked sources. None of them is a Namespace manifest dropping out of a kustomize render; the triggers differ.