Your pod keeps restarting, but kubectl get pods says 1/1 Running, the logs show no error, and the last exit code is 0. That pattern usually means nothing inside the container is failing. Something outside it is stopping a healthy app, and the usual suspect is a liveness probe. Run kubectl describe pod and read the events at the bottom. If you see Liveness probe failed: HTTP probe failed with statuscode: 404, the kubelet is killing your container on purpose.

I broke a small Go service on purpose to show this on a real cluster. A values update changed one setting in its Deployment while a load generator sent it steady traffic. Every output below is copied from that recording (I lined up the columns, and cut long lines where noted).

What it looked like from outside

Users saw a flicker, not an outage. Most requests succeeded, and failures came in short bursts. The app's own request count dropped to almost nothing at the same moments, because with one replica there was no pod left to serve:

Grafana: left, the app's own request rate drops to near zero in the shaded "no available pods" bands. Right, the load generator's requests show red failed tops in bursts

The pod looked fine. Look at the restart count, though:

$ kubectl get pods -n go-api NAME READY STATUS RESTARTS AGE go-api-76995dc6c8-cfskf 1/1 Running 6 (102s ago) 5m42s

Six restarts in under six minutes, for a pod that was ready every time I looked. kubectl get rs showed a new ReplicaSet about five minutes old and the old one scaled to zero, so a change to the pod template had just rolled out. Grafana's Pod Resources dashboard had caught the same pod in CrashLoopBackOff between restarts:

Grafana: container restarts for the go-api pod climb from 1 to 5 in five minutes, and the "Containers not running, by reason" panel shows CrashLoopBackOff for the same pod

Running is a snapshot. After a few restarts the kubelet waits longer before each new start (kubernetes.io, container restarts), and in those gaps the same pod shows CrashLoopBackOff. Look while a container is up and the pod reads healthy.

The logs show a clean shutdown

The previous container's last lines are where you'd expect a crash. There wasn't one:

$ kubectl logs go-api-76995dc6c8-cfskf --previous --tail 5 -n go-api ... {"time":"2026-10-26T08:41:27.687561633Z","level":"INFO","msg":"request",...,"path":"/healthz","status":404,...} {"time":"2026-10-26T08:41:27.70856972Z","level":"INFO","msg":"received signal, shutting down","signal":15} {"time":"2026-10-26T08:41:27.709079931Z","level":"INFO","msg":"server stopped"}

(I dropped two normal 200 request lines and the fields the terminal cut off.) Every line is INFO. The app served requests, answered one request for /healthz with a 404, then received signal 15 (SIGTERM) and stopped cleanly. It didn't crash. Something told it to stop.

Who stopped the container?

The exit code tells you who to ask next. kubectl describe pod has it under Last State:

$ kubectl describe pod go-api-76995dc6c8-cfskf -n go-api ... Last State: Terminated Reason: Completed Exit Code: 0 Started: Mon, 26 Oct 2026 14:10:50 +0530 Finished: Mon, 26 Oct 2026 14:11:27 +0530 Ready: False Restart Count: 6 ...

Completed and exit code 0 rule out the two usual stories. An app that crashes on its own exits with an error code, and its logs say why. A container the kernel kills for memory shows OOMKilled and exit code 137 (kubernetes.io, determine the reason for pod failure). Exit code 0 is this app's doing: it handles SIGTERM and shuts down gracefully. An app that ignores SIGTERM is killed after the grace period and shows 137 instead, so don't read the exit code alone. The events say who sent the signal.

The container also lived exactly 37 seconds.

The events name the kubelet

The events are at the very end of the same describe output:

Events: Type Reason Age From Message ---- ------ ---- ---- ------- ... Normal Killing 2m23s (x6 over 5m43s) kubelet spec.containers{go-api}: Container go-api failed liveness probe, will be restarted Warning BackOff 94s (x7 over 2m23s) kubelet spec.containers{go-api}: Back-off restarting failed container go-api in pod go-api-76995dc6c8-cfskf_go-api(...) ... Warning Unhealthy 43s (x19 over 6m3s) kubelet spec.containers{go-api}: Liveness probe failed: HTTP probe failed with statuscode: 404

(I cut the Scheduled, Started, Pulled and Created lines and the pod's UID.) The kubelet, the agent on each node that runs the containers, asked the liveness probe's path 19 times, got a 404 every time, and killed the container six times. Higher up, the same output shows the probe:

 Liveness: http-get http://:http/healthz delay=15s timeout=5s period=10s #success=1 #failure=3

That explains the 37 seconds. The first probe runs 15 seconds after the start, then one every 10 seconds, and three failures in a row trigger a kill. The probes at roughly 15, 25 and 35 seconds all got a 404, and the app was gone at 37.

Why the probe fails

An HTTP probe passes on any status from 200 to 399, and anything else counts as a failure (kubernetes.io, container probes). I opened both paths in the browser:

Browser: go-api's /healthz answers "404 page not found", while /health answers {"status":"ok"}

The app serves /health. The values update had renamed the probe's path to /healthz, the /healthz convention many projects use, but not this app. One letter, and a perfectly healthy app was killed every 37 seconds.

Liveness, readiness and startup probes do different jobs

A readiness probe decides whether a pod gets traffic. When it fails, the pod leaves the Service but keeps running. A liveness probe decides whether the container is alive. When it fails, the kubelet restarts the container. A startup probe holds the other two off until the app has started once (kubernetes.io, configure probes). Here the readiness probe on /ready passed between kills, which is why the pod kept reading 1/1.

The tempting fix makes it slower, not better

The quick fix many people try is to give the probe more patience. I raised failureThreshold from 3 to 10 with kubectl edit deploy go-api -n go-api. The restarts slowed down: ten failures ten seconds apart is about a hundred seconds per life instead of 37. The probe still asked for a path that returns 404, so the kills just came on a longer cycle. The lab's check still failed it, because it asks the probe's path the way the kubelet does.

Deleting the probe is the same mistake made bigger. The dashboards go green, but nothing checks the app any more, so the next deadlock stays up serving errors until someone notices.

The fix: point the probe at a path the app serves

I put failureThreshold back to 3 and changed the path to /health, both in kubectl edit. A one-line patch does the path:

kubectl -n go-api patch deploy go-api --type=json \ -p '[{"op":"replace","path":"/spec/template/spec/containers/0/livenessProbe/httpGet/path","value":"/health"}]'

kubectl rollout undo deploy/go-api -n go-api works too. Then:

$ kubectl rollout status deploy/go-api -n go-api Waiting for deployment "go-api" rollout to finish: 1 old replicas are pending termination... deployment "go-api" successfully rolled out $ kubectl get pods -n go-api NAME READY STATUS RESTARTS AGE go-api-6bb665f764-mqm4m 1/1 Running 0 80s

Zero restarts after 80 seconds, more than twice the old lifetime. The pod's name tells one more story: 6bb665f764 is the original ReplicaSet from before the bad change. With both settings back to what the chart sets, the pod template matched the old one again, so the Deployment scaled that ReplicaSet back up instead of making a new one (kubernetes.io, pod-template-hash).

Questions people ask next

Should a liveness probe check the database?

No. A liveness endpoint that pings a dependency restarts every pod when that dependency blips, which turns a short database outage into an outage of your whole service. Liveness should check only that this process can make progress. Dependency checks belong in readiness, if anywhere.

How do I choose timeoutSeconds and failureThreshold?

Size them so a slow moment isn't a death sentence, then check the arithmetic. Your app must start within initialDelaySeconds plus failureThreshold times periodSeconds, or it is killed before it's ready, every time. If start-up is slow or varies, add a startupProbe instead of stretching the liveness delay.

Other reasons a liveness probe kills an app

The event text tells them apart:

  • A path or port the app doesn't serve: statuscode: 404, or connection refused for a wrong port. This drill.
  • A slow start: the container dies at the same age on every restart, before it ever becomes ready. Add a startup probe.
  • A timeout under load: context deadline exceeded, and kills that come with traffic peaks.
  • A dependency check: every pod restarts at the same moment.

To reproduce this on a test cluster, patch a Deployment's probe to a path its app doesn't serve:

kubectl -n <ns> patch deploy <app> --type=json \ -p '[{"op":"replace","path":"/spec/template/spec/containers/0/livenessProbe/httpGet/path","value":"/healthz"}]'

I ran it as a drill in SnowOps Labs (disclosure: I build it).

The ladder to remember

  1. RESTARTS climbing on a Running pod is the first clue. Don't trust the snapshot.
  2. Read Last State in kubectl describe pod. Exit code 0 or Completed means read the events, not just the logs.
  3. Liveness probe failed in the events means the kubelet is the one stopping it.
  4. Open the probe's path yourself and compare it with what the app serves.
  5. Fix the probe. Don't delete it, and don't only raise the threshold.
  6. Prove it: zero restarts after more than one old lifetime.

*This is part of a series I'm writing on debugging Kubernetes in production. You can also simulate this and many other scenarios and incident in our free and open source project snowopslabs.

Github link - https://www.github.com/sagar2395/snowopslabs