Exit code 143: why a graceful shutdown still says Error

You searched for an exit code, which usually means something broke. Here is the twist with this one: exit code 143 is, most of the time, the sound of something working. A container ended, kubectl describe stamped it reason: Error, someone opened an incident channel, and what actually happened was a shutdown that was requested and politely obeyed.

That "most of the time" is doing real work, though. The same number shows up when a liveness probe is quietly restarting your service every forty seconds, and when your autoscaler is thrashing, and when a CI runner tears down jobs mid-write. The number tells you how the process ended. Whether that ending was routine or a problem depends on two questions it cannot answer by itself: who asked for the shutdown, and were they supposed to?

This post is the map for those two questions, plus the one genuinely confusing bit of Kubernetes behaviour that sends people here in the first place: why a healthy, graceful exit still gets labelled Error.

What 143 actually says

Exit codes above 128 conventionally encode death by signal: 128 plus the signal number. 143 is 128 + 15, and signal 15 is SIGTERM, the polite half of the kill vocabulary. Unlike SIGKILL (whose deaths report as 137), SIGTERM can be caught and handled: the process gets to flush buffers, close connections, finish the in-flight request, and leave on its own terms. A 143 means it took that exit while the offer stood: it responded to the polite signal instead of outliving it. Some runtimes make the convention explicit: the JVM, famously, traps SIGTERM and exits with status 143, which is why half the search results for this number are Java stack overflows.

Now the labelling gotcha. Unless the runtime supplies a more specific verdict (OOMKilled being the big one), Kubernetes marks a terminated container reason: Error for any nonzero exit code, signal-encoded or not. So the record of a completely healthy graceful shutdown looks like this:

kubectl get pod <pod> -o jsonpath='{.status.containerStatuses[0].lastState.terminated}'

{ "exitCode": 143, "reason": "Error", "startedAt": "...", "finishedAt": "..." }

That "Error" is a formality, not a verdict. It means "nonzero", nothing more. The diagnosis lives in the exit code, the timestamps, and the question of who sent the signal.

The shutdown sequence 143 belongs to

Every orderly container shutdown in Kubernetes runs the same script (forced deletions and custom stop signals aside). The kubelet decides a container must stop (a pod deletion, a rollout, a node drain, a failed liveness probe), and the clock starts immediately: terminationGracePeriodSeconds, thirty seconds by default, covers everything that follows. The pod's preStop hook runs first, if it has one, inside that window. Then the kubelet sends SIGTERM to the container's main process. If the process exits during the window, the record shows whatever status it left: 143 where the signal convention reports the death, a plain 0 from apps that treat shutdown as success. If the clock runs out first, the kubelet escalates to SIGKILL, and the record says 137 instead.

Which means 143 and 137 are not two different errors. They are the two endings of one sequence, and the ratio between them is a health signal in itself. A service that exits 143 during every rollout is answering its shutdowns (whether it drains them cleanly is a further question, taken up at the end of this post). A service that always exits 137 during rollouts is either too slow to stop or never acting on the signal at all, and is likely dropping whatever was in flight on every deploy without anyone noticing, because rollouts are not incidents.

When 143 is the system working

Kubernetes asks containers to stop constantly, on purpose. A deployment rollout terminates every old pod. The horizontal autoscaler scales in after the morning peak. A node drain for an upgrade evicts everything on the node. A Job gets deleted, a docker stop or docker compose down lands in local dev. By default all of it is SIGTERM first, and from well-behaved containers it produces 143s and clean zero exits.

So the first diagnostic move is correlation, not investigation: line the finishedAt timestamp up against what the cluster was doing. kubectl rollout history, the pod's events, the node's drain window, the autoscaler's scale events. If the 143 coincides with an operation that terminates pods, you are looking at the system working, and the only follow-up worth doing is the rollout-health one above: confirm it was 143 and not 137, then move on.

Most exit codes describe a failure. 143 usually describes an agreement.

When 143 is a diagnosis

The 143s that matter are the ones nothing accounts for. The sender to check first is the kubelet acting on a failed liveness probe: probe fails its threshold, kubelet restarts the container, and a process with working signal handling exits 143 on the way down. The signature is regularity, and the crash-loop playbook applies directly: a lifetime that is the same N seconds on every restart is probe arithmetic signing its work, whether the exit code is 143 or 137.

Beyond probes, unexplained 143s tend to trace to an actor you have not considered yet: an autoscaler flapping between scale-up and scale-down, a CI system tearing down its own job pods on timeout, a cluster-autoscaler consolidating nodes, a colleague's kubectl delete in the wrong terminal. The exit code is identical in every case; the difference is entirely in whose decision preceded it. That is not a weakness of the diagnosis, it is the diagnosis: a 143 with no matching decision in the cluster's history is how you find automation you did not know you had.

The PID 1 problem, or why your app never gets the memo

Here is the sharpest edge in this whole territory, and the reason some teams have never seen a 143 from their own services. The main process of a container runs as PID 1 in its own process namespace, and the kernel treats PID 1 specially: default signal behaviour does not apply. A normal process that never installed a SIGTERM handler dies to the default action; for PID 1, the kernel simply discards any catchable signal it has not installed a handler for.

The consequence: a container whose entrypoint is a naive shell wrapper, or an app that never registered a handler, ignores the polite request entirely, sits out the full grace period, and dies 137. Every rollout, every drain, every kubectl delete hangs for thirty seconds and ends in the impolite kill. The fixes are boring and absolute: exec the real process from entrypoint scripts so it becomes PID 1 with its own handlers, use a minimal init like tini as the entrypoint, or rely on a runtime that handles SIGTERM (the JVM, nginx, most mature servers do).

A repro you can run

Two pods, identical liveness probes pointed at a port nothing listens on. The only difference is a signal handler. We ran exactly this pair before writing this post:

apiVersion: v1
kind: Pod
metadata:
  name: term-handled
spec:
  containers:
    - name: app
      image: busybox:1.36
      command: ['sh', '-c', 'trap "exit 143" TERM; while true; do sleep 1; done']
      livenessProbe:
        tcpSocket:
          port: 9999
        initialDelaySeconds: 5
        periodSeconds: 5
        failureThreshold: 2
---
apiVersion: v1
kind: Pod
metadata:
  name: term-ignored
spec:
  containers:
    - name: app
      image: busybox:1.36
      command: ['sh', '-c', 'sleep 3600']
      livenessProbe:
        tcpSocket:
          port: 9999
        initialDelaySeconds: 5
        periodSeconds: 5
        failureThreshold: 2

Apply both, wait a minute, then read each record with the lastState.terminated command from above. term-handled shows exit code 143 with a lifetime of roughly ten seconds: the probe's second failure lands at 10s, SIGTERM arrives, the trap fires, done. term-ignored shows exit code 137 with a lifetime of about forty seconds: same kill decision at 10s, but PID 1 has no handler, so the kernel discards the signal, the thirty-second grace period expires in silence, and SIGKILL finishes the job. Same probe, same pod shape, and the presence of one trap is the entire difference between a graceful shutdown and a hung one. Both records, note, say reason: "Error".

The honest limit: graceful is not the same as harmless

A 143 proves the process obeyed the shutdown request. It does not prove the shutdown was clean from the user's point of view. Endpoint removal and SIGTERM race each other during pod termination, so a container that exits instantly on SIGTERM can still drop the requests that were mid-flight or newly routed to it. Real shutdown hygiene is work the exit code cannot see: stop accepting new work, drain what is in progress, and only then exit, with terminationGracePeriodSeconds sized to match. And one more honest caveat: 128 + signal is convention, not law. A process can exit with status 143 voluntarily, as the JVM does by design, so the number is strong evidence of a SIGTERM death, not proof.

Where we fit in

The skill in this post is not memorising that 143 means SIGTERM. It is the move behind it: taking an ambiguous signal, asking who sent it and whether they were supposed to, and reading the cluster's history until the answer is evidence rather than a guess. That move is the same one that carries the 137 investigation and the crash-loop tree, and it is precisely what separates engineers who have operated Kubernetes from engineers who have read about it.

You can practise it anywhere the repro above runs. Proving it is the harder part, and it is the part hiring actually cares about. On SkillBricks, working a live failure end to end becomes a verified brick on your wall: the diagnosis path, not just the answer. If you knew about the PID 1 rule before this post told you, you have something worth proving.