Telling a flaky test vs infrastructure failure apart comes down to who stopped the job: exit code 137 or 143, a runner that lost communication with the server, a network or registry error, or a full disk points at the machine, while an assertion message, a test runner timeout, or a failure that depends on order or shared state points at the test. Re-running on a fresh machine is the right call for the first group and hides a bug in the second.
Read the failed step from the bottom up. Who ended the process decides the branch; whether it moves on a fresh machine confirms it.
One job is red and someone has a finger on the re-run button. Whether that is the right move depends on a single question: did the machine fail, or did the test? The log almost always says, if you know which lines to read.
This page is the decision for one failure. Finding every flaky test in a repository is find flaky tests in GitHub Actions; fixing a test once you know it is flaky is GitHub Actions flaky tests that pass on retry.
Read the failed step from the bottom
Start with the last lines of the step that failed, not the first error you see while scrolling. The last line says how the process ended, and that is the first branch of the tree: a test runner that finished and reported failures exits with its own code, usually 1, while a process stopped from outside exits with 128 plus the signal number.
Then note which step failed. A failure in checkout, dependency install, image pull or browser download happened before a single test ran, so it cannot be a flaky test, whatever it looks like.
# the failed steps only, last lines first gh run view <run-id> --log-failed | tail -n 40 # which step failed, by name gh api "repos/{owner}/{repo}/actions/jobs/<job-id>" \ --jq '.steps[] | select(.conclusion == "failure") | .name' Signals that point at the machine
These end a job from outside the code. Each has a page in the failures library that covers causes and fixes; the table is the lookup. Exit codes are 128 plus the signal: 137 is SIGKILL (9), 143 is SIGTERM (15).
A canceled step goes through the same ladder on purpose. GitHub documents that the runner sends SIGINT, waits 7500 ms, sends SIGTERM, waits 2500 ms, then kills the process tree, so a 143 on a run someone canceled, or one that hit timeout-minutes, is the runner doing its job.
| What the log says | What happened | Read next |
|---|---|---|
Process completed with exit code 137. | SIGKILL, on a runner almost always the out-of-memory killer | Exit code 137 |
| exit code 143 | SIGTERM: a cancel, a timeout or a machine being taken away | Exit code 143 |
lost communication with the server | The runner process stopped answering GitHub mid-job | Runner lost communication |
The runner has received a shutdown signal. | The agent or its machine was stopped while the job ran | Runner shutdown signal |
No space left on device | The runner disk filled and the next write failed | No space left on device |
ECONNRESET, EAI_AGAIN, 429 Too Many Requests in an install or pull step | The network path or a registry refused or dropped the request | ECONNRESET, Docker Hub rate limit |
Cancellation sequence from the workflow cancellation reference on docs.github.com, read 2026-11-12. Log strings are the ones quoted on the linked pages.
Signals that point at the test
The test runner finished and is telling you what it saw. An assertion message with expected and received values (expect(received).toBe(expected) in Jest, Vitest and Playwright, AssertionError in Python and Node) is the test disagreeing with the code. A runner-level timeout such as Playwright's Test timeout of 30000ms exceeded. is the test waiting for something that did not happen in time, which on a healthy machine is usually a race or a fixed wait.
The rest show up only across runs. Passing alone and failing in the suite is shared state. Failing in one order and passing in another is order dependence. Failing only under parallel workers is contention between tests. The three probes that separate them are on GitHub Actions flaky tests that pass on retry.
The middle, where both are true
Some failures carry a machine signal and a test cause. A test step killed with 137 hit the machine's memory ceiling, but if peak memory grows every week, a leak in the code under test is the real defect. A test timeout on a two-core runner may be CPU contention from too many workers rather than a race. An ECONNRESET raised inside a test that calls a real external service is a test design problem: the test depends on a network it does not control.
The tie-breaker is a fresh machine. Re-run the job, which on GitHub-hosted runners means a new virtual machine. The same failure in the same place again is deterministic, so read it as a bug. A failure that moves or vanishes was environmental or flaky, and the signal above tells you which.
# re-run only the failed jobs, on new machines, and wait for the result gh run rerun <run-id> --failed gh run watch <run-id> --exit-status && echo "passed on a fresh machine: environmental or flaky" # still red: compare where each attempt stopped gh run view <run-id> --attempt 1 --log-failed | tail -n 20 gh run view <run-id> --attempt 2 --log-failed | tail -n 20 GitHub documents that each GitHub-hosted runner, single-CPU runners aside, is a new virtual machine (GitHub-hosted runners concepts page, read 2026-11-12). The
--failed,--exit-statusand--attemptflags are from the gh 2.102.0 manual.
The tree as a script
The script reads the failed steps of one run and prints the first machine signal it finds, else the first test signal, else asks you to read the log. The machine check runs first on purpose: a test that fails after the runner ran out of disk is not evidence about the test.
It is a first pass, not a verdict. Extend the patterns with the messages your own stack prints, and keep the output with the run so the decision is recorded next to the evidence.
#!/usr/bin/env bash # Usage: triage.sh RUN_ID (inside the repository, or with GH_REPO set) log=$(gh run view "$1" --log-failed) machine='exit code 137|exit code 143|lost communication with the server|received a shutdown signal|No space left on device|ECONNRESET|EAI_AGAIN|ETIMEDOUT|429 Too Many Requests' in_test='AssertionError|expect\(received\)|Test timeout of [0-9]+ms exceeded' if sig=$(grep -m1 -oE "$machine" <<<"$log"); then echo "machine: $sig" elif sig=$(grep -m1 -oE "$in_test" <<<"$log"); then echo "test: $sig" else echo "unclassified: read the failed step" fi FAQ
How do you tell if a test failure is flaky or a real bug?
Run the same commit again on a fresh machine. A failure that repeats in the same place is deterministic and almost certainly a real bug. One that moves or disappears is flaky or environmental, and the log tells you which: an exit code like 137 or a runner message points at the machine, an assertion or test timeout points at the test.
What does exit code 137 mean in GitHub Actions?
The process was killed with SIGKILL, since 137 is 128 plus signal 9. On a CI runner that is almost always the kernel out-of-memory killer, so nothing in your code threw. Lower the job's peak memory or give it a bigger machine.
Should I just re-run failed GitHub Actions jobs?
Only when the failure is the machine's: an out-of-memory kill, a lost runner, a full disk or a network error before the tests ran. Re-running an assertion failure until it passes ships the bug and erases the evidence. Record every re-run so the rate stays visible.
Why does my test fail in CI but pass locally?
Because CI is a different machine: fewer cores, less memory, a cold cache, other tests running in parallel and a different timezone or locale. Check the failed step for a machine signal first. If there is none, the test depends on something your laptop provides and the runner does not.
