A load balancer is easy to describe: requests come in, and you send each one to one of several backends. The interesting questions start right after that sentence. Which backend? How do you know one of them is dead? What do you do with a request that was already sent to a dead one? And how do you change the list of backends without dropping the traffic that is flowing right now?

loadbalancer is my learning project for those questions. It is an HTTP reverse-proxy load balancer in Go, using only the standard library and fsnotify. It has three balancing strategies, active health checks, a passive circuit breaker per backend, retries for idempotent requests, config hot reload and Prometheus metrics.

What it does

The request path is short enough to show in full:

request ─► pick(): candidates = healthy ∧ breaker ready ∧ not tried yet strategy.Pick(candidates) → breaker.Acquire() ─► forward(): ReverseProxy to the backend transport error → breaker.Failure(), retry if idempotent 5xx → breaker.Failure(), response is passed through otherwise → breaker.Success() 

A backend is a candidate only if the health checker says it is healthy, its circuit breaker is ready, and this request has not tried it yet. The strategy picks one of the candidates, and the request is forwarded through httputil.ReverseProxy.

Next to the proxy port there is an admin port with /metrics in the Prometheus text format and /status, a JSON view of the pool.

What I built from scratch, and what it teaches

Retries around httputil.ReverseProxy. By default, when the proxy cannot reach a backend, it writes a 502 to the client, and then it is too late to try another one. Here the error handler records the failure instead of writing a 502, so the request can still go to another backend. Retries are deliberately narrow: only GET, HEAD, OPTIONS, TRACE, PUT and DELETE requests without a body, and only when the backend could not be reached, so nothing has been written to the client yet. A 5xx response is never retried; it is passed through.

Balancing algorithms. There are three strategies: round robin, weighted, and least connections with rotating tie-breaks. The weighted one is nginx's smooth weighted round robin. With weights 5, 1, 1, a naive implementation gives a a a a a b c: five requests in a row to the same backend. The smooth version gives a a b a c a a, which keeps the same proportions and spreads them out.

Two ways to detect a dead backend. Active checks probe a path such as /healthz on a timer. One goroutine probes every backend concurrently each interval, and thresholds stop one lost probe from making the pool flap. Passive checks watch real traffic, and they work even when the backend has no health endpoint. That is the circuit breaker.

The circuit breaker state machine. After failure_threshold consecutive failures, the breaker goes from closed to open and the backend stops receiving traffic. Once cooldown has passed, exactly one trial request is let through: the half-open state. A success closes the breaker, and a failure opens it again. Two methods keep this correct under concurrency: Ready() has no side effects and is used to filter candidates, and Acquire() reserves the single half-open slot. When the health checker sees a backend recover, its breaker is reset too.

Reloading without dropping traffic. Apply validates the new config first, so an invalid file is rejected and the old config stays. It then reuses the *Backend values whose URL is unchanged, keeping their in-flight count, health and breaker state, and restarts the health checker. Both SIGHUP and file changes trigger it, and file changes are debounced. One detail I did not expect: the watcher looks at the directory, not the file, because editors replace files by renaming.

How to use it

Install the binary with Go:

go install github.com/AbdeldjalilChougui/loadbalancer/cmd/loadbalancer@latest 

Or run it from a clone of the repository:

cp lb.example.json lb.json # edit the backend URLs go run ./cmd/loadbalancer -config lb.json curl localhost:8080/ # proxied curl localhost:9090/metrics # Prometheus metrics curl localhost:9090/status # JSON view of the pool kill -HUP <pid> # reload (saving the file also reloads) 

The config is one JSON file:

{ "listen": ":8080", "admin": ":9090", "strategy": "weighted", // round_robin | weighted | least_conn "retries": 2, // extra attempts for idempotent requests "backends": [{"url": "http://127.0.0.1:9001", "weight": 3}], "health_check": {"path": "/healthz", "interval": "5s", "timeout": "1s", "healthy_threshold": 2, "unhealthy_threshold": 2}, "circuit_breaker": {"failure_threshold": 5, "cooldown": "10s"} } 

Leave health_check.path empty to turn off active checks, and set failure_threshold to 0 to turn off the breaker.

The tests need nothing external:

go test -race ./... 

They all run against httptest backends. They cover strategy order and distribution, least connections with a blocked backend, breaker state transitions on a fake clock, a retry around a dead backend, health checks taking a backend out and putting it back, a reload that keeps state, the fsnotify watcher and the metrics output.

Things to know

This is a learning project. The README lists what a production load balancer has and this one does not:

  • It serves HTTP/1.1 and HTTP/2 to the client through net/http only. There is no TLS termination config: put it behind something, or add ListenAndServeTLS. There is no WebSocket-specific handling beyond what ReverseProxy already does, and no TCP (L4) mode.
  • No sticky sessions, no consistent hashing, no slow start for recovered backends, and no outlier detection beyond consecutive failures.
  • Changing listen or admin needs a restart. The reload logs a warning.
  • Metrics are written by hand in the Prometheus text format, and there are no latency histograms.
  • The breaker counts any 5xx from the backend as a failure. That includes a 500 caused by one bad request.

What I learned

The first lesson is that a retry is a promise about the request, not about the proxy. You can only retry when you know nothing reached the client and the request is safe to send twice. That is why the list of methods is short and why bodies are excluded. Being generous here is how a load balancer ends up creating two orders.

The second is that active and passive health detection are not alternatives. A probe every five seconds is too slow to protect the requests arriving in between, and a breaker alone never notices a backend that receives no traffic. Each covers the other's blind spot.

The third is about state. The simple way to reload is to throw away the pool and build a new one. That resets every breaker, every health result and every in-flight counter, at the exact moment someone is changing things. Reusing the unchanged backends is more code, and it is the difference between a reload and a small outage.

And the smooth weighted round robin is a nice example of an algorithm where the obvious version is correct on average and wrong in practice.

Get the code

The source, the example config and the tests are on GitHub: https://github.com/AbdeldjalilChougui/loadbalancer. It is MIT licensed. The breaker and the strategies are small files, and they are a good place to start reading.

Originally published at abdeldjalilchougui.biz.