Pattern of the week: Circuit Breaker

Every automation I've built that talks to another system has, at some point, been let down by that system. The API goes slow. The database hangs. The service returns a 503 for twenty minutes while someone somewhere restarts a pod. The question isn't whether it happens. It's what your workflow does while it's happening.

Most people reach for retries. Retries are good. But retries are built for transient failures, the blip that clears in a second or two. When the thing on the other end is properly broken, retries just mean you hammer it harder, queue up more waiting work, and make the outage worse for everyone. That's the problem the circuit breaker solves.

The short version

A circuit breaker sits in front of a dependency and counts failures. Once failures cross a threshold, it "opens" and stops calling the dependency at all. Calls fail immediately instead of hanging. After a cooldown, it lets one probe request through. If that works, traffic flows again. If not, it stays open for another round.

Three states: closed (normal), open (fail fast), half-open (testing the water). That's the whole idea.

Rendering diagram…

When to use it

Reach for it anywhere slow is as bad as down: third-party APIs, a database query that sometimes locks up, an internal service you don't control. If a dependency going sideways can tie up your workers or stall a whole queue of approvals, you want a breaker in front of it.

You don't need one for a call that already fails fast and cheaply. Don't add machinery for the sake of it.

The pitfall I see most

One global breaker for "all external calls." It looks tidy. Then one flaky service trips it and every integration in the workflow goes dark, including the ones that were perfectly healthy. Keep one breaker per dependency, ideally per operation. The point is to contain the damage, not spread it.

A close second: not deciding what counts as a failure. A 5xx or a timeout, yes. A 4xx usually means you sent something wrong, and tripping the breaker on that just hides your own bug.

Pair it with a plan B

An open circuit only helps if your workflow knows what to do next. Serve a cached value, return a safe default, park the item for later, or fail with a clear message. Pick one on purpose for each call site. And make the state visible: "circuit opened for the payments API" should be an alert someone sees, not a line buried in a log.

The full pattern, with the flow diagram, the thresholds I'd start with and how it fits with retries and timeouts, is here: Circuit Breaker.