A gateway that fails fast instead of dragging everyone down
Three services behind one entry point. When one of them slows to three seconds, the other two must stay up — and so must the gateway.
The business problem
You have Orders, Inventory and Pricing behind a gateway. Inventory degrades. Without protection, the gateway's threads fill up waiting, and every route goes down with it.
Build the gateway, then break Inventory on purpose and watch what happens. The project is not finished until the failure is boring.
What you will have at the end
- Per-route timeouts that add up to a bounded total
- A circuit breaker with a fallback you chose
- Bulkheads, so one slow dependency cannot take the pool
- A load test that proves the above rather than assuming it
Milestones
Each one ends in something you can observe. Without that a milestone is a heading, and you have no way to know you finished.
Routing and a timeout budget
Every downstream call gets a timeout. Add them up: the total must be less than what your own caller will wait.
done whenA hung downstream returns an error within the budget rather than hanging the request.
Retry with a budget
Retries only on idempotent verbs, capped, jittered. A retry on POST is a duplicate order.
done whenA downstream that fails every time is called a bounded number of times, not indefinitely.
The circuit breaker
Open on a failure rate over a window. Half-open lets one request through to test. Decide what the fallback returns.
done whenWith Inventory down, the breaker opens and the gateway stops calling it.
Bulkheads
A separate pool per downstream. Shared pools are how one slow service becomes three.
done whenInventory at three seconds does not slow the Pricing route.
Prove it under load
k6 at a realistic rate with Inventory artificially slowed. Record p95 for every route.
done whenYou have a graph showing Orders and Pricing flat while Inventory degrades.