Design a Rate Limiter
MediumSystem design · Traffic Control · Stripe
Design a rate limiter for an API platform. It must enforce per-key, per-tenant and per-IP limits across a fleet of gateway instances, add well under a millisecond to each request, and keep working when its own dependencies fail. Focus on the algorithm, where the counters live (shared Redis versus local state), atomicity, failure modes and rule distribution. Out of scope: the API gateway itself and DDoS protection at the network layer.
Requirements
- Make an allow/deny decision on every request in under 1ms p99
- Be correct across many gateway instances: atomic check-and-update, no read-then-write race
- Support bursts, multiple simultaneous keys (API key, IP, global) and rules updated at runtime
- Define behaviour when the shared store is slow or down: fail open versus closed, and what still applies locally