API Rate Limiting Explained: Why Every API Needs a Gatekeeper
Why Limit Anything
An API without rate limits is a shared resource with no admission control. One misconfigured client loop can exhaust your database connections. One scraper can consume your entire bandwidth budget. One attacker can make your service appear down. Rate limiting answers a simple question: how many requests is too many, and what do we do when someone exceeds that number.
The goal is not to punish users. It is to protect the system so that every legitimate caller gets a fair share, and to signal to automated consumers that they need to back off and retry gracefully.
The Token Bucket Algorithm
The most widely deployed algorithm is the token bucket. Imagine a bucket that holds tokens. Each request costs one token. The bucket refills at a constant rate, and it has a maximum capacity. If a request arrives and a token is available, it is consumed and the request proceeds. If the bucket is empty, the request is rejected or delayed.
Token buckets handle bursts well because the bucket capacity acts as a burst allowance. A client that has been idle accumulates tokens up to the bucket's limit and can then send a burst of requests at full speed before settling into the steady refill rate. This models real traffic patterns better than a simple fixed-window counter.
Sliding Windows and Leaky Buckets
Fixed-window rate limiting divides time into buckets — say, one-minute windows — and counts requests in each. The problem is the boundary: a client can send 100 requests at 11:59:59 and 100 more at 12:00:00, effectively sending 200 requests in two seconds while staying under a 100-per-minute limit.
The sliding window fixes this by weighing the current window's count against the previous window's count in proportion to how far into the current window we are. It is computationally more expensive but smoother.
The leaky bucket takes the opposite approach from the token bucket. Instead of allowing bursts, it enforces a steady outflow rate using a queue. Requests enter the queue and are processed at a fixed rate. If the queue fills, requests are dropped. Leaky buckets are good for traffic shaping but less forgiving of bursty patterns.
Communicating Limits to Clients
Good rate limiters tell the client where they stand through response headers. The IETF has drafted standards for RateLimit-Limit, RateLimit-Remaining, and RateLimit-Reset. When a client sees RateLimit-Remaining: 0 it should pause and wait until the reset timestamp before trying again. Returning HTTP 429 Too Many Requests without these headers leaves the client guessing.
The Edge Review explains infrastructure concepts for general readers. Rate limiting strategies vary by platform; consult your provider's documentation for production implementation.