The busiest river crossing in Qinghe town was called Three-Plank Bridge. Every morning, cargo boats, fishing skiffs, and passenger ferries jostled in the bend, waiting to pass through the lock. The lock was narrow — only one boat at a time. Let them all in at once and hulls would scrape, poles would tangle; at best scratched paint, at worst a capsized boat.

The old lock-keeper was surnamed Lu. People called him Uncle Lu. He could not read, but in forty years of tending the lock there had never been an accident. His secret was a bamboo tube and a handful of copper coins.

The tube hung on the lock-gate post, mouth facing up. Every incense-stick of time, Uncle Lu dropped one copper coin into the tube. It did not matter whether boats were waiting — when the time came, in went the coin. The tube held at most ten coins. Once full, he stopped; extra coins went back in his pocket.

To pass the lock, there was only one rule: take a coin from the tube and hand it to Uncle Lu. Coin in the tube? Take it and go. No coin? Pull aside and wait until Uncle Lu dropped the next one.

On ordinary days boats were few and the tube was always full. One boat arrived, took a coin, passed through. A second came, took another, passed too. Ten boats in a row could pass without stopping — ten coins, ten passages. But the eleventh had to wait for Uncle Lu’s next scheduled drop.

One year the river flooded and boats from three upstream docks all crowded into Three-Plank Bridge. The boatmen cursed and shouted at Uncle Lu to open the lock. Uncle Lu did not flinch. “Coins in the tube, you pass. No coins, you wait. The river may be high, but the lock is still the same width.”

A young boatman protested: “Why should your coin-dropping speed be the law?”

Uncle Lu pointed at the lock gate. “See those stone piers, one on each side? When a boat passes, the current squeezes through and pushes against them. Those piers can take only so much force. One coin per incense-stick is the speed the piers can bear. You think it’s slow. I think it’s slow. But the piers don’t think it’s slow — and if the piers collapse, nobody passes.”

The boatman pressed: “When there are no boats at dawn, aren’t you wasting coins dropping them into an empty tube?”

“Not wasting. Saving,” said Uncle Lu. “No boats in the morning, the tube fills to ten. At noon a wave of boats arrives, the first ten pass in a burst, no waiting. That is what I call: rest when you can, move when you must. Without the tube, every boat would wait one incense-stick, even if the river were empty.”

The town’s bookkeeper overheard and stroked his beard. “Clever. The coins are tokens, the tube is capacity, one incense-stick is the rate, ten is the ceiling. What you have here is—”

Uncle Lu cut him off. “What I have here is no capsized boats.”

— By now you have probably recognized it: the bamboo tube is a token bucket, the copper coins are tokens, one coin per incense-stick is the refill rate, the ten-coin limit is the bucket capacity (the burst size), waiting when the tube is empty is throttling, and the stone piers’ structural limit is the downstream service’s processing capacity. This is the story of token bucket rate limiting.

What it is

The token bucket is one of the most widely used traffic-control algorithms. The system maintains a “bucket” holding tokens. Every request must consume one token to proceed; if the bucket is empty, the request is either queued or rejected (HTTP 429 Too Many Requests). Tokens are added at a fixed refill rate, and the bucket has a maximum burst size.

There are only two core parameters:

1. Rate: how many tokens are added per second — this sets the steady-state maximum throughput.
2. Burst: how many tokens the bucket can hold — this sets how many requests can pass in an instant.

The difference from the leaky bucket: a leaky bucket releases requests at a constant rate, perfectly smoothing traffic; a token bucket allows all accumulated tokens to be consumed at once, naturally supporting bursts. The two are complementary — an API gateway commonly uses a token bucket for coarse rate limiting while a backend queue uses a leaky bucket for fine traffic shaping.

In Kubernetes, the API server enforces priority and fairness (APF) on each client — essentially a multi-level token bucket. Envoy’s local rate limit filter and global rate limit service are token-bucket implementations. Nginx’s limit_req uses a leaky bucket.

Why it matters

A system without rate limiting is like a river crossing without a lock gate — clear skies, no problem; flood day, total disaster. A single runaway client can fire an avalanche of requests, overwhelm the service, and drag down every other tenant. This is the noisy neighbor problem.

The elegance of the token bucket is that two parameters solve two tensions at once:

• Steady throughput vs. burst tolerance: the rate controls average speed while the burst allows short peaks — far more flexible than a hard “N per second” cap.
• Backend protection vs. user experience: accumulated tokens let occasional spikes pass instantly, invisible to the user; only sustained overload triggers waiting or rejection.

On multi-tenant cloud platforms, rate limiting is the first line of fairness. Nearly every API on AWS, GCP, and Azure has token-bucket-based throttling, and the error message reads ThrottlingException or 429 — that is Uncle Lu’s copper coin.

Metaphor mapping

• Three-Plank Bridge crossing — API gateway / service entry point
• Bamboo tube — token bucket
• Copper coins — tokens
• One coin per incense-stick — refill rate
• Ten-coin tube limit — burst size / bucket capacity
• Taking a coin to pass — request consuming a token before being admitted
• Empty tube, wait — throttling (429 Too Many Requests)
• Dawn coins accumulate, noon burst passes — burst capability: tokens accumulate during idle time, burst through during spikes
• Stone piers’ structural limit — downstream service’s actual processing capacity
• Flood bringing boats from three docks — traffic spike / noisy neighbor
• “Coins are tokens, tube is capacity” — the bookkeeper’s abstraction = engineer’s model