Beneath the precipitous cliffs north of Qingyan Town lay a celebrated deep well known far and wide as the Dragon Tongue Well.

Its shaft reached one hundred and eighty fathoms into the bowels of the mountain. Carved entirely through bedrock of adamantine blue granite, the shaft was narrower than a slit in a fortress wall—its mouth permitting only a single large vermillion-lacquered cask to pass up and down. Yet its water was so sweet and piercingly crisp that thirty-six tea houses, ten distilleries, and dozens of great estates throughout the valley depended on it for their first morning brew.

Old Master Song, who had guarded the wellhead for forty years, had lived through the Dragon Tongue Well’s most perilous dawns.

In the old days, there was no order at the wellhead. The moment the dawn bell tolled, runners and porters from every quarter descended upon the stone terrace in a shouting swarm. One tea master’s boy screamed that his kettles were dry; a brewer hollered that his fermenting vats would spoil. Thirty coarse hemp ropes were hurled into the dark, razor-narrow shaft all at once.

Down in the black throat of the gorge, wild winds howled. The ropes tangled into a hopeless knot midair; three iron-banded buckets shattered against the rocks; and the snarled lines jammed the wooden windlass tight. By mid-morning, the deep shaft was choked with severed twine and splintered stave-wood. The town’s hearths stood cold, and the tea ovens blew bitter grey smoke.

After that disastrous morning, Master Song erected a black-lacquered rack beside the windlass, studded with thirty-six sliding brass rings. Directly above the well shaft, he suspended a single eight-bushel cask hanging from an enormous braided hawser.

He painted the new law on the cliff face in lime wash: “When the brass ring hangs, all ropes must hang back.”

The following morning before dawn, mist still clung to the gully. A frantic boy from the first teahouse arrived clutching two empty pewter kettles: “Master Song! Our early-spring tea needs the first drawing! A matter of life and death!”

Without glancing up, Master Song reached over to the wooden rack and slid the brass ring engraved with Dragon Tongue Spring straight up into its lock. With the ring raised, Song kicked the windlass catch. The great eight-bushel cask plunged into the dark chasm with a hollow roar.

Within breaths, thirty runners from neighboring wineries and inns arrived gasping for breath, hugging their empty jars and shouting to cast down their ropes.

Master Song rapped his long pipe against the frame: “Can’t you see the brass ring hanging? The great cask is already drawing from the vein a thousand cubits below. Coiling your ropes, and sit yourselves down on the stone benches!”

The runners traded glances, but none dared break the rule. Two dozen apprentices lined up their empty vessels along the flags and sat down in an orderly row upon the long stone benches, waiting in silence.

After half the burning of an incense stick, a low rumble echoed from the deep. The heavy windlass groaned as the brimming cask crested the lip of the well. Vapor billowed out into the morning chill like white clouds, and inside sat eight full bushels of snow-clear water.

Resting his arm against the cask rim, Master Song raised a long-handled copper ladle. In one fluid rhythm, he scooped the sparkling water into each of the twenty-odd waiting barrels. One ladle per vessel, without a drop wasted. Within minutes, every barrel stood brimming. The boys hoisted their carrying poles, bowed in gratitude, and scattered down the mountain trails.

Only when the last vessel was filled did Master Song slide the brass ring back to its resting place. Had a thirty-first courier arrived that instant, the ring would have risen anew, and the cask would have begun another descent.

Sitting upon the stone rim and tapping out his pipe embers, Master Song puffed a blue cloud toward his young apprentice:

“At a narrow well, the worst folly is for thirty throats to holler at once. If every runner drops his own rope, the shaft will jam till midday. Let the first man run the errand; as for all who seek the very same draft, seat them on the benches and let them share the single haul when it returns.”

Pausing, the old well-keeper squinted toward the taut hawser, adding quietly:

“Yet heed this well: if that hawser snaps halfway down, or the cask scoops up mud, all thirty boys on those benches will go home empty-handed together.”

— By now, you have probably recognized the pattern: this is Request Coalescing, known throughout modern backend engineering as Singleflight.

What it is

In distributed architectures and high-concurrency microservices, when a popular cached key expires (Cache Miss) or a cold cache is rebooted, an avalanche of concurrent requests often strikes the backend within the exact same millisecond. This phenomenon is known as a Cache Stampede or the Thundering Herd Problem.

Without safeguards, hundreds or thousands of concurrent threads or coroutines will simultaneously dispatch the exact same query downstream to the primary database or an expensive remote service. This triggers:

• Exhaustion of database connection pools within milliseconds;
• Sudden spikes to 100% database CPU, provoking cascading timeouts;
• Cascading brownouts across downstream dependencies.

Request Coalescing (Singleflight) is an exceptionally clean concurrency pattern—most famously implemented in Go’s standard extended library golang.org/x/sync/singleflight and Envoy’s HTTP Request Collapsing:

1. The In-Flight Map: The proxy or application maintains an in-memory map keyed by the unique identifier of the requested query or resource.
2. The Leader: When a request arrives, it looks up its key. If no identical request is currently “in flight”, this request becomes the Leader and dispatches the genuine call to the expensive downstream resource.
3. Coalesced Waiters (Followers): While the Leader’s call is outstanding, any subsequent requests arriving with the identical key refrain from calling the downstream service. Instead, they suspend execution (in Go, by listening on a shared completion channel) and wait for the Leader’s return.
4. Broadcast Dispatch: The instant the Leader completes its downstream execution and receives the result (or an error), the system copies and delivers that single outcome to every waiting follower concurrently.
5. Flight Cleanup: Once delivery is complete, the key is evicted from the in-flight map. Any new request arriving thereafter will launch a brand-new Leader cycle.

Why it matters

Singleflight differs fundamentally from traditional caching or distributed locking:

• Zero Persistent Memory Overhead: Unlike a Redis cache, Singleflight retains zero data long-term. It operates purely within the transient concurrency window while a call is active. Once resolved, memory is immediately reclaimed, completely sidestepping LRU/TTL cache-sizing concerns.
• A Fortress for Fragile Backends: During sudden viral traffic bursts, container cold-starts, or cache invalidation events, thousands of redundant read operations per second are collapsed at the edge into a single outbound query, defusing thundering herds before they reach storage.

Engineering Pitfalls & Tradeoffs:

• Shared Fate (Error Blast Radius): As Master Song warned, the Leader’s fate is shared by all followers. If the Leader encounters a timeout, network drop, or panic, all waiting requests fail simultaneously with the exact same error.
• Head-of-Line Blocking: If a slow Leader stalls in downstream I/O, every piggybacking follower is held hostage. Industrial implementations must provide context-cancellation hooks (DoChan + context.WithTimeout), allowing followers to abandon the wait or dispatch a hedged fallback if the Leader takes too long.
• Strictly for Idempotent Reads: Request coalescing applies exclusively to safe, idempotent queries. State-mutating operations (such as payments, account debits, or inventory decrements) must never be coalesced.

Metaphor mapping

• The narrow, 180-fathom well shaft → The shared, high-latency downstream bottleneck (database / external API)
• Swarms of teahouse apprentices clamoring for water → Concurrent identical requests during a cache stampede
• Thirty tangled ropes jamming the shaft → Exhausted database connection pools and collapsed throughput
• Master Song’s sliding brass rings on the rack → The in-memory in-flight map tracking active calls
• The first runner raising the brass ring → The Leader request executing the underlying remote call
• Subsequent runners sitting quietly on the stone benches → Followers blocked on a shared completion channel / future
• The single 8-bushel cask ladling water into every vessel → A single query result broadcast to all waiting callers
• Sliding the brass ring down once the cask is emptied → Evicting the completed key from the flight map
• A broken hawser leaving all thirty boys thirsty → Shared fate / error blast radius propagating across all waiters
• A sluggish windlass delaying all seated runners → Head-of-line blocking by an unoptimized or slow Leader