Qingxi Landing sits where the Yuan River bends.

There is a tally house at the landing, and in the tally house sits an old clerk named Shen. His work is simple: record the grain that eleven villages send down from upriver into a ledger, and hand it to the provincial granary boat before dusk.

Here is the rule. Every boat of grain carries a slip, and the slip bears the hour the grain left the village — loaded at dawn, loaded at noon, loaded at dusk. What the provincial office wants is not when the boat reached the landing. It wants when the grain left. Harvest has a flood season; grain taken before it and grain taken after it do not fetch the same price.

So each day, Clerk Shen’s job is to sort the boats that arrive into three night-watches according to the hour on their slips: the dawn watch, the noon watch, the night watch.

— —

The first year, he did it stupidly.

He reasoned that the boats must eventually all be in. So he sat at the landing and waited, and only when there was no longer a single boat on the river did he begin to write.

Then came the storm upriver.

Rain fell in the mountains above Shangping, Xiaping, and Shamuao, and washed out two plank roads. The boats did not fail to come. They were slow. Slow was bearable. What was not bearable was that the provincial granary boat did not wait. The third watch drum sounded, the warehouse gate shut, and that night’s ledger was worthless.

His first wait-for-everyone finished at the fourth watch. The ledger arrived a day late, the dawn grain was recorded as noon grain, and a batch harvested before the flood season was taxed as a batch harvested after it.

That winter, the provincial granary fined him half a year of rice.

— —

From the second year, he travelled upriver and looked at the post stations.

He noticed one thing: boats run late, post stations do not. Each station has a flag. One wooden pole, one bolt of blue cloth. When a boat from that station’s district has been loaded and has departed, the station hoists the flag high. While any village is still waiting, the flag stays at half-mast.

The flags do not rise together. The fog clears early at Shangping, so its flag goes up early. Shamuao is behind the mountain and rises two hours late.

Clerk Shen drew a line on the whitewashed wall of the landing.

He did not look at the highest flag. He looked at the lowest one.

Three stations: two flags up at noon, one still pressed down at dawn. He drew the red line at dawn. Because only that pressed-down flag is saying one sentence: every boat loaded before dawn has already gone.

Once the red line passed an hour, Clerk Shen held it settled: grain loaded at that hour had either arrived, or would never arrive.

He called that line the water line.

— —

The first night he sealed the ledger by the line, and the ledger made the third watch drum.

But the line is a lie.

Not a malicious lie. A human one. Shamuao’s flag being down only means that flag has not risen. Perhaps the post rider is sick. Perhaps the fog is too thick to see the road. Perhaps a boat turned over halfway and the grain went to the fish. The line passing an hour does not mean there was no boat.

On the ninth night, a boat did cross the line.

A small boat, its slip marked dawn, arriving at daybreak. Clerk Shen’s ledger was sealed. The clay was already stamped.

He thought all night: burn the slip, or break the seal and rewrite?

Break the seal, and the provincial office would receive two dawn ledgers — a smaller one, and an amended one. He did not know what the provincial office would make of that. But he did know the grain was real. He broke the seal.

From then on he had a second rule: after the line passes, every ledger stays unsealed for one more watch.

That extra watch is called the slow seal. A boat arriving during the slow seal is folded back into whichever ledger its slip names, with a small line added: amended.

A boat too far past the line is refused even by the slow seal. Those go into a separate book called the leakage account, settled at the end of the month. The leakage account never enters the day’s three ledgers.

— —

In the third year’s rains, Shamuao’s flag did not rise for three days.

For three days the red line did not move, and Clerk Shen’s three ledgers stayed blank. Grain on the bank piled to the roof of the shed. Riders from Shangping and Xiaping came on time to ask: why are you not sealing the ledger?

He said: one flag is still pressed down.

What he did not say was that he could not tell whether that flag was standing still on its own, or had been tangled by wind.

On the fourth day the flag went up, the red line leapt from dawn to mid-afternoon in one motion, and all three ledgers caught up within the hour. In the margin Clerk Shen wrote four characters: one line, jammed dead.

— —

By the fourth year, Clerk Shen’s ledger had become the most important object at Qingxi Landing.

The provincial grain officer came once, turned the pages, and asked: how do you know you drew the line in the right place?

Clerk Shen said: I don’t. The line never guarantees anything.

The officer said: then you are gambling.

Clerk Shen said: I am. But what I am gambling on is when to concede. Draw the line too early, and I lose the boats that come late. Draw it too late, and I lose the third watch drum. That one watch of slow seal is what lets me concede without conceding absolutely.

The officer asked: sealing watch by watch — isn’t that needlessly chopped up?

Clerk Shen said: It isn’t chopped up. Dawn is dawn and noon is noon. The third watch drum wants three numbers, not one total. Stir the whole day into a single pot and nobody can tell where the flood season turned.

The officer was quiet for a while, then added a requirement: not only the day’s three watches. He wanted, at any moment, the total for the three watches counting back from now. He wanted to watch the flood season move.

So Clerk Shen kept a second abacus. Each time a boat came, he slid the oldest watch’s grain out and the newest watch’s grain in. The window slid along the river without ever stopping.

— —

In the year Clerk Shen died, he was teaching an apprentice named He.

The apprentice asked: master, how far does the line have to be drawn before today is finished?

Clerk Shen, from his bed, said: until the last post station raises its flag.

The apprentice said: isn’t that the same as waiting for everyone?

Clerk Shen said: no. Waiting for everyone is waiting for boats. Drawing the line is waiting for news. Boats capsize, sink, get half their cargo stolen. News does not. News says exactly one thing — above me, there are no boats.

The apprentice said: but that could be false.

Clerk Shen smiled a little: which is why I never seal it shut. I keep one watch.

— —

By now you’ve probably recognized it: this is the event-time watermark of stream processing.

What it is

In stream processing, every event carries two times:

• Event time: when it actually happened — the loading hour written on the slip.
• Processing time: when the system saw it — the hour the boat reached the landing.

These are never equal. Networks reorder, sources back up, partition read rates diverge. Windowing by processing time lets the speed of the machine decide the answer, and replaying the same data twice yields two different results. So the correct choice is to window by event time.

But windowing by event time runs straight into Clerk Shen’s problem: nothing can wait indefinitely for an event that may never come.

A watermark is the system’s assertion about completeness: no more events with event time ≤ T will arrive. It is a monotonically advancing scalar, derived as the minimum across upstream partition progress (low watermark), and it propagates downstream with the data. It is not a guarantee. It is a wager, and the part you lose is called late data.

Five mechanisms orbit that line:

• Windows: slice the stream by event time — fixed windows (dawn, noon, night), sliding windows (the three watches counting back from now), session windows (break when the gap between adjacent events exceeds a threshold).
• Triggers: when to emit. It can be “watermark passes the window end” (on-time, final), or “every five hundred piculs send a runner” or “every stick of incense report” (early / speculative firing), so downstream does not have to wait for the ledger to close before starting work.
• Allowed lateness: how long after the window end the system keeps accepting late data, amending results it already emitted. This is the slow seal.
• Late side output: events beyond allowed lateness no longer enter the window; they are routed to a separate stream. The leakage account.
• Accumulating vs. discarding: when an amendment arrives, do you re-emit the complete new total, or only a delta? Clerk Shen chose to re-emit, because the provincial office only accepts whole ledgers.

Why it matters

Aggregating by processing time is the most common default mistake in stream processing. It is always correct in a demo, because demo data is never late. In production, one backed-up source, one consumer restart, one cross-region link blip, and replaying the same history twice produces two different numbers — while billing, alerting, metering, and reporting are all built on that number.

The watermark turns “when is this result complete” from something implicit and machine-speed-determined into something explicit: tunable, observable, arguable. It forces three questions to be answered at once:

• Which clock are you windowing on?
• How long are you willing to wait? The longer you wait, the higher the latency and the less late data you lose.
• Where does the part you could not wait for go? Dropped, side-output, or kept as a correction you can merge?

It is also the bridge that makes unified stream-and-batch possible: batch processing is just the special case where the watermark is pushed to infinity. The same windowing, triggering, and allowed-lateness semantics, run over an unbounded stream, give a bounded approximation; run over bounded data, they give an exact answer. Google’s Dataflow model (Akidau et al., 2015) split this into four orthogonal pieces — windowing, triggering, accumulation, watermarks — and Apache Flink, Beam, and Spark Structured Streaming all build on that decomposition.

The cost sits in the slowest pipe. A watermark takes a global minimum, so one stuck partition pins the whole stream’s watermark, windows never fire, and state grows without bound behind the backlog. This is not an implementation bug; it is intrinsic to the design — to be “complete,” you must let the slowest one speak. Production systems therefore add watermark idleness detection to drop empty partitions out of the minimum, state TTL as a backstop for late data, and monitoring for watermark skew across partitions. The four characters Clerk Shen wrote in the margin are that metric.

Metaphor mapping

• The loading hour on the slip → event time
• The hour the boat reaches the landing → processing time
• Eleven upriver villages whose boats arrive out of loading order → events arriving out of order across partitions
• The three ledgers for dawn, noon, and night → tumbling windows
• The second abacus counting back three watches at any moment → sliding windows
• A post station’s blue flag raised high → a single partition’s progress, its per-partition watermark
• Clerk Shen looking only at the lowest flag → the low watermark, the global minimum
• The red line drawn on the whitewashed wall → watermark T: “no more events with event time ≤ T”
• Waiting for the final flag before sealing → watermark pushed to infinity, which is batch processing
• The small boat crossing the line → late data
• Breaking the seal and adding a line marked amended → allowed lateness and result revision
• Keeping each ledger unsealed one extra watch → retaining a window past its end for allowed lateness
• Sending a runner every five hundred piculs → early / speculative trigger
• The last ledger before the third watch drum → the window’s final trigger
• Re-sending the whole ledger rather than only the difference → accumulating mode
• The leakage account, settled monthly → late side output
• One pot-stirred total for the whole day → the question you cannot answer once window boundaries are gone
• A sick post rider, thick fog, a capsized boat → the watermark is an assertion, not a guarantee; root causes of late data
• The third watch drum closing the warehouse gate → downstream’s hard latency constraint
• Shamuao’s flag down for three days, the red line jammed → watermark skew: one lagging partition pins global progress
• Dropping an empty post station out of “the lowest flag” → partition idleness detection
• Boats still too late for the slow seal → eviction beyond allowed lateness, backstopped by state TTL