The rules at Qinghe wharf had not changed in forty years.

At first light the grain boat ties up. Boatmen haul the sacks out of the hold and stack them on the stone quay. Porters carry them, one sack at a time, into the inner hall of the warehouse — the clerk’s domain. There the clerk unties every sack, weighs it, ties it up again, and enters it in the ledger. When he is finished the porters carry the sacks back out of the inner hall, the whole length of the quay, and load them onto the barge waiting downstream.

One sack of grain. Set down three times, carried four.

There is a worse rule at the door of the inner hall: every time a porter passes through it he must stop at the gatehouse, give his name, set down his pole, and have the hour written into a book. For thirty sacks, the stopping and starting at that one door eats half the morning.

The new harbor commissioner that year was named Shen. On his first day he stood on the quay for four hours and said nothing. The next morning he sent for the clerk.

“When you untie a sack, what is it for?”

“To know how many pecks are inside, and of what grade.”

“Does the boatman’s seal say how many pecks, and of what grade?”

“It does.”

“Then do you untie it to change the grain, or to know the number?”

The clerk was quiet for a long moment. “…To know the number.”

That same day Shen had the carpenters build a long wooden chute, one end resting at the boat’s hatch, the other reaching into the barge’s side door, nothing in between touching the ground. And he took away the clerk’s scales and ledger and gave him something small instead: a wooden tally board with only three lines on it — which hold the pile is in, which sack it starts at, how many sacks it holds.

After that, mornings at the wharf went like this. The boatmen swing the hatch open and the sacks slide down the chute into the barge by themselves. The clerk stands to one side turning his tally boards over, checking them off, and from beginning to end he never touches a single grain. The porters sit on empty crates in the sun.

Same grain. Same numbers. Only nobody carries it back and forth any more.

Shen kept two exceptions.

First: if a boat’s grain must be graded by hand, or dusted with lime against the damp, then that boat goes into the inner hall the old way — because whoever changes a thing must put his hands on it. Later he built a small shed right beside the chute so the inspector could do the work there, saving at least the walk to the inner hall and back.

Second: fast as the chute is, it must not run one sack at a time. He had the tally boards grouped by pile, one report per pile, and the stop-and-start at the gatehouse fell from thirty times to three.

— By now you have probably recognized it: this is not a story about a wharf. The sacks are the data, the clerk’s inner hall is the user-space buffer, the porter’s shoulder pole is a CPU memcpy, the chute is DMA, and the tally board is a descriptor. This is a story about zero-copy I/O.

What this is

The naive way to send a file from disk to the network is read() then write(). The data moves four times: DMA from disk into the page cache, a CPU copy from page cache into the user buffer, a CPU copy from the user buffer into the socket buffer, then DMA out to the NIC. Two of those are memcpys performed by the CPU itself — pure hauling, with not one byte altered — plus four user/kernel transitions for the two syscalls.

Zero-copy rests on one observation: if the application never intends to change the data, the data should never pass through the application’s address space. sendfile() hands the kernel the whole sentence — “from this offset in this file, send this many bytes to this socket”. With scatter-gather DMA on the NIC, what lands in the socket buffer is only a descriptor (address + length); the NIC pulls the bytes straight out of the page cache and the CPU never touches the payload at all. splice() does the same between two file descriptors through a pipe, moving nothing but page references.

Why it matters

At 25/100 GbE the bottleneck stopped being the NIC long ago; it is memory bandwidth and CPU cycles. What you save is not only the time spent in memcpy but the L3 cache that gets flushed by data the CPU had no reason to look at. Kafka’s throughput, nginx’s sendfile on, HAProxy splicing TCP, container image layer distribution — all the same trick underneath.

Its limit is just as clear: the moment the application must change the data, zero-copy is off. TLS was long the great spoiler — to encrypt, you must touch. The answer was not to give up zero-copy but to move the encryption onto the data path: kTLS encrypts inside the kernel’s send path, and NIC crypto offload goes further still. That is Shen’s shed beside the chute.

One more thing worth keeping: zero-copy saves copies, not syscalls. Slide the sacks down one at a time and the gatehouse will still ruin your morning — which is why io_uring’s batched submission and registered buffers are a companion to zero-copy, not the same thing.

Metaphor mapping

• grain in the sacks → the payload itself (file contents / network bytes)
• the boat’s hold → the page cache: the copy the kernel already has
• the clerk’s inner hall → the user-space buffer
• each trip a porter makes → a CPU memcpy, paid for in cycles and memory bandwidth
• signing in at the gatehouse → syscalls and user/kernel context switches
• the wooden chute → DMA / scatter-gather DMA: data reaching the destination without passing through the app
• the clerk’s tally board → the descriptor: address + offset + length, which is all sendfile()/splice() moves
• reporting one pile instead of one sack → batched submission (io_uring, large sendfile chunks) to cut syscall count
• the boat that must be dusted with lime → cases where the app must transform data — compression, encryption, parsing — and zero-copy no longer applies
• the shed beside the chute → kTLS and NIC crypto offload: move the transformation onto the data path instead of moving the data to the transformation