In a small town there was an old-name print shop with seven or eight apprentices, but only one printing press — presses were expensive back then, and the owner wasn’t buying a second. To keep any one apprentice from hogging it, the owner set a rule: every incense-stick’s burn (about a hundred breaths) was divided up, and each apprentice got a fixed slice of it — say, Old San could use the press for twenty breaths out of each stick. Once he’d used his twenty, the pedal locked, and it wouldn’t unlock until the next stick was lit.

At first it seemed fair enough. Old San worked slowly, and often a whole stick burned down without him using all twenty breaths — the pedal sat idle half the time. The owner checked the ledger and felt reassured: tallied up each evening, Old San used the press less than half his allotted time for the whole day, well under his share, plenty to spare.

Then a busy stretch hit the shop, and Old San found a new trick: he’d stack up a pile of work, and the moment a fresh stick was lit, he’d pound away at full speed, barely pausing — often burning through his twenty breaths before the stick was even a fifth gone. The pedal locked with a click. No matter how hard he stomped, it wouldn’t budge. He could only stand there while the remaining four-fifths of the stick burned away uselessly, waiting for the next one to start before he could work again.

The other apprentices were baffled: the ledger plainly showed “Old San used less than half the press-time today” — so why did his work keep stalling, and why did it get worse the busier things got? Eventually the owner started a second ledger, one that tracked only “how many times today was Old San locked out, and for how long.” Flipping through it, he found Old San was locked out for seventy or eighty percent of nearly every single stick — his actual pedaling time was scattered into tiny, broken fragments. It turned out “how much total time was used today” and “how much time was spent locked out waiting” were two completely different books. Reading only the first, you’d never discover the second kind of suffering.

The owner eventually understood: Old San wasn’t lazy, and he didn’t have too little work — he was cramming all his effort into the very start of each stick, and running straight into the “twenty breaths per stick” ceiling. Either loosen the allowance so he could borrow across sticks, or convince him to stop front-loading everything and instead print at a steady trickle, so he’d stop slamming into that locked pedal over and over.

— By now you’ve probably recognized it: that printing press is really Linux’s cgroup CPU quota mechanism, and Old San’s repeatedly locked pedal is the common but often-overlooked phenomenon of CPU throttling in containers.

What it is and why it matters When you set a CPU limit on a container in Kubernetes/OpenShift, it’s enforced under the hood by Linux’s CFS (Completely Fair Scheduler) bandwidth controller: the kernel gives each cgroup a fixed period (default 100ms) and a quota it may use within that period (e.g. a 500m limit becomes a 50ms quota). If a container’s threads burst concurrently right at the start of a period and burn through the full 50ms almost immediately (easy to do across multiple cores in just a few milliseconds), the kernel suspends it until the next period begins — that’s throttling. The strange part: a metric like container_cpu_usage_seconds_total (average usage) looks perfectly healthy, well below the limit, because it smooths “time used” over a much longer window; the problem only shows up in a dedicated metric, container_cpu_cfs_throttled_periods_total / throttled_seconds_total. This is why Go/Java services under bursty concurrency often show baffling tail-latency spikes while their CPU usage graphs “look idle” — the real cause is concurrent thread count outrunning what the quota can sustain within a short period.

Metaphor mapping

• The print shop, the single press → the node’s CPU cores / a cgroup’s available compute
• One incense-stick (fixed length) → cfs_period_us (the default 100ms scheduling period)
• Old San’s twenty breaths per stick → cfs_quota_us (the quota derived from the container’s CPU limit)
• Old San stacking work and pounding it out in one burst → concurrent threads/goroutines bursting and exhausting the quota very early in the period
• The pedal locking with a click, the rest of the stick wasted → the process getting throttled, forcibly suspended until the next period
• The old ledger of “total time used today” → the average CPU usage metric, which hides the real problem
• The new ledger of “how many times locked out” → dedicated throttling metrics like container_cpu_cfs_throttled_periods/seconds_total
• The owner’s advice to “spread the work instead of front-loading it” → the fix: raise the limit, loosen the period, or cap concurrency so load smooths out