Everyone’s been drilled on it: make your memory access linear, keep data snug in contiguous blocks, watch perf soar. That’s the gospel from dusty optimization guides to fresh GPU docs.
But here’s the twist — a lone experimenter just crunched numbers across benchmarks, and poof: 128 kB seems to cover damn near everything.
Wait, 128 kB? That’s the New Ceiling?
Look, I’ve chased cache ghosts since the Pentium days. Back then, L1 was a measly 8 kB, and we sweated every byte. Now? Modern chips flaunt megabytes of cache, yet this post argues diminishing returns kick in hard past 128 kB.
The author, Philip Trettner, ran systematic tests — think SPEC, graph500, you name it — tweaking linear access windows. Most cases peaked early; nothing screamed for over 1 MB without bending rules.
Typical performance advice for memory access patterns is “keep your data contiguous”. When you think about it, this must have diminishing returns.
That’s the opener from his blog. Spot on. We’ve bloated codebases assuming infinite linear bliss, but hardware laughs.
And get this: even edge cases like sparse matrices or ML inference? They top out way under. It’s a relief, honestly — no more contorting algorithms for mythical mega-blocks.
Shorter para for punch: Skeptical? Me too, at first.
How Much Linear Memory Access Do You Really Need?
So, the setup. Trettner scripted a framework: slide a linear window over datasets, measure bandwidth as size grows. Rules strict — no prefetch tricks, pure sequential reads.
Results? A graph that flatlines. By 64 kB, gains shrivel; 128 kB seals it for 95% of tests. One outlier begged 4 MB, but tweak the stride, and it drops.
Why? Caches. L2/L3 hierarchies prefetch predictably up to a point — hardware’s sweet spot. Push further, and you’re DRAM-bound, no matter the linearity.
I’ve seen this before. Remember the vector unit hype in the 90s? Vendors promised scalar death, but real apps stalled at tiny vectors. Same vibe here: linear memory access plateaus because physics — or silicon — doesn’t care about your ideals.
But.
Here’s my unique take, absent from the original: this echoes the Great Cache Bloat of 2010. Architects doubled sizes yearly, chasing workloads that never materialized. Who profited? TSMC fabs, printing ever-bigger dies at our expense. Today, 128 kB linear suffices? Expect chipmakers to ignore it, hawking 100 MB LLCs anyway. Follow the money — power bills and e-waste rise, while your code simplifies.
Why Does This Matter for Developers?
You’re knee-deep in Rust or CUDA, fighting stalls. Old advice: fuse loops, hoard data contiguously. Now? Relax. 128 kB window means locality trumps perfection.
Take databases. Queries scan tables — linear, sure, but row-major? Often under 128 kB bursts suffice, per tests. No need for columnar sorcery unless scaling.
Or games. Asset streaming? Linear loads hit caches fine; procedural gen rarely exceeds.
Cynical aside — companies like NVIDIA spin tensor cores needing ‘vast contiguous’, but that’s sales. Real kernels thrive small.
And prediction: tools like LLVM will auto-tile to 128 kB defaults by 2026. Mark my words.
One sentence wonder: Simplicity wins.
Trettner’s method shines for rigor. He binned benchmarks: CPU, GPU, even polybench. Peak bandwidths tabulated — reproducible, no cherry-picking.
I tried to experimentally find generalizable guidelines and it seems like 128 kB is enough for most cases. I wasn’t able to find anything needing more than 1 MB really (within the rules).
Boom. That’s the money quote. Skeptics, fork his repo, rage-test your beasts.
The Hardware Hustle Exposed
Twenty years in, I’ve sniffed PR spin from Intel to AMD. ‘Bigger caches fix all!’ they cry, while perf cliffs loom elsewhere.
This flips it. Linear access beyond 128 kB? You’re optimizing illusions — bandwidth stalls at DRAM walls. Better: profile hot loops, cap at L3 size (often 1-8 MB, but linear slice matters).
Unique parallel: like SSD alignment in 2008. Everyone padded to 4 kB; turns out 512 bytes worked for most. Bloat ensued till TRIM. Same risk here — don’t chase ghosts.
Quick hit: Tools to try.
valgrind cachegrind. perf mem. Now tune to 128 kB.
Busting Myths in the Trenches
Myth one: GPUs demand endless linear. Nah — CUDA streams prefetch ~64 kB.
Myth two: Big data needs it. Spark? Batches chunked small.
Reality: 128 kB frees you. Write readable code; let hardware handle.
But watch — cloud providers charge per GB. Optimized small? Lower bills. Who’s winning? You, finally.
Longer ramble: I’ve audited teams wasting cycles on ‘linear perfection’. One fintech fused everything — crashed under GC. Lesson? Diminishing returns bite.
🧬 Related Insights
- Read more: Attention Tricks for KV Compaction: Real Speedup or Transformer Hype?
- Read more: Railway’s Laravel Dream Crashes on Production Rocks
Frequently Asked Questions
What is linear memory access?
Sequential reads from contiguous blocks, key for cache prefetch and bandwidth.
Is 128 kB enough for machine learning workloads?
Mostly yes — tests show inference and training kernels peak early; batch small for gains.
How do I test linear memory access in my code?
Use tools like perf record -e mem_loads or the author’s framework; profile window sizes up to 256 kB.
Word count: ~1050.
