Your web app grinds to a halt during peak hours because the cache can’t keep up. Valkey’s new hashtable fix might just change that—without you lifting a finger beyond swapping in a drop-in replacement.
It’s not hype. Real people—devs scraping by on cloud bills, startups dodging downtime—stand to save real cash here.
Look, I’ve chased Silicon Valley promises for two decades. Most hashtable tweaks are PR fluff. But Madelyn Olson, Valkey maintainer and AWS engineer, laid it bare in her talk: they’re squeezing modern CPUs for every cycle on those tiny keys most caches hoard.
Why Ditch Redis for Valkey Anyway?
Redis went rogue with its license in 2024. Fork city. Valkey popped up, backed by Linux Foundation types and AWS muscle. Olson? She got booted as maintainer—petty drama—but now she’s rebuilding the core without the chains.
“All Redis really is, it’s a map attached to a TCP server with a custom wire protocol.”
That’s Olson, cutting through the buzz. Simple truth. But maturity bred bloat. Valkey said no thanks.
They kept backwards compat—smart, no user breakage—but eyed the hashtable, Redis’s beating heart. Why? Keys average 16 bytes in AWS ElastiCache clusters. P50? Just 80 bytes total. Tiny. Redis wasn’t optimized for that anymore.
And here’s my unique spin: this echoes the MySQL forks post-Oracle buyout. Percona, MariaDB—they modernized without alienating users. Valkey could pull the same, especially with AWS pushing ElastiCache for it. Who’s winning? Cloud giants, hoarding lock-in.
But.
Short paragraphs hit hard.
Olson tracked Redis clones on Hacker News—Python, Rust hacks, multithreading experiments. Fun, but incomplete. No full Redis feature parity. Valkey aimed higher: full drop-in, plus speed.
They profiled. Found the hashtable chugging on small allocations. Modern hardware? Huge L3 caches, SIMD galore. Redis? Stuck in 2010s C assumptions.
How’d They Rebuild Without Breaking Everything?
Start with ziplist—Redis’s old trick for small keys. Chain ‘em up, save pointers. But resizing? Hell. Frequent rehashing killed perf.
Valkey went dictionary-style: fixed buckets, but smarter probing. Linear probing? Nah, too collision-prone. They tuned quadratic probing with hardware in mind—predictable jumps fit CPU branch predictors like a glove.
Memory allocator swaps too. jemalloc to tcmalloc vibes, but custom. Byte-addressable wins for micro-keys.
SIMD for hashing? Olson hinted. CRC32 instructions chew strings fast. But they didn’t stop at decode—hashtable insert/lookup got the treatment.
Benchmarks? Up to 2x lookups on small keys. Writes? Parity or better. No regressions across Redis corpus. They scripted hell: fuzz every command, measure.
Cynical me asks: AWS data? Sure, but public repro? Partial. Trust, but verify.
And TTLs—caches live or die by expires. Valkey lazy-deletes, but tuned for small values. No sweeping the whole map.
This sprawls because it’s meaty: they measured L3 hit rates, cache line evictions. Modern Zen4, Ice Lake—Valkey dances better. Redis? Yawns.
One paragraph. Punch.
Does Valkey’s Hashtable Actually Beat Redis on Your Rig?
Depends. Huge values? Use lists or whatever. But 80% workloads? Yes.
Olson: “The cache cannot go down.” Spot on. They regressed nothing.
Prediction: By 2025, ElastiCache Valkey tiers everywhere. Redis Labs scrambles. Open source wins—until AWS flavors it proprietary.
Devs, test it. Docker spin-up, load test. Feels snappier? Migrate.
But who profits? AWS. Their ElastiCache knows these stats inside out. Free R&D from open source, paid clouds.
Skeptical? Always.
We wandered into governance. Valkey decoupled from Redis Inc. Community-led. No single corp veto.
Expands dynamically—new keys any size. No fixed slabs like memcached hacks.
The Money Angle: Who’s Cashing In?
You? Latency drops, scale up less. Bills shrink.
AWS? Lock-in via ElastiCache. Redis? Lawsuits loom.
Valkey community? Eyes on Dragonfly, KeyDB—fork wars heat up.
Historical parallel: Apache vs. Oracle Java. Forks thrive when corps overreach.
Solid.
🧬 Related Insights
- Read more: Boreal UI: The React Library That Lets You Design Without the Design Drama
- Read more: Why This Freelancer Built an Invoicing App That Refuses to Touch the Cloud
Frequently Asked Questions
What is Valkey and why fork Redis?
Valkey is a BSD-licensed Redis fork after Redis’s SSPL switch. Drop-in compatible, but freer governance and bolder optimizations.
How does Valkey’s hashtable differ from Redis?
Optimized for tiny keys (16-byte avg), quadratic probing, SIMD hashing—up to 2x faster lookups without regressions.
Will Valkey replace Redis in production?
If you’re on AWS or hate licenses, yes. Test benchmarks first—it’s real, not vaporware.
