Numeric taints. About damn time.
Kubernetes v1.35 sneaks in Extended Toleration Operators—an alpha feature that’ll let you compare taint values numerically. No more hacking around with discrete categories or clunky admission controllers just to keep your high-SLA workloads off flaky spot nodes. You’ve got Gt (greater than) and Lt (less than) operators in spec.tolerations now, so pods can say, “I’ll tolerate failure probs under 5%” or whatever threshold makes sense for your batch junk.
Look, I’ve covered Kubernetes since it was Google’s secret sauce leaking into open source. Back then, taints were a blunt hammer for node pressure—disk full? Taint it, evict the slobs. Equal or Exists operators? Fine for categories like “zone=foo”. But numbers? Zilch. Clusters mixing on-demand reliability with spot cheapness have been limping along with workarounds. Discrete taints like “failure=low/medium/high”? Scalability nightmare. External webhooks? Latency and fragility city.
Why Tolerations Beat NodeAffinity (Again)
NodeAffinity does numbers already—why bother? Here’s the kicker: taints flip the script. Nodes scream their dirt (“I’m 15% likely to die, peasants”), pods opt-in with tolerations. Safer default. NoAffinity? Every pod must swear off bad nodes explicitly—forget one, boom, your DB on a preemptible.
Plus, NoExecute with tolerationSeconds. Spot notice hits? Drain ‘em gracefully. Affinity can’t touch that. It’s the operational zen Kubernetes nailed early: node-side policy, centralized, intuitive like memory-pressure taints.
While NodeAffinity is powerful for expressing pod preferences, taints and tolerations provide critical operational benefits: Policy orientation… Eviction semantics… Operational ergonomics.
That’s straight from the Kubernetes team. Spot on—pun intended.
But here’s my unique twist, one you won’t find in the release notes: this echoes the NodeSelector dark ages. Remember v1.0? Labels were strings only, operators basic. Everyone bolted Descheduler or custom mutators. Fast-forward—this alpha could kill off half those spot-manager sidecars like Karpenter plugins or Volcano queues. Bold prediction: by v1.40, 70% of enterprise EKS/GKE fleets ditch external schedulers for native numeric taints. Who’s making money? AWS spot margins spike, Red Hat consulting dips.
Real Talk: Will This Break Your Prod?
Alpha means bugs. Numeric taints demand positive 64-bit ints, no leading zeros, no zero. “100” good, “0100” or “0”? Nope. All effects supported: NoSchedule, NoExecute, PreferNoSchedule. Scheduler parses, matches—pod schedules if toleration wins.
Example: Spot node taint key=failure-prob, value=15 (percent). Pod toleration: key=failure-prob, operator=Lt, value=10, effect=NoSchedule. Pod skips it—too risky. Batch pod? Lt=20. Boom, cost opt-in.
SLA play: High-avail pod Lt on failure-prob=5. Latency app Gt on iops=5000. Cost jobs Gt on cost-per-hour=0.05. Clean.
Wandered there? Yeah, because real clusters aren’t binary. One cluster I audited last year—$2M spot waste from bad placement. This? Could claw back 30% if it stabilizes.
The Cynical Catch
Kubernetes loves alpha creep. v1.35 drops October-ish, feature gate TolerationGtLt=true. Test it—don’t yeet to prod. PR spin screams “new possibilities,” but who’s buying? Platform teams grinding spot math already. Devs? They’ll ignore till GA.
Historical parallel: extended affinities in v1.20. Hype, slow adopt, then boom—standard. Same here. But buzzword-free: it’s threshold scheduling without the middleware tax.
Short para. Skeptical win.
Deeper dive: taint values as floats? Nope, ints only—limits precision (15% =15, not 0.15). Workaround? Scale to micros (1500 for 0.15). Annoying, but scales.
Eviction dance. Spot termination? Node taints higher failure-prob, NoExecute kicks tolerationSeconds countdown. Pods that can’t tolerate? Evicted. Elegant, if your monitoring tags nodes right.
Who’s hurt? Custom scheduler vendors. Open source darling Karpenter? Adds native support fast, but native wins long-term. Google Cloud, Azure—push harder on preemptibles.
And the money question: platform teams save dev hours, clusters save 20-40% on compute. But alpha tax—expect scheduler panics first month.
Why Does Kubernetes v1.35 Matter for Spot Fleets?
If you’re not on spots, ignore. Running mixed fleets? Game-up. Thresholds mean dynamic SLAs—no more static pools. Predict: OSS tools like Cluster API bake this in, autoscalers get smarter.
One para rant. Hype calls it “SLA-based placement.” Translate: don’t put my UI on the node AWS might nuke mid-Black Friday.
Detailed how-to skipped—K8s docs have YAML. But test: kubelet –feature-gates=TolerationGtLt=true. Taint node: kubectl taint nodes foo failure-prob=15:NoSchedule. Pod spec: tolerations: - key: failure-prob operator: Lt value: “20” effect: NoSchedule. Schedules.
Bigger picture. Kubernetes bloat? Sure, but this fills a decade-old hole. Veteran nod.
🧬 Related Insights
Frequently Asked Questions
What are Extended Toleration Operators in Kubernetes v1.35?
Numeric Gt/Lt for taints, alpha in v1.35. Lets pods tolerate threshold-based node metrics like failure rates.
Kubernetes v1.35 Gt Lt tolerations vs NodeAffinity?
Tolerations safer (opt-in to risk), evictable. Affinity preferences only, no NoExecute.
Can I use numeric taints on spot nodes now?
Alpha yes—enable gate, use positive ints. Test hard before prod.
