If you’re a solo dev or small team building that SaaS side hustle on Ruby on Rails performance tricks, this story hits home. FirstPromoter, tracking referrals for 7,000+ programs, just doubled their weekly throughput to 35.5 million requests — all while shaving latencies. Real people? That’s you, not rewriting in Rust when Stripe webhooks flood in at 1.47 million a day, threatening your revenue.
Damn.
Look, I’ve covered Valley flameouts for two decades. Apps that start zippy on Rails, then crawl under load, forcing ugly rewrites. But FirstPromoter didn’t bail. They tuned. Hard. And here’s the cynical truth: most ‘scaling stories’ are PR fluff. This one’s raw production logs. No vaporware.
That’s more than double the throughput from last year (15.7M → 35.5M) — while simultaneously making things faster.
They’re processing payment hooks from every processor under the sun — Stripe, Chargebee, you name it. Each one’s gotta be lightning, or customers’ commissions vanish. Forget that, and you’re toast.
Why Chasing Counts Killed Their Dashboard
Dashboards everywhere screaming COUNT(*) queries. Active promoters. Pending referrals. Commissions stacking up. PostgreSQL choking on real-time hits.
They started with Rails’ counter_cache — fine for basics. Then leveled up to counter_culture for conditionals, multi-caches. Smart. But the killer move? Scoping refreshes to active companies only. No more cron jobs thrashing dead data.
Before: blanket refresh. After: targeted, with async workers hitting replicas. Dashboard endpoints? From 10+ seconds to 2.5. Listings under 500ms. That’s not theory. That’s cash flowing smoother.
And get this — they confess to over-reconciling early on. Rookie trap. Now it’s surgical.
Here’s the thing. I’ve seen this before, back in 2008 when Twitter’s Rails monolith buckled under firehose tweets. They bolted counters everywhere too. But FirstPromoter’s twist? They treat counters like ammo — rationed, not sprayed.
One paragraph. Boom.
N+1s Aren’t the Only Preload Trap
Everyone preaches fixing N+1s. Duh. Bullet and Prosopite in dev? Good hygiene. But preload everything? Memory balloon city.
Discipline: profile first. Preload only what serializers touch. Nothing extra. Overdo it, and you’re trading query time for RAM hell. We’ve all profiled a bloated includes() chain — pages of associations nobody needs.
Skeptical eye: this screams junior dev syndrome. Throw includes at walls, hope it sticks. FirstPromoter learned late, but learned.
Redis. Four jobs, all critical.
First, webhook queues — validate payload, enqueue Sidekiq, 200 back. No inline processing. Handles 10M+ weekly, responses snappy.
Atomic counters for emails, events — sidestep DB contention. Flush to Postgres later.
Revenue caches — complex joins? Redis it, refresh scheduled. Dashboards instant, data fresh-ish.
Rails cache store, pooled connections. No exhaustion under storm.
Can Sidekiq Survive 100K Queued Jobs?
100K+ jobs queued always. Tuning? Mandatory.
Separate processes: tracking (latency king), mailers, general. No CSV export starving webhooks.
Queue limits via sidekiq-limit_fetch. Priorities locked.
Consolidate workers — archiving promoters, referrals? One job, SQL FILTER agg. Fewer round-trips, simpler brain.
:queues: - [tracking, 10] - [critical, 8] - [mailers, 5] - [default, 3] - [low, 1]
That’s orchestration. Not magic.
But who’s paying? FirstPromoter’s customers — affiliates banking on reliable tracking. Scale fails, they bolt to competitors. Money flows to the steady.
Analytics crushed Postgres. Trending charts, revenue splits, time-series — scanning millions, timing out at 30s.
BigQuery migration. Queries now <2s. But they reverted some — complexity tax too high. Rule: hundreds of thousands rows or time-series? BigQuery. Else, Postgres.
Post-migration: 95% APIs under 3.4s. Heavy ones? Slashed.
Why Not Microservices Hype?
Here’s my unique take, absent from their post: this reeks of Basecamp’s playbook. 20 years on Rails monolith, no microservices Kool-Aid. Remember when everyone preached ‘Rails doesn’t scale’? Basecamp laughed, iterated. FirstPromoter echoes that — tune the stack you know, don’t chase buzz. Prediction: in five years, half these BigQuery migrants will envy the Postgres purists who optimized queries first.
Corporate spin? None here. This is engineer confessional. No ‘revolutionary architecture.’ Just grind.
So, for real people — indie hackers hitting growth walls. Don’t panic-migrate. Scope counters. Queue smart. Profile ruthlessly. Rails scales if you do.
But watch the bills. Redis clusters, BigQuery scans — costs creep. Who’s making money? You, if latencies drop and churn does.
A long one: We’ve profiled apps where devs ignored replicas, hammered primaries till smoke. Or Redis sans pools, connections dead. FirstPromoter dodged those bullets, but how many startups don’t? Their logs scream ‘test in prod-scale’ — staging’s a liar.
Short. Redis rules.
When Postgres Gives Up the Ghost
Not every query fits OLTP. BigQuery for OLAP. Simple as that. But they admit overreach — reverted writes-heavy stuff. Lesson: hybrid stacks win, but pick battles.
Cynical? BigQuery’s Google lock-in. Vendor risk. Postgres loyalists smirk.
Wrapping the cynicism: Rails in 2024? Still king for MVPs to millions. But demands respect. Ignore these lessons, join the rewrite graveyard.
🧬 Related Insights
- Read more: 82% of Breaches Linger Undetected: ML Anomaly Detection’s Quiet Promise—and Pitfalls
- Read more: Can a $199 AI Auditor From Idaho Really Secure Your Smart Contracts?
Frequently Asked Questions
What does counter_culture do in Rails?
Handles conditional counter caches beyond Rails defaults — perfect for active-only counts without full rescans.
How to scale Sidekiq for Rails webhooks?
Separate queues with limits, dedicated processes per priority, consolidate similar jobs to cut DB hits.
Does Ruby on Rails scale to millions of users?
Yes, with Redis caching, replica reads, scoped counters — but profile everything first, or it’ll bite.
