AI engineering is barreling forward at three to four times the speed of DevOps or data engineering waves.
That’s straight from Sam Bhagwat, co-founder and CEO of Mastra, an open-source JS/TS framework for AI agents. I’ve chased Silicon Valley hype for 20 years—React frameworks, Kubernetes gold rushes, you name it—and this guy’s seen the playbook before. Gatsby? He co-founded that React darling back when components were controversial. Now he’s betting on AI agents. But here’s the thing: speed doesn’t mean success. Who’s actually cashing checks here?
Look, Bhagwat’s podcast with InfoQ’s Shane Hastie cuts through the buzz. No “agentic revolution” fluff. Just gritty talk on evolving open source crowds, custom evals, and why your org’s weird data beats generic benchmarks.
“The highest-value evals for agentic applications are those written against an organisation’s own unique data and domain expertise, not generic off-the-shelf benchmarks.”
Sam nails it. Off-the-shelf? Please. That’s like judging a race car on a dirt track.
Why Open Source AI Feels Like Gatsby 2.0—But Faster
Bhagwat’s story starts 10 years back. React’s rising, controversial as hell. He and his buddy build a framework, open it up, watch it explode. Tinkerers flock in—Poles, Indians, directors, students. Permissionless magic. Layers stack, integrations bloom. Fun, right?
But communities morph. Early days? Enthusiastic hackers. Traction hits, production users complain. “Inherited this crap, hate your choices.” Maintainers? Light touch. Ditch opinionated walls. Listen—or die.
Cynic alert: This ain’t charity. Successful OSS firms need hybrids—generous souls with profit noses. Purists starve; sharks alienate. Bhagwat’s Mastra? Balances that tightrope. JS/TS for agents. Production-ready, maybe. But who’s monetizing? Dual-license dreams, I bet.
Short para punch: It’s evolution, Darwin-style.
And AI engineering? Same arc, turbocharged. DevOps took years to mature. AI? Months. Why? Urgency. Everyone’s scrambling for agent edge. But speed kills without structure.
Tiger Teams: Marrying Code Nazis and Data Hippies
Shipping agents ain’t solo dev heroics. Bhagwat’s core: Cross-functional tiger teams. Software eng’s rigor—bugs die young. Data science’s chill—stats ain’t certainties. Blend ‘em, or flop.
Picture it. Agent hallucinates on your CRM data? Generic eval says fine. Custom one, tuned to your sales quirks? Flags the mess. Org-specific gold.
I’ve seen this movie. DevOps rose on SRE squads. Data eng on platform teams. AI? Same, but frantic. Prediction—my unique spin: By 2026, 70% of Fortune 500 AI fails trace to siloed evals. Tiger teams win. But PR spin incoming: Every vendor’ll hawk “agent platforms.” Buyer beware—who funds their benchmarks?
Bhagwat’s Gatsby parallel? Spot on. Tinker to production. But AI’s stakes higher—hallucinations cost real dough.
One sentence wonder: Teams, not tools, ship.
Now, Mastra. Open JS agent framework. Agents as code? Typescript safety? Sounds sane. No Python monopoly—web devs rejoice. But cynical me asks: Will it Gatsby or ghost town? Contributors worldwide, sure. Production pull? Jury’s out.
Is Custom Evals the AI Money Maker?
Generic benchmarks? HuggingFace leaderboards? Fun for papers. Useless for payroll.
Your data’s moat. Pharma agents eval on trial data. Banks on fraud patterns. Write evals there—value soars.
Bhagwat pushes org-unique tests. Smart. But execution? Tiger teams again. Eng writes tests. Data tunes ‘em. Iterate.
Historical nod—my insight: Like Webpack’s plugin ecosystem. Emergent, user-driven. AI agents? Same permissionless vibe. But uncertainty amps drama.
Hype check: “Agentic apps” everywhere. Reality? Most pilots die. Why? No tigers prowling.
Deep dive para: Open source health? Evolve receptive. Feedback loops. Flexibility over dogma. Bhagwat’s seen breakdowns—nasty GitHub wars. Solution? Assume excitement first. Complaints second. Can’t please all. But try, adapt. Commercial twist: Pragmatism pays bills. Generosity builds moat.
Why Does This Matter for Your Stack?
Dev? Dust off tiger skills. Data folk? Code up. Or get left.
Mastra’s OSS bet: Global collab. Like Gatsby’s Polish star. Unpredictable wins.
Skeptical close: AI engineering’s hot. 3x speed thrills VCs. But money follows rigor. Tiger teams, custom evals—playbook’s here. Ignore at peril.
🧬 Related Insights
- Read more: PaperMod Shortcodes Unlock Hugo’s Visual Power
- Read more: Zero AWS Experience to CLF-C02 Certified: The Raw Prep Blueprint
Frequently Asked Questions
What are tiger teams in AI engineering?
Cross-functional squads blending software rigor and data uncertainty to ship agentic apps.
How do custom evals beat generic benchmarks for AI agents?
They test against your unique data and domain, catching real-world fails generics miss.
Is Mastra the best open source framework for AI agents?
Promising JS/TS option from Gatsby vets, but watch community traction.
