Pascal Cescato’s comment landed mid-afternoon, coffee cooling beside my keyboard: ‘You don’t need an LLM for this. Everything you’re sending to Claude can be done directly in Python — zero cost, fully deterministic, no hallucination risk.’
He nailed it. But missed the nuance. And that sparked a thread rewriting my entire SEO audit agent.
Look, the original tool crawled URLs, fired them at Claude for checks like title length, meta descriptions, H1 counts, canonical tags. Simple stuff. At $0.006 a pop, a 50-URL run hit 30 cents. Pascal called bullshit—rightly. Those are regex jobs. Count characters. Spot absences. No Shakespeare needed.
Why Bother with LLMs at All?
I pushed back hard. Flags like ‘title reads like a navigation label’ demand judgment. Not binary. Pascal conceded, then dropped gold: two-pass. Deterministic Python first for mechanical fails. Escalate only the weird ones to models.
Julian Oczkowski piled on: “Deterministic rules first, lightweight models for triage, larger models reserved for genuinely ambiguous edge cases. Keeps latency low, costs predictable, reduces unnecessary LLM dependency.”
“Deterministic rules first, lightweight models for triage, larger models reserved for genuinely ambiguous edge cases.”
That’s the blueprint. They named it in a thread—two-pass, tiered, cost curve. I stole it all.
Tier 1: Pure Python. $0.
Title over 60 chars? Fail. No description? Fail. Zero H1s? Fail. Canonical missing? Fail. These scream on agency sites. No model touches ‘em.
Tier 2: Haiku-scale model. ~$0.0001.
Weird passes—like a 4-char title or redirect status. Cheap enough to triage without regret.
Tier 3: Sonnet heft. ~$0.006.
Semantic calls only. “This title’s a nav label.” “Description duplicates title.” Here, the model shines.
Routing decides it all. Most pages flop at Tier 1. The rest? Edge cases worth the dime.
Tested on 50 URLs: 42 died in Python. None saw a model. 8 hit Sonnet. Total cost: $0.05. Down 83% from $0.30. Those 8? Gold—subtle issues regexes missed.
Is This the New Architecture for Every Agent?
Here’s my take, absent from the thread: it’s compiler optimization, 1970s style. Early Fortran scanned everything linearly. Then came peephole optimizers—cheap passes spotting 80% of low-hanging fruit. Heavy analysis only for survivors. Same here. AI agents waste billions on dumb tasks. This cost curve enforces discipline, like indexes taming database full-scans.
Predict this: every production agent goes tiered by 2025. Not hype—economics. LLMs hit $100B runrates; tiering slashes that.
Repo’s reborn around it.
core/ stays MIT-free, untouched. Seven modules. python core/index.py runs v1 forever.
premium/ layers on—licensed, closed. cost_curve.py routes: audit_url(snapshot, tiered=True). Python first. Haiku if fuzzy. Sonnet if flagged.
multi_client.py: –project acme silos state in projects/acme/.
enhanced_reporter.py: WeasyPrint PDFs, screenshots, severity-sorted issues, fixes.
rewrite_agent.py: Post-audit magic. –rewrite tiers suggestions—truncate titles free, Haiku metas, Sonnet intros. –voice-sample your.txt? Outputs sound like you.
main.py unifies. Free: python main.py. Pro: –pro + license. Full blast: python main.py –project client-x –pro –tiered –rewrite.
CSV in, PDF out. History tracked. Screenshots. Your voice.
core/ ignores premium/. Fork-proof trust. Public good auditable. Premium builds atop.
Mads Hansen nailed the blind-spot risk: recurring patterns fool tools. Solved: state.json logs runs—timestamps, pass/fail counts, paths. Spot trends. “This site’s descriptions fail every time? Fix upstream.”
But here’s the PR spin callout—the original post glossed costs as ‘negligible.’ Nah. At scale, $0.006 compounds. Agencies audit thousands. Tiering’s no tweak; it’s survival.
How Does the Real-World Math Stack Up?
Scale it. 1,000 URLs/month agency grind.
Old way: $6.
Tiered: Say 20% hit Sonnet (optimistic). $1.20 tops. Python handles 60%, Haiku 20%. Pennies.
Latency? Python’s instant. Haiku sub-second. Sonnet waits, but rare.
Hallucinations? Zilch in Tier 1. Triage prunes junk.
Forks thrive—core’s pure. Pros pay for polish.
Wander a bit: imagine this in security scans, API audits. Regex gates everywhere. Models freed for genius.
It’s not just SEO. It’s agent OS.
What About Edge Cases That Break It?
Trickiest: dynamic SPAs. JS-rendered metas. Snapshot with Playwright first—core does it. But redirects chain deep? Tier 1 flags status 3xx chains.
Semantic drift—“navigation label” evolves. Haiku learns? Fine-tune on your history. state.json feeds it.
Competition? Screaming Frog’s $200/year. This? Free core, pro maybe $20/month. Deterministic wins trust.
🧬 Related Insights
- Read more: RiskReady: 254 AI Tools for GRC, Locked Behind Human Approval
- Read more: Git’s Shiny Tutorials Lied: The Real Hell Awaits Beginners
Frequently Asked Questions
What is the SEO audit cost curve?
It’s tiered routing: free Python for binary checks, cheap models for triage, full LLMs only for semantic flags—dropping costs 80%+ on real sites.
How do I run the tiered SEO auditor?
Clone the repo, python main.py –project myclient –pro –tiered –rewrite. Needs license for premium; core’s free forever.
Does tiered SEO auditing work at scale?
Yes—1,000 URLs cost ~$1 vs $6 pre-tiering, with history tracking to spot patterns and latency under 1s for most.
