Look, we’ve all been sold the dream: slap some OpenAI embeddings on your product catalog, fire up a quick Supabase query, and boom – your chatbot’s a shopping savant, nailing every vague customer whim.
But Ali, grinding away on Provia from Gaza (yeah, building an AI sales platform amid actual chaos), hit the wall hard. Customer types ‘show me something for a date night,’ and the top hit? ‘Cozy Night Deluxe Loungewear Set.’ Pajamas. Because in the foggy 1536-dimensional soup of text-embedding-3-small, ‘date night’ and ‘night loungewear’ are kissing cousins.
Why Vector Search Fails at Shopping Intent
Semantic search? More like semantic suck. It measures cosmic distances between word clouds, not human shopping vibes. ‘Night out outfit’ – pajamas. ‘Good night cream’ – still pajamas. That loungewear black hole slurped up anything with a whiff of ‘night,’ thanks to its keyword-stuffed name and desc.
Here’s Ali’s raw take:
A customer typed “show me something for a date night” and my AI chatbot returned the “Cozy Night Deluxe Loungewear Set” — pajamas — as the top result. Because “night” in “date night” is semantically close to “night” in “loungewear set.”
Brutal. And this in a tiny 15-product catalog. Scale to Shopify’s millions? Nightmare fuel.
The setup was textbook: embed products with OpenAI’s model, query embed, cosine similarity via pgvector in Supabase. Threshold at 0.1 – laughably loose. Anything vaguely English-adjacent passed.
First fix? Crank to 0.3. Killed pajamas. Also killed ‘show me jackets’ – scored 0.28 on ‘Classic Cool Denim Jacket.’ Too tight for terse queries.
Two-tier hack: strong matches over 0.3, nearbies 0.2-0.3 as ‘suggestions.’ Worse. Bot’d pitch pajamas as ‘might work,’ same trash UX.
Bump to 0.25, fetch 10 results. Noise explosion. Pajamas tag along, bot sometimes chats ‘em up.
Is OpenAI’s Embedding Model Useless for E-Comm?
Not useless. Just blind to context. Vector space clusters concepts by co-occurrence stats, not buyer psychology. Date night’s sultry dress? Nah, cozy PJs win on ‘night’ proximity. It’s like 90s keyword search on steroids – same pitfalls, fancier math.
My hot take, unseen in Ali’s post: this echoes AltaVista’s collapse. Remember? Early web search drowned in spam pages juiced with hot keywords. Google crushed it with PageRank’s intent signals. Today’s vector shops need that – hybrid ranks blending embeddings with rules, categories, even sales velocity. Without? You’re funding OpenAI’s embed fees while customers bounce.
Ali’s winner: fallback chain. No matches at 0.3? Don’t panic.
Step 1: Tight semantic (0.3 thresh).
Step 2: Keyword fallback – full-text search on name/desc.
Step 3: Category broadscan – if query hints ‘outfit,’ scan apparel.
Step 4: Zero results? ‘Tell me more’ prompt.
Code evolved smart. Updated func defaults to 0.3, app layers fallbacks:
// Pseudo-ish
if (semanticResults.length === 0) {
keywordResults = await supabase.rpc('keyword_search', {query});
if (keywordResults.length === 0) {
catResults = await get_category_products(extract_category(query));
}
}
Result? Date night now surfaces dresses, tops. No more sleepwear seduction.
But here’s the cynicism: who’s cashing in? OpenAI, billing per embed. Supabase, per query. Devs like Ali burn nights tweaking. Customers? Still wary of AI recs. Small catalogs dodge the worst, but enterprise e-comm? They’ll need armies of engineers – or ditch pure vectors for agentic chains (pricey).
Who Wins from These AI Sales Fails?
Venture types hype ‘AI-native commerce.’ Reality: brittle. One viral pajama tweet tanks trust. Ali’s from Gaza – props for shipping amid blackouts – but his tale screams: embeddings are step one, not the gospel.
Bold prediction: by 2025, 70% of prod search ditches raw similarity for rerankers (Cohere’s?) or fine-tuned intents. Or perish in irrelevance hell.
Devs, steal this: always chain fallbacks. Vectors for fuzzy, keywords for precision, rules for sanity. Test with adversarial queries – ‘date night’ PJs edition.
And yeah, Provia’s open-sourcing bits. Hero move.
Short queries kill thresholds. Long ones? Gold. Train users? Nah, fix the pipe.
Scale warning: 15 products? Cute. 15k? Recrawl embeddings on desc tweaks, or drift kills ya.
Why Does This Matter for AI Devs Building Sales Tools?
Because PR spins ‘semantic search’ as solved. It’s not. Ali’s bug isn’t edge – it’s the norm. I’ve seen it in dev Slack channels weekly: ‘Why’s my RAG hallucinating products?’
Unique insight: this foreshadows agent economy woes. Agents query embeddings, act. Wrong vector hits? Wrong buys, returns spike, blame AI.
Fixes beyond Ali’s:
-
Hybrid: BM25 + vectors (pgvector has it).
-
Fine-tune embeddings on shopping data (pricey, but Anthropic’s got tools).
-
Intent classifiers first – spaCy/NER flags ‘clothing/date’ buckets.
Ali nailed the chain. Pragmatic win.
🧬 Related Insights
- Read more: ckpt: Git-Powered Undo Buttons for AI Agents That Keep Wrecking Your Code
- Read more: 6 Hours to 8,000 Lines: The AI Pipeline That Nailed a Live Artemis II Tracker
Frequently Asked Questions
Why does AI recommend pajamas for date night?
Vector embeddings cluster by word similarity, not context – ‘night’ links PJs to dates.
How to fix vector search bugs in product catalogs?
Use fallback chains: tight semantic -> keyword -> category scan. Raise thresholds wisely.
Is OpenAI text-embedding-3-small good for e-commerce?
Decent start, but needs hybrids/rerankers for intent-heavy tasks like shopping.
