Recommendation Engine

Candidate Generation Aur Ranking

💡 Recommendation engine EK ACHHA SALESMAN hai — wo poora godaam nahi dikhata, pehle 500 cheezein chhaantta hai jo tumhe pasand aa sakti hain, phir unme se best 10 saamne rakhta hai.

Har production recommender DO STAGE ka hota hai. CANDIDATE GENERATION: crore items mein se ~1000 nikaalo — fast, approximate methods se (collaborative filtering, embedding similarity, trending). RANKING: un 1000 ko heavy ML model se score karke top 10 nikaalo. Do stage isliye ki crore items par heavy model chalana impossible hai.

Serving path mein latency budget ~100ms hai, isliye user aur item EMBEDDINGS precomputed hote hain aur vector DB (approximate nearest neighbour) se similarity search hoti hai. Model training completely offline hota hai — daily ya hourly batch. Cold start (naya user) ke liye popularity-based fallback rakho.

// Two-stage — har bade recommender ka shape
Request → Candidate Generation (~1000 items, <20ms)
            ├─ collaborative filtering (similar users)
            ├─ embedding ANN search (vector DB)
            └─ trending / popular
        → Ranking model (~1000 items, <50ms)
        → business rules (diversity, already-seen filter)
        → top 10

🎯

Recommendation engine EK ACHHA SALESMAN hai — wo poora godaam nahi dikhata, pehle 500 cheezein chhaantta hai jo tumhe pasand aa sakti hain, phir unme se best 10 saamne rakhta hai.

1 / 2

⚡ Quick Recap

Do stage: candidate generation (fast, approximate) → ranking (heavy ML)
Embeddings precomputed, vector DB se ANN search
Training offline batch; cold start ke liye popularity fallback

Is page mein (2 subtopics)

COLLABORATIVE filtering kehta hai "tumhare jaise logon ne ye pasand kiya" — ye user behaviour par chalta hai aur surprising, achhi recommendations deta hai. Par naye items ke liye kaam nahi karta (koi interaction hi nahi hai).

CONTENT-BASED filtering item ke attributes par chalta hai — "tumne action movie dekhi, ye bhi action hai". Ye naye items handle kar leta hai par recommendations boring aur predictable ho jaate hain. Production systems dono ka HYBRID use karte hain.

💡Tip: Filter bubble ka zikr karo — sirf similar cheezein dikhane se user ka experience sikud jaata hai. Isiliye ranking ke baad EXPLORATION (kuch random/diverse items) inject kiya jaata hai.

Model training BATCH hoti hai — roz ya har kuch ghante, poore historical data par. Serving ONLINE hoti hai — 100ms mein. Ye do bilkul alag systems hain aur inhe alag rakhna hi correct architecture hai.

Beech ka pul FEATURE STORE hai: training aur serving dono ek hi feature definitions use karte hain. Agar training aur serving alag tarike se features banayein to TRAINING-SERVING SKEW aata hai — model offline achha dikhta hai par production mein kharab karta hai.

💡Tip: Training-serving skew ka naam lena bahut strong signal hai — ye ML systems ka sabse common production failure hai aur feature store ka asli reason hai.

← Previous🔍 Search Autocomplete System Next →📢 Online Advertising System

Code10x

Recommendation Engine

Collaborative vs Content-Based Filtering

Offline Training, Online Serving