Social Media News Feed
FANOUT ON WRITE: post karte hi wo saare followers ki precomputed feed list mein likh di jaati hai. Feed read BAHUT tez (ek lookup), par 10 crore followers wale celebrity ke ek post par 10 crore writes — ye system tod deta hai.
FANOUT ON READ: feed maangne par followees ke posts real-time mein merge kiye jaate hain. Write sasta, par read slow. Asli systems HYBRID use karte hain — normal users ke liye fanout on write, aur celebrities ke liye fanout on read. Ye hybrid answer hi is problem ka expected jawab hai.
// Hybrid — production systems yahi karte hain
post(user):
if user.followers < 10_000:
fanout_on_write(user.followers) // precompute feed
else:
mark_as_celebrity(user) // kuch mat karo
getFeed(user):
feed = precomputed_feed(user) // normal followees
for celeb in user.celebrity_followees: // celebrities alag se
feed.merge(recent_posts(celeb))
return rank(feed)- Fanout on write: read tez, celebrity par write explosion
- Fanout on read: write sasta, read slow
- Hybrid hi asli jawab hai — celebrities ko alag treat karo
Chronological feed simple hai par engagement kam deta hai, isliye asli systems RANKED feed dete hain — score = recency × affinity × engagement prediction. Ranking ek alag ML service hoti hai.
Ranked feed mein PAGINATION mushkil hai. Offset-based pagination (LIMIT 20 OFFSET 40) tootta hai kyunki naye posts aane par items shift ho jaate hain aur user ko duplicate dikhte hain. Solution: cursor-based pagination with a stable snapshot — feed session ke liye ek fixed cursor rakho.
// ❌ Offset — naye posts aane par duplicates dikhte hain
GET /feed?offset=40&limit=20
// ✅ Cursor — stable, duplicate nahi
GET /feed?cursor=eyJzY29yZSI6MC44LCJpZCI6MTIzfQ&limit=20Precomputed feed lists Redis mein rehti hain (per user ek list). Par har user ki poori history rakhna memory mein possible nahi. Isliye feed ko TRIM karo — sirf latest 500-1000 entries rakho, purana chahiye to DB se laao.
Inactive users ke liye fanout skip karna ek bada optimization hai — jo user 30 din se login nahi kiya, uske liye feed precompute karna waste hai. Login karte hi on-demand build kar do.