Code10x

Chhe steps — functional requirements, non-functional requirements, scale estimation, API design, data model, aur high level architecture plus deep dive. Har problem inhi par chalti hai; turant boxes banana shuru karna sabse badi galti hai.

Functional batate hain system KYA karega (upload, feed, search). Non-functional batate hain KAISA hoga — scale, latency, availability, consistency. Architecture non-functional se nikalta hai; same features 10 lakh aur 10 crore users par bilkul alag design maangte hain.

DAU × requests per user = requests/day. Usse divide 100,000 (86,400 ka round) = average QPS. Peak QPS usually 2-3x. Storage = record size × records/day × retention. Precision nahi, order of magnitude chahiye.

Uska architectural conclusion — "30,000 QPS par single DB nahi chalega, isliye read replicas aur sharding chahiye". Numbers nikaalna aadha kaam hai; unka matlab batana asli skill hai.

Poore system ke liye ek choice mat karo — PER FEATURE choose karo. Payment ledger CP (consistency zaroori), social media like count AP (availability zaroori). Ek hi system mein dono ho sakte hain.

Read-heavy ka jawab cache, read replicas aur CDN hai. Write-heavy ka jawab queue, batching aur append-optimized storage (LSM tree) hai. Estimation ke turant baad ye ratio bolna chahiye taaki baaki design justified lage.

Range-based range queries ke liye achhi hai par hotspot banati hai (ek popular range ek shard maar deti hai). Hash-based data barabar baantti hai par range queries todti hai. Resharding ke liye consistent hashing chahiye.

Simple "hash(key) % N" mein N badalte hi lagbhag saara data move karna padta hai. Consistent hashing mein nodes ek ring par hote hain, isliye naya node add karne par sirf padosi ka thoda data move hota hai.

Popular key expire hote hi hazaaron requests ek saath DB par chali jaati hain. Fix: ek hi request ko refresh karne do (lock), baaki ko purana value do; ya probabilistic early refresh jisse keys ek saath expire na hon.

Jo key exist hi nahi karti uske liye har request cache miss hokar DB tak jaati hai — attackers isse DB gira sakte hain. Fix: null results ko bhi short TTL ke saath cache karo, ya bloom filter se pehle hi reject karo.

Cache-aside: app miss par DB se laakar cache bharta hai (sabse common). Write-through: write cache aur DB dono mein ek saath (consistent, slow). Write-behind: pehle cache, baad mein DB (fast, par crash par data loss).

Har transaction do entries banata hai — ek debit, ek credit — aur unka jod hamesha zero rehta hai. Isse hisaab hamesha balance rehta hai aur koi bhi gadbadi turant pakdi jaati hai. Ledger append-only hota hai; correction bhi nayi entry se hoti hai.

Saga. 2PC blocking hai aur coordinator down hone par saare participants atak jaate hain. Saga mein har step ka compensating action hota hai aur fail hone par ulte order mein chalte hain. Compensating action ek naya transaction hota hai (refund), undo nahi.

Ledger ke liye kabhi nahi — wahan strong consistency chahiye. Par notification, analytics, reporting aur dashboards ke liye eventual consistency bilkul theek hai. Ye farak batana strong signal hai.

Tokenization — card details seedhe payment provider ko jaati hain aur wo token lautaata hai. Tum sirf token store karte ho, jisse future charges ho sakte hain par jo tumhare merchant account ke bahar bekaar hai. Isse PCI-DSS scope bahut chhota ho jaata hai.

Balance ko mutable column nahi, ledger entries ka DERIVED jod maano. Performance ke liye periodic snapshot rakho taaki poora ledger baar-baar na padhna pade. Source of truth hamesha ledger rehta hai.

Atomic conditional update — "UPDATE ... SET balance = balance - amt WHERE balance >= amt AND version = :v". Affected rows 0 aaye to ya paise kam the ya koi aur pehle update kar gaya. Read-then-write race condition hai.

Ek popular merchant wallet par hazaaron concurrent transactions us ek row par lock contention bana dete hain. Solution: balance sharding — ek logical wallet ko N sub-balances mein baanto, credit random shard par, balance sabka jod. Contention N guna kam.

SQL LIKE query se na relevance milti hai, na typo tolerance, na facets. Elasticsearch alag rehta hai aur catalog DB se async sync hota hai — matlab search index thoda peeche reh sakta hai, jo acceptable hai aur explicitly bolna chahiye.

Inventory counter Redis mein atomic DECR se rakho aur orders queue se serialize karo. Direct DB par 1 lakh concurrent writes kaam nahi karenge. Counter khatam hote hi turant reject karo, DB tak jaane hi mat do.

Redis atomic INCR se — limit cross hone par DECR karke reject. DB counter par race condition ban jaayegi. Per-user limit ke liye Redis SET mein userId rakho.

Reserve with TTL. User checkout chhod sakta hai — turant redeem karne par quota jhoothe redemptions se khatam ho jaayega. Order confirm hone par hi final redeem karo.

10 lakh drivers har 4 second update bhejein to ~250K writes/sec — koi bhi OLTP database gir jaayega. Live position in-memory geo store (Redis GEO) mein jaati hai, aur history Kafka se cold storage mein.

Cell boundary par do bahut nazdeek points ke geohash bilkul alag ho sakte hain, isliye sirf apni cell dekhna kaafi nahi — 8 padosi cells bhi check karni padti hain. Redis GEO ye andar handle kar leta hai.

Ride ko rollback mat karo — wo physically ho chuki hai. Use "completed but unpaid" mark karo aur alag recovery flow chalao. Physical duniya ke actions undo nahi hote, ye distinction important hai.

Order state change ek event banata hai jise kai independent consumers sunte hain — notification, delivery assignment, analytics, ETA. Naya feature (loyalty, fraud check) ek naya consumer hai; order service badalni nahi padti.

Static data (location, cuisine, rating) ke saath DYNAMIC state bhi chahiye — restaurant abhi khula hai, kitchen overloaded to nahi. Static geo index se candidates lo, phir Redis se dynamic state lookup karke filter karo.

Normal load ke liye nahi, BURST ke liye. Normal 100 QPS, drop par 5 lakh QPS. Isliye virtual waiting room se traffic flatten karo aur seat inventory Redis mein rakho, DB par nahi.

Users ko queue mein rakha jaata hai (Redis sorted set), har ek ko position aur token milta hai, aur system controlled rate se tokens active karta hai. Backend ko flat predictable load milta hai aur user ko random error ke bajaye position dikhti hai.

Nahi — do sources of truth se oversell hota hai. Sahi jawab hai booking BAND kar dena. Oversell reputational disaster hai aur refund se maamla theek nahi hota; yahan availability se correctness chunna sahi call hai.

Product page par "in stock" dikhana eventual consistency se chal sakta hai (3 second purana chalega), par checkout par reserve karna strong consistency maangta hai. Ye farak pehchanana is problem ka core insight hai.

Kabhi nahi. Video blob storage (S3) mein chunks ke roop mein jaata hai; database mein sirf metadata aur pointer rehta hai. Ye galti turant pakdi jaati hai.

Video ko chunks mein todkar PARALLEL transcode karo — 100 chunks, 100 workers. Pipeline ek DAG hai: validate → split → parallel transcode (har resolution) → thumbnail → package (HLS/DASH) → CDN push. Har step retry-able hona chahiye.

PLAYER, server nahi. Manifest file mein saare quality levels ke segment URLs hote hain; player apni network speed dekhkar agla segment kis quality mein lena hai wo khud decide karta hai.

On-demand mein poora video pehle se hai aur transcode ka time hai. Live mein transcoding real-time honi chahiye aur segments chhote (2-4 sec). Sub-second latency chahiye to WebRTC, par wo CDN se scale nahi karta.

Invalidation karni hi na pade — VERSIONED URLs use karo. Content hash URL mein daalo (app.a3f9c2.js) aur immutable cache header lagao. File badle to URL badlo; purana cache apne aap bekaar ho jaata hai.

Edges aur origin ke beech ek extra cache layer. Bina shield ke 100 edges ek hi file ke liye origin par 100 requests bhejte hain; shield ke saath origin ko sirf 1 request jaati hai. Ye origin par thundering herd rokta hai.

Fanout on write: feed precompute hoti hai, read bahut tez, par celebrity ke ek post par crore writes. Fanout on read: write sasta, read slow. Asli jawab HYBRID hai — normal users ke liye write fanout, celebrities ke liye read fanout.

Offset-based pagination tootta hai — naye posts aane par items shift ho jaate hain aur user ko duplicates dikhte hain. Solution cursor-based pagination hai with a stable snapshot per feed session.

Skip kar dete hain. Jo user 30 din se login nahi kiya uske liye feed precompute karna waste hai — login karte hi on-demand build karo. Bade platforms par 70-80% users daily active nahi hote, isliye ye bada optimization hai.

Trie, jisme har node par top-K suggestions PRECOMPUTED hon. Bina precompute ke har query par poora subtree scan karna padega, aur latency budget sirf ~100ms hai.

Nahi. Search logs Kafka mein jaate hain, batch job (hourly) frequencies aggregate karta hai, naya trie build hota hai aur atomically swap ho jaata hai. Read path aur write path bilkul alag hain — yahi is problem ka core insight hai.

Client-side debounce (har keystroke par request mat bhejo), browser cache, aur edge caching. Ye backend traffic 5-10 guna kam kar dete hain aur bahut kam candidates mention karte hain.

Crore items par heavy ML model chalana impossible hai. Isliye pehle candidate generation (~1000 items, fast approximate methods se), phir un 1000 par heavy ranking model. Ye har production recommender ka shape hai.

Collaborative user behaviour par chalta hai ("tumhare jaise logon ne ye pasand kiya") — surprising recommendations deta hai par naye items handle nahi karta. Content-based item attributes par chalta hai — naye items handle karta hai par boring hota hai. Production hybrid use karta hai.

Naye user ke paas history nahi hai, isliye popularity-based aur demographic-based recommendations do, aur jaise interactions aayein personalization badhao. Ye lagbhag hamesha poocha jaata hai.

Jab training aur serving alag tareeke se features compute karte hain — model offline achha dikhta hai par production mein kharab karta hai. Feature store ka asli reason yahi hai: dono ek hi feature definitions use karein.

Ranking eCPM par hoti hai = bid × predicted CTR. Kam bid par high-CTR ad zyada bid par low-CTR ad ko haraa sakta hai, kyunki platform apni expected earning maximize karta hai.

Har server ko budget ka chhota slice allot karo (local counter) aur periodically central se sync karo. Exact global counter har impression par network round-trip maangega, jo 100ms budget mein possible nahi. Thoda overspend acceptable maana jaata hai.

Kuch nahi — wo chalti rahegi. Agent local disk par buffer karta hai aur buffer bharne par purane logs DROP karta hai. Logging kabhi bhi critical path mein nahi honi chahiye.

Backpressure absorb karne ke liye. Elasticsearch slow ho ya down ho to Kafka buffer kar leta hai aur logs khote nahi. Bina buffer ke index down hone par logs seedhe gum ho jaate hain.

Tiered retention (7 din hot, 30 din warm, 1 saal cold S3), sampling (ERROR 100%, INFO 10%, DEBUG 1%), time-based indices taaki purana data drop karna instant ho, aur non-searchable fields ko index na karna.

Metrics numeric time series hain aur purpose-built TSDB unhe delta+compression se 10-20x kam jagah mein rakhta hai. Logs text hain aur full-text index maangte hain. Dono ke storage aur query patterns bilkul alag hain.

user_id ya request_id ko label banane par crore time series ban jaati hain aur monitoring system gir jaata hai. Labels hamesha bounded cardinality ke hone chahiye (endpoint, status, region) — unbounded IDs kabhi nahi.

Pull (Prometheus) simple hai, config central hai, aur scrape ka fail hona hi free "target down" signal hai. Push short-lived jobs (batch, serverless) ke liye zaroori hai jo scrape hone tak zinda hi nahi rehte. Practical systems dono support karte hain.

SYMPTOM par hota hai, cause par nahi — "error rate 5% se upar" achha hai, "CPU 80%" bura hai. Aur uske saath DURATION hona chahiye ("5 minute tak"), warna flapping se alert fatigue aa jaati hai.

Application log developer ke liye hai — sampling, delete aur best-effort delivery allowed. Audit log legal evidence hai — sampling kabhi nahi, delete kabhi nahi, guaranteed delivery. Audit event store na ho paaye to ACTION HI FAIL hona chahiye.

Hash chaining — har record mein pichhle record ka hash hota hai, isliye beech ka koi record badalne par poori chain toot jaati hai. Storage WORM hoti hai (S3 Object Lock) jahan delete allowed hi nahi.

Lambda mein do paths hain — speed layer (real-time, approximate) aur batch layer (accurate) — matlab same logic do jagah maintain karni padti hai. Kappa mein sirf stream processing hai aur historical reprocessing ke liye stream replay kiya jaata hai. Aaj Kappa default hai.

Percentiles ka average lena GALAT hai — do servers ke p99 ka average p99 nahi hota. Isliye mergeable sketches (t-digest, HdrHistogram) use hote hain jinhe jodkar sahi combined percentile nikalta hai.

Metadata aur data ko ALAG karna. Content blob storage mein chunks ke roop mein, metadata (naam, path, permissions, chunk list) database mein. Metadata chhota par transactional hai, data bada par simple — scaling needs bilkul alag hain.

Teen — parallel upload, resume on failure, aur chunk-level deduplication. Har chunk ka hash nikaalo; wahi chunk pehle se ho to dobara store mat karo. Sync ke liye sirf badle hue chunks bhejo, poori file nahi.

3x replication simple hai aur read fast hai par 3 guna storage leta hai. Erasure coding (jaise 6+3) 1.5x storage mein wahi durability deta hai par reconstruction slow hai. Hot data replication par, cold data erasure coding par.

"Last write wins" mat karo — wo data loss hai. Dono versions rakho aur ek ko "conflicted copy" naam se dikhao. Automatic merge sirf structured data ke liye possible hai (CRDTs), binary files ke liye nahi.

202 Accepted — request queue mein daalkar. SMTP slow hai aur caller ko usse block nahi karna chahiye. Actual sending workers async karte hain.

Transactional email (OTP, password reset) bulk marketing ke peeche kabhi nahi lagni chahiye. Bulk campaign chalte waqt OTP 10 minute late aana product failure hai.

Per-domain throttling (har provider ki apni limits), hard bounce par turant suppression list, soft bounce par limited retry, aur complaint rate 0.1% se neeche rakhna. Warna sender reputation girti hai aur saari mails spam mein jaati hain.

Nahi — distributed systems mein exactly-once delivery possible nahi hai. Jo milta hai wo at-least-once delivery plus IDEMPOTENT jobs hai, jisse effect exactly-once jaisa ho jaata hai. Ye distinction interviewers bahut value karte hain.

Atomic conditional claim — "UPDATE jobs SET status='RUNNING', worker_id=:me WHERE job_id=:id AND status='PENDING'". Affected rows 1 aaye to tumne claim kiya. Worker crash par lease/heartbeat timeout se job wapas PENDING ho jaati hai.

Sirf SCHEDULING decisions ke liye (cron evaluate karna) — taaki ek hi jagah faisla ho. EXECUTION ke liye nahi; workers stateless hain aur atomic claim se coordinate kar lete hain. Har jagah leader lagana bewajah single point of failure banata hai.

SANDBOXING — tum anjaan code chala rahe ho. Har submission isolated container/microVM mein chale jisme no network, read-only filesystem, aur hard CPU/memory/process/output limits hon. Container har submission ke liye naya bane aur turant destroy ho.

Wall clock par sleep() se cheat kiya ja sakta hai aur network wait bhi count ho jaata hai. CPU time actual computation measure karta hai. Saath mein process limit bhi chahiye warna fork bomb machine gira dega.

Shared database + shared schema (tenant_id column) — sasta aur scalable par data leak ka risk sabse zyada. Schema per tenant — beech ka rasta. Database per tenant — sabse safe, enterprise clients ki demand, par hazaaron tenants par operationally bhaari.

Tenant filtering ko developer discipline par mat chhodo — ek missing WHERE clause data breach hai. Framework level par enforce karo: Postgres row-level security, ya ek mandatory repository layer jo tenant_id apne aap inject kare.

Ek bada tenant saare shared resources kha jaata hai aur baaki tenants slow ho jaate hain. Solutions: per-tenant rate limiting aur quotas, aur bade tenants ko dedicated infrastructure par shift karna. Practical answer hybrid hai — chhote shared, bade dedicated.

Har request par remote call latency add karta hai aur ek naya single point of failure banata hai. SDK saare flag rules locally cache karta hai aur evaluation in-process hoti hai — microseconds mein. Rules chhote hote hain, isliye ye practical hai.

Kuch nahi tootna chahiye. SDK last known config use kare, aur wo bhi na ho to code mein diya gaya default. Flag service ka down hona kabhi application down nahi karna chahiye.

Consistent hashing — bucket = hash(flagKey + userId) % 100. Same user hamesha same bucket mein rehta hai. flagKey ko hash mein daalna zaroori hai, warna har flag ke liye wahi 10% users select honge.

Do layers — pehle rules engine (fast, deterministic, explainable), phir ML model score. Combined output teen actions deta hai: allow, challenge (OTP), ya block. Poora budget ~50ms ka hota hai.

Request time par compute karna impossible hai. Streaming job (Flink) ye aggregates continuously maintain karta hai aur feature store (Redis) mein likhta hai; serving time par sirf O(1) lookup hoti hai. Yahi is system ka sabse important decision hai.

False positive ka cost bahut zyada hai — asli grahak ko block karna aksar fraud se mehnga padta hai. Isliye teesra action chahiye: CHALLENGE (OTP, captcha) jo suspicious par blocked nahi wale cases handle karta hai.

Chargeback aane mein hafte lag sakte hain, isliye training labels late milte hain. Do tarah ke signals use hote hain — fast (analyst review, complaints) aur slow (chargebacks) — aur model evaluation mein ye delay account karna padta hai.