Fraud Detection System
Architecture do layers ka hota hai. RULES ENGINE — fast, deterministic, explainable ("ek ghante mein 5 se zyada failed payments = block"). ML MODEL — patterns pakadta hai jo rules miss karte hain. Rules pehle chalte hain (sasta, turant), phir model score karta hai. Dono ka combined output action decide karta hai: allow, challenge (OTP/captcha), ya block.
Sabse mushkil hissa FEATURE COMPUTATION hai. Model ko chahiye "is user ke pichhle 24 ghante ke transactions" — ye real-time mein compute nahi ho sakta. Isliye FEATURE STORE use hota hai jahan streaming aggregates (Flink) pehle se calculate hokar padi rehti hain aur serving time par sirf lookup hoti hai.
Transaction → Rules Engine (in-memory, <5ms)
↓ pass
Feature Store lookup (precomputed aggregates, <10ms)
↓
ML model score (<30ms)
↓
score > 0.9 -> block
score > 0.6 -> challenge (OTP)
else -> allow
// Async: har decision Kafka mein -> model retraining + analyst review- Do layers: fast rules engine, phir ML score
- Feature store mein precomputed aggregates — runtime compute impossible hai
- Teen actions chahiye: allow / challenge / block, sirf do nahi
Model ko chahiye "pichhle 1 ghante mein is card se kitne transactions", "is device se kitne alag accounts". Ye aggregates request time par compute karna impossible hai — 50ms budget mein poora history scan nahi ho sakta.
Isliye streaming job (Flink) ye aggregates continuously maintain karta hai aur feature store (Redis) mein likhta hai. Serving time par sirf lookup hoti hai — O(1). Yahi is system ka sabse important architectural decision hai.
// Streaming aggregation — continuously updated
Transactions → Flink (sliding windows)
card_txn_count_1h, device_account_count_24h,
user_avg_amount_30d, ip_velocity_5m
→ Redis feature store
// Serving: sirf lookup, compute nahiFraud patterns badalte rehte hain, isliye model ko continuously retrain karna padta hai. Labels aate hain chargebacks aur manual analyst review se — par chargeback aane mein hafte lag sakte hain, isliye label delay ek asli problem hai.
Isliye do tarah ke signals use hote hain: fast signals (analyst review, user complaints) aur slow signals (chargebacks). Model evaluation mein ye delay account karna padta hai, warna model apne aap ko galat aankta hai.