📈 शाळेच्या पद्धतीने scaling शिका

# the 60-second wow — results day, simulated:
git clone https://github.com/BaluRaut/learn-scaling-school.git && cd learn-scaling-school
python3 scale/demo.py              # 13 lessons: every number, every run the same
python3 scale/test_scale.py        # 13 checks across the lessons

तुम्हाला काय दिसायला हवे (छाटलेले — यश असे दिसते):

═══ why ═══
── results day: 40 parents a second became 2,000 when the results went up
   scale UP  = a bigger counter (one server with more CPU) — simple, but there is always a biggest size
   scale OUT = more counters (more servers behind a load balancer) — no ceiling, but the app must allow it
      40 req/s at 150 req/s per server (run at 60%) →   1 server(s)
     400 req/s at 150 req/s per server (run at 60%) →   5 server(s)
    2000 req/s at 150 req/s per server (run at 60%) →  23 server(s)
   the slowest part sets the pace: the page, the API and the database each need their own plan

═══ measure ═══
── 20 requests timed at the counter (ms): 80 85 90 92 95 98 100 105 110 120 130 150 180 240 400 95 88 102 97 1200
   p50    100 ms
   p95    400 ms
   p99   1200 ms
   average 182.8 ms — one 1,200 ms visit hides inside it; percentiles show it
── Little's law: 2,000 req/s × 0.120 s = 240 requests in flight at once
   that is how many threads / connections / Lambda environments you need at that moment

═══ cdn ═══
── 3,000 requests in one minute for 6 pages, popular pages asked far more
   TTL   0 s → hit ratio   0.0% · origin (S3) sees 3,000 requests
   TTL   1 s → hit ratio  88.0% · origin (S3) sees   359 requests
   TTL  10 s → hit ratio  98.8% · origin (S3) sees    36 requests
   TTL  60 s → hit ratio  99.8% · origin (S3) sees     6 requests
── /app.js cached for a day: MISS then HIT; new release → invalidate → MISS
   better: versioned file names (app.3f9c.js) — a new name is a new cache key, no invalidation needed

═══ edge ═══
── what each response tells CloudFront and the browser
   /app.3f9c.js   Cache-Control: public, max-age=31536000, immutable
   /results/3A    Cache-Control: public, s-maxage=10, stale-while-revalidate=30
   /me            Cache-Control: private, no-store
── 5 edge locations, s-maxage 10 s, 60 s of traffic: origin gets 30 requests; with an origin shield: 6
   a 10-second cache on a busy dynamic page turns 2,000 req/s into 1 request per 10 s per location
   CloudFront Functions / Lambda@Edge: tiny code at the edge — redirects, headers, A/B cookies, auth checks

═══ balance ═══
── Katrina's 6 clicks, round-robin across 3 servers
   click  server  session in server memory      session in Redis (shared)
       1  srv-1   logged in                     logged in
       2  srv-2   NOT FOUND → login page        logged in
       3  srv-3   NOT FOUND → login page        logged in
       4  srv-1   logged in                     logged in
       5  srv-2   NOT FOUND → login page        logged in
       6  srv-3   NOT FOUND → login page        logged in
   stateless servers: keep sessions, uploads and caches OUTSIDE the server — then any server can answer
   the load balancer health-checks every server and only sends visitors to healthy ones

═══ autoscale ═══
── results go up at minute 2: target 60% CPU, 150 req/s per server, 2-minute warm-up
   min  req/s  serving  util  over capacity (slow / 5xx)
     0    150        2   50%        0
     1    150        2   50%        0
     2    900        2  100%      600
     3   1800        2  100%     1500
     4   2400       10  100%      900
     5   2400       20   80%        0
     6   2400       20   80%        0
     7   1200       20   40%        0
     8    600       19   21%        0
     9    300       18   11%        0
    10    300       17   12%        0
    11    300       16   12%        0
   the spike arrives faster than new servers warm up → schedule a scale-out BEFORE results day,
   keep a warm minimum, and scale in slowly (one server at a time)

═══ kubernetes ═══
── HorizontalPodAutoscaler on CPU utilisation: desired = ceil(replicas × current % / target %), 10% tolerance
   4 pods at 90% CPU, target 60% → 6 pods
   6 pods at 63% CPU, target 60% → 6 pods
   6 pods at 30% CPU, target 60% → 3 pods
   3 pods at 240% CPU, target 60% → 12 pods
── 16 pods requesting 500m CPU on 2,000m nodes → 4 nodes (4 pods fit per node)
   the same pods requesting 1,100m → 16 nodes (1 fits, 900m wasted per node)
   pods that cannot be placed stay Pending → Cluster Autoscaler / Karpenter adds nodes
   honest CPU/memory requests are what make both autoscalers work

═══ serverless ═══
── Lambda: concurrency ≈ requests per second × average duration (steady traffic; bursts need more)
     100 req/s × 200 ms →    20 environments at once
    2000 req/s × 200 ms →   400 environments at once
    2000 req/s ×  50 ms →   100 environments at once
── the results spike, reserved concurrency 300, cold start ~600 ms:
   second 0:   100 req/s → needs  20, runs  20, cold starts  20, throttled     0
   second 1:  2000 req/s → needs 400, runs 300, cold starts 280, throttled   500
   second 2:  2000 req/s → needs 400, runs 300, cold starts   0, throttled   500
   second 3:   500 req/s → needs 100, runs 100, cold starts   0, throttled     0
── with 400 provisioned (pre-warmed) environments: cold starts at 2,000 req/s = 0
   account default: 1,000 concurrent executions per Region (a quota you can raise)

═══ cache ═══
── 6,000 reads of results:3A in a minute, cache TTL 30 s → database reads: 2, hits: 5,998
   the key expires while 500 parents are waiting — single-flight off → database reads: 500
   the key expires while 500 parents are waiting — single-flight on  → database reads: 1
   cache-aside: read the cache, on a miss read the DB and fill the cache; on a write, delete the key

═══ replicas ═══
── Katrina's grade is written to the primary at t=1000 ms; the replica lags 200 ms
   read from the replica at t=1050 ms → None
   read from the replica at t=1150 ms → None
   read from the replica at t=1250 ms → A+
   read-your-own-writes: after a write, read that user's data from the primary for a moment
── 800 Lambda environments × 1 connection, max_connections 500, proxy pool   — → opened 500, refused 300
── 800 Lambda environments × 1 connection, max_connections 500, proxy pool 100 → opened 100, refused 0
   RDS Proxy (or PgBouncer) shares a small pool of real connections among many callers

═══ shards ═══
── 1,000 pupils hashed across 4 partitions → [245, 262, 249, 244]
   partition key = class, results day (900 of 1,000 writes are 3A) → [19, 31, 928, 22] — one HOT partition
   write-sharding the hot key (class-3A#0 … #39) → [179, 253, 277, 291]
   DynamoDB: a partition gives up to 3,000 RCU + 1,000 WCU (1 RCU = one strong 4 KB read/s, 1 WCU = one 1 KB write/s)
   bigger items use more units; adaptive capacity helps a bit — a key that spreads is still the real fix

═══ queues ═══
── certificate PDFs: 800 requests/s for 3 seconds; each worker makes 100 PDFs/s
   fixed 3 workers                                      peak backlog  1500 · left after 10 s  150
   scale on backlog (1 worker per 100 waiting, max 10)  peak backlog   800 · left after 10 s    0
   the queue (SQS) absorbs the burst; workers process messages asynchronously, at their own pace
   SQS delivers at least once: a visibility timeout, retries, a DLQ and idempotent workers make failed work safe to recover
── the whole fair: CloudFront → load balancer / API Gateway → pods or Lambda → Redis → primary + replicas,
   slow work through a queue — and every tier scales on its own signal

═══ resilience ═══
── timeouts: the results API usually answers in 120 ms; today it is stuck at 30 s
   latency    120 ms, timeout 1,000 ms → ('ok', 120)
   latency  30000 ms, timeout 1,000 ms → ('timeout', 1000)
   without a timeout every thread waits 30 s — 2,000 req/s × 30 s = 60,000 stuck requests (Little's law again)
── retries: 500 clients fail at once and retry 4 times (backoff 100 → 800 ms)
   no jitter   → busiest 10 ms:  500 retries hit the database together
   full jitter → busiest 10 ms:   81 retries hit the database together
── circuit breaker (5 failures → open for 10 s → one trial call); the payments service is down for 30 s
   calls that reached the sick service: 7 · failed fast without waiting: 27 · state at 40 s: closed
── graceful degradation: the grade service is down → show the results page from Redis, marked '5 minutes old'
   a slower, older page beats an error page; non-essential parts (photos, recommendations) can switch off
── at-least-once: 4 deliveries for 3 payments → plain worker charges 4, idempotent worker charges 3
── a poison message: 9 PDFs made, ['pdf-7'] moved to the DLQ after 3 receives (12 receives in total)
── multi-AZ: one AZ at 99.5% → two AZs 99.9975% · three AZs 99.999988%
   a chain API 99.9% × Redis 99.9% × database 99.95% = 99.75% — every dependency lowers the total
   scaling adds capacity; resilience keeps the fair open when one part of it breaks

✅ done — the fair survived results day

🗺️ संपूर्ण चित्र — एक आकृती, पूर्ण कोर्स

संपूर्ण कोर्स एका कॅनव्हासवर. 4K आवृत्तीसाठी क्लिक करा.

🖥️ भाग 1 — UI चे scaling (धडे 1–4)

एक git branch = एक कल्पना; branch 04 मध्ये धडे 01–04 आहेत. प्रत्येक आकडा scale/sim.py मधून येतो — cloud account ची गरज नाही.

🔌 भाग 2 — API चे scaling (धडे 5–8)

UI मागे क्षमता वाढवण्याचे चार मार्ग — servers, Auto Scaling groups, Kubernetes आणि serverless — आणि प्रत्येकाला तुमच्या code कडून काय हवे.

🗄️ भाग 3 — database चे scaling (धडे 9–12)

Database ही सहसा सहज copy करता न येणारी शेवटची गोष्ट असते — ती cache करा, वाचनासाठी तिच्या प्रती करा, तिची विभागणी करा, आणि हळू कामे रांगेत टाका.

🛟 भाग 4 — बिघाडातून टिकणे (धडा 13)

क्षमता हे scaling चे फक्त अर्धे काम आहे: जत्रेचा एखादा भाग बिघडला तरी हा धडा जत्रा चालू ठेवतो.

📐 धड्यांच्या आकृत्या — क्रमांक अनुसरा

प्रत्येक धडा एका क्रमांकित बॉक्स-आणि-बाण आकृतीत — स्वतंत्र पानावरही.