📐 शाळेच्या पद्धतीने system design शिका

# the 60-second wow — the planning office, live:
git clone https://github.com/BaluRaut/learn-system-design-school.git && cd learn-system-design-school
python3 design/demo.py             # 18 lessons: every estimate and every trade-off
python3 design/test_design.py      # 18 checks across the lessons

तुम्हाला काय दिसायला हवे (छाटलेले — यश असे दिसते):

═══ requirements ═══
── the brief: 'build a notice board app for 5 million parents'. Before drawing a box, write the sheet:
   functional     a teacher posts a notice to a class
                  a parent reads the notices of their children's classes
                  a parent gets a notification for urgent notices
   non-functional read p99 < 200 ms · post p99 < 500 ms
                  99.9% available
                  reads outnumber posts ~100 : 1
                  a notice must never be lost once 'posted' is shown
   constraints    team of 4 · AWS · Marathi + English
                  budget: a few thousand dollars a month
                  launch in 3 months
   out of scope (say it!): chat, payments, video · questions to ask: how many schools? peak time? data kept how long?

═══ capacity ═══
── 5M parents · 1M notices a day across all schools · 100 reads per notice · 2 KB each · peak ×3 · 5 years · 3 copies
   writes_per_s   11.6
   reads_per_s    1157.4
   avg_rps        1169
   peak_rps       3507
   servers        12
   storage_tb     10.95
   egress_mb_s    7.0
   rules of thumb: a day ≈ 86,400 s ≈ 10^5 · 1M/day ≈ 12/s · 2 KB × 1M = 2.0 GB
   the answer is a SHAPE, not a digit: tiny writes, heavy reads, ~11 TB in 5 years → cache the reads; one database cluster is enough

═══ api ═══
── the contract first — resources, not actions:
   POST /classes/3A/notices          body {title, text, urgent} · header Idempotency-Key → 201 + Location
   GET  /classes/3A/notices?limit=20&cursor=eyJpZCI6OTB9 → 200 {items, next_cursor}
   GET  /parents/me/feed?limit=20   → the notices of all my children's classes
   errors: 400 bad input · 401 who are you · 403 not your class · 404 · 409 key reused with a different body · 429 slow down
── pagination: rows the database walks to show page N (20 per page)
   page     1: OFFSET walks      20 rows · a cursor walks  20
   page    50: OFFSET walks   1,000 rows · a cursor walks  20
   page  5000: OFFSET walks 100,000 rows · a cursor walks  20
   version in the path (/v1) or a header; never break a published field — add new ones

═══ data ═══
── model the data from the questions you will ask it (access patterns), then pick the store
   joins, transactions          → relational (PostgreSQL / Aurora)
   huge-scale, key-lookup       → key-value / wide-column (DynamoDB / Cassandra)
   search                       → search engine (OpenSearch)
   similarity                   → vector index (pgvector / OpenSearch k-NN)
   files                        → object storage (S3)
   (nothing special)            → relational (PostgreSQL) — the safe default until an access pattern says otherwise
   notice board: notices(id, class_id, author, title, text, urgent, created_at) · index (class_id, created_at DESC)
   one table, one index, one question answered fast — add stores only when a new access pattern needs one

═══ cache ═══
── 20,000 feed reads over 5,000 classes (a few classes are very popular) through an LRU cache
   cache holds    50 classes (1% of them) → hit ratio  33.8%
   cache holds   500 classes (10% of them) → hit ratio  64.0%
   cache holds  2500 classes (50% of them) → hit ratio  83.1%
   a cache holding 1% of the classes already catches a third of the reads — sizing is a curve, measure it
   where: browser · CDN · API (Redis) · database buffer · invalidate on write or use a short TTL

═══ balance ═══
── 10,000 classes on 4 cache servers; a 5th server joins
   hash % N         → 8,001 keys move (80%)
   consistent hash  → 1,975 keys move (20%) — about 1/5, only the new server's share
   keys per server with 100 virtual nodes each: {'s0': 1883, 's1': 2016, 's2': 1989, 's3': 2137, 's4': 1975}
   load balancers: L4 (TCP, fast) vs L7 (HTTP paths, headers) · health checks · round robin / least connections

═══ queues ═══
── 'send 5M urgent SMS' must not run inside the request
   request: save the notice, put ONE job on a queue, answer 201 in ~50 ms
   workers: take jobs, send in batches of 500, retry with backoff, park failures in a DLQ
    10 workers × 50 SMS/s → all 5M sent in  166.7 min
    50 workers × 50 SMS/s → all 5M sent in   33.3 min
   200 workers × 50 SMS/s → all 5M sent in    8.3 min
   queues decouple speed: the request is fast, the work is as fast as you pay for · delivery is at-least-once

═══ events ═══
── 'NoticePosted' is one event; many services react (email, SMS, search index, analytics) — pub/sub
   dual write, crash between DB and broker = False → ('notice saved', 'NoticePosted')
   dual write, crash between DB and broker = True  → ('notice saved', None)
   outbox, crash before publishing → saved (event waits in the outbox) · relay runs later → saved + published [('NoticePosted', 'N-1')]
   fan-out push: a class with 40 parents, parents follow 3 classes, 5 posts/day, 400 reads/day → {'writes': 200, 'read_lookups': 400}
   fan-out pull: a class with 40 parents, parents follow 3 classes, 5 posts/day, 400 reads/day → {'writes': 5, 'read_lookups': 1200}
   events are facts in the past tense; consumers must be idempotent (the same event may arrive twice)

═══ services ═══
── one request that calls services in a line (each 20 ms, each 99.9% available)
   1 hops →  20 ms, availability 99.90%
   3 hops →  60 ms, availability 99.70%
   8 hops → 160 ms, availability 99.20%
   start as a MODULAR MONOLITH: one deploy, clear modules; split a service out when a team or a scaling need asks
   a 'distributed monolith' = many services that must deploy together — all the cost, none of the benefit

═══ consistency ═══
── 3 copies (N=3): which read/write sizes are sure to see the latest write? (R + W > N)
   R=1 W=1 → eventual: may read a stale copy
   R=1 W=3 → strong: overlaps
   R=2 W=2 → strong: overlaps
   R=3 W=1 → strong: overlaps
   copies [(7, 'exam on Monday'), (8, 'exam moved to Tuesday'), (7, 'exam on Monday')] · read R=1 → 'exam on Monday' · read R=2 → 'exam moved to Tuesday'
   CAP: during a network split you choose — answer (maybe stale) or refuse (stay correct)
   most apps: strong for money and grades, eventual for likes and view counts

═══ transactions ═══
── a school-trip booking touches 3 services: seats, payment, bus
   2PC, all vote yes → COMMIT · one votes no → ABORT
   2PC, the coordinator dies after 'prepare' → BLOCKED — participants hold locks until the coordinator returns
   saga, the bus is full → COMPENSATED: ✓ reserve seat · ✓ charge Katrina ₹500 · ✗ book the bus · ↩ refund Katrina ₹500 · ↩ release seat
   sagas trade 'all at once' for 'eventually consistent, with an undo for every step'

═══ ratelimit ═══
── one parent's app sends 12 requests at t=0 s and 5 more at t=0.5 s
   token bucket (5/s, burst 10) → 12 of 17 allowed · sliding window (10 per 1 s) → 10 of 17
   the bucket refilled 2.5 tokens in 0.5 s, so 2 more got in; the window still counts the first 10 until t=1.0 s
   token bucket: smooth rate + a burst allowance · window: a hard count per period
   limit per user, per API key and globally; answer 429 with Retry-After; keep the counters in Redis

═══ resilience ═══
── the budget for one page is 800 ms; each dependency gets a timeout inside it
   auth           timeout  50 ms
   notices DB     timeout 150 ms
   photo service  timeout 300 ms
   photo service down → show notices without photos (degrade), breaker open, retries with jitter
   one AZ 99.5% → three AZs 99.99999% · deeper dive: the Scaling school, lesson 13

═══ security ═══
── STRIDE, asked of every box on the diagram (here: the notice API)
   Spoofing                 notice API: who is really calling? → authentication (OIDC, mTLS, signed requests)
   Tampering                notice API: was it changed on the way or at rest? → TLS, signatures, checksums, least-privilege writes
   Repudiation              notice API: can someone deny doing it? → audit logs they cannot edit (CloudTrail, append-only)
   Information disclosure   notice API: who can read it? → encryption, authorization per object, no secrets in logs
   Denial of service        notice API: can it be flooded? → rate limits, WAF, quotas, autoscaling with a ceiling
   Elevation of privilege   notice API: can a user become an admin? → least privilege, input validation, isolation
   plus: parents see only their children's classes (authorization per object), PII encrypted, secrets in a vault

═══ observability ═══
── decide what 'healthy' means BEFORE launch
   SLO 99.00% → error budget  432.0 minutes per 30 days
   SLO 99.90% → error budget   43.2 minutes per 30 days
   SLO 99.99% → error budget    4.3 minutes per 30 days
   RED per API (Rate, Errors, Duration) · USE per resource (Utilisation, Saturation, Errors)
   logs with a request id · metrics for alerts · traces across services · alert on the SLO, not on CPU

═══ cost ═══
── a first bill for the notice board (example prices)
   servers   $   360.00
   storage   $    46.00
   egress    $   450.00
   requests  $   360.00
   total     $ 1,216.00
── the decision, written down:
   # ADR 001: Cache the class feed in Redis
   
   Status: Accepted
   
   ## Context
   Reads are 100x writes; p99 must stay under 200 ms at 3x peak.
   
   ## Decision
   Cache-aside per class, TTL 60 s, delete on post.
   
   ## Consequences
   - ~80% of reads skip the database (lesson 05's curve)
   - a notice can be up to 60 s late for some readers
   - one more thing to run
   
═══ designs1 ═══
── URL shortener: 100M new links/month, 100 reads each → 39 writes/s, 3,858 reads/s, peak 11,690 req/s
   6,000,000,000 ids in 5 years → base62 needs 6 characters (62^6 ≈ 56.8 billion) · the last one, id 5,999,999,999 → '6y3o5x'
   read path: cache → key-value store → 301/302 redirect · write path: id generator → store
── notifications: an author with       300 followers posts 3 times → {'mode': 'push on write', 'inbox_writes_per_day': 900}
── notifications: an author with 2,000,000 followers posts 3 times → {'mode': 'pull on read (celebrity)', 'inbox_writes_per_day': 0}
── chat: 1M online, 50k WebSocket connections per server → 20 gateway servers · 463 msg/s · 8.0 GB/day
   arrival order ['see you', 'hi', 'hello'] → by sequence number ['hi', 'hello', 'see you']

═══ designs2 ═══
── file storage: a file split into chunks by content hash; one word changed → upload 2 of 12 chunks
   metadata (who, which chunks, which version) in a database · chunks in object storage · dedupe for free
── video: one uploaded minute becomes 4 renditions (1080p, 720p, 480p, 240p) → 0.07 GB per minute stored
   1M views × 10 minutes at 2.5 Mb/s → 187.5 TB from the CDN — egress, not storage, is the bill
── RAG over 10,000 school documents × 20 pages → 400,000 chunks · vector index ~1.64 GB · ~1675 prompt tokens per question
   ingest (chunk → embed → index) runs offline; ask (embed question → top-5 → prompt → answer with sources) runs online
── the method, every time: requirements → numbers → API → data → blocks → failure → security → observability → cost → ADR

✅ done — the planning office signed off

🗺️ संपूर्ण चित्र — एक आकृती, पूर्ण कोर्स

संपूर्ण कोर्स एका कॅनव्हासवर. 4K आवृत्तीसाठी क्लिक करा.

📋 भाग 1 — पद्धत (धडे 1–4)

एक git branch = एक कल्पना; branch 04 मध्ये धडे 01–04 आहेत. प्रत्येक आकडा design/ मधून येतो — cloud account लागत नाही.

🧱 भाग 2 — बांधणीचे ठोकळे (धडे 5–12)

जवळजवळ प्रत्येक रचना ज्या तुकड्यांपासून बनते ते — प्रत्येकासोबत तो कधी लागतो हे सांगणारा एक आकडा.

🛡️ भाग 3 — प्रत्येक box मध्ये (धडे 13–16)

आकृतीतील प्रत्येक box ला विचारायचे प्रश्न, फक्त एकालाच नव्हे.

✏️ भाग 4 — सोडवलेल्या रचना (धडे 17–18)

पद्धत, सुरुवातीपासून शेवटपर्यंत सहा वेळा वापरलेली — आकडे आकार ठरवतात.

📐 धड्यांच्या आकृत्या — क्रमांक अनुसरा

प्रत्येक धडा एका क्रमांकित बॉक्स-आणि-बाण आकृतीत — स्वतंत्र पानावरही.