🏎️ शाळेच्या पद्धतीने performance engineering शिका

# the 60-second wow — one program, measured and made fast:
git clone https://github.com/BaluRaut/learn-performance-school.git && cd learn-performance-school
python3 perf/demo.py               # 12 lessons: stopwatches, hot spots, N+1, the GIL, queues, query plans, budgets
python3 perf/test_perf.py          # 12 checks across the lessons

तुम्हाला काय दिसायला हवे (छाटलेले — यश असे दिसते):

═══ latency ═══
── 1000 requests timed by the finish-line stopwatches (ms per request = latency)
   mean 51.3 · p50 40 · p95 59 · p99 260 · max 1364
   37 of 1000 took 100 ms or more · 7 took 800 ms or more — the tail
   the mean (51.3) describes nobody: most finish near 40, a few take far longer
── throughput = finishers per second: one lane at 40 ms a lap → 25/s · four lanes → 100/s · each lap is still 40 ms
   a page that waits for   1 call(s): 1.0% of page loads meet at least one call slower than its p99
   a page that waits for  10 call(s): 9.6% of page loads meet at least one call slower than its p99
   a page that waits for 100 call(s): 63.4% of page loads meet at least one call slower than its p99
   report percentiles, not the mean — and remember the p99 of each call becomes common in a big fan-out

═══ bench ═══
── Katrina times version A 20 times (simulated stopwatch): first three runs [42.1, 31.7, 21.6] ms — cold, warming up
   one run says 42.1 ms · the mean of all 20 says 15.5 ms · after 3 warm-up runs, the median of 17 says 12.4 ms
   spread of the 17: min 11.2 · p25 11.7 · median 12.4 · p75 12.7 · max 19.9 (one spike: another program woke up)
── A vs B1 (a small tweak): medians 12.4 vs 12.0 ms · middle halves 11.7–12.7 vs 11.3–12.3 → no clear difference: the change is smaller than the noise
── A vs B2 (a real fix): medians 12.4 vs 8.1 ms · middle halves 11.7–12.7 vs 7.8–8.4 → B is faster (1.53x)
   warm up, repeat, report the median and the spread; claim a win only when it is bigger than the noise

═══ profile ═══
── Dipika profiles the results program with cProfile: 300 runners, 2000 laps (we read CALL COUNTS, not times)
   find_runner 2000× · best_of 4× · results_slow 1×
   find_runner walks the runner list every time: 297,098 comparisons for 2000 lookups — the hot spot
── fix: build a dict by bib once →   best_of 4× · results_fast 1×
   find_runner comparisons now 0 · same results: True · best 100 m per house (ms) {'blue': 11000, 'green': 11006, 'red': 11006, 'yellow': 11005}
   measure first: the slow part is rarely where you guessed, and one hot function is usually most of it

═══ complexity ═══
── any duplicate bib numbers? three ways, counting comparisons (no duplicates, so each does its full work)
       n   every pair O(n²)   sort + neighbours O(n log n)   a set O(n)
     250             31,125                          1,921          250
     500            124,750                          4,355          500
    1000            499,500                          9,724        1,000
    2000          1,999,000                         21,436        2,000
   double n: every pair does 4× the work · sorting a little over 2× · the set exactly 2×
   big-O is how the work GROWS; at n = 2000 the pairwise way already does 93× the comparisons of sorting

═══ memory ═══
── the sum of squares of 0..99,999: a list holds every result, a generator makes one at a time
   same answer: True (333,328,333,350,000)
   results held at once: list 100,000 · generator 1
   tracemalloc peak: the list needs over 100× the generator's memory: True
   stream data you only walk through once; keep a list when you need it again, in any order, or its length

═══ caching ═══
── fib(25) = 75025: plain recursion does the work 242,785 times · memoized (lru_cache) 26 times (23 cache hits)
── parents look up results: 2000 requests over 500 runners, a few asked for far more often · a hit costs 1 ms, a miss 50 ms
   LRU cache of  10 → hit ratio 15.4% · average 42.5 ms per request
   LRU cache of  50 → hit ratio 33.0% · average 33.9 ms per request
   LRU cache of 100 → hit ratio 44.9% · average 28.0 ms per request
   LRU cache of 250 → hit ratio 67.7% · average 16.8 ms per request
   a cache is only as good as its hit ratio — and every cached answer can be stale until it is refreshed

═══ io ═══
── best lap for each of 50 runners, from SQLite (sqlite3's trace callback counts the queries)
   N+1: one query for the runners, then one per runner → 51 queries
   batched with IN (...)                            → 2 queries
   one JOIN + GROUP BY                              → 1 query · same answer all three ways: True
   at 1 ms round trip to a database server: 51 ms vs 2 ms vs 1 ms of waiting (here SQLite is in-process)
── writing 10,000 result lines (210,000 bytes) unbuffered     → 10,000 raw writes
── writing 10,000 result lines (210,000 bytes) buffered (8 KB) → 26 raw writes
   every trip costs a fixed toll: carry many things per trip

═══ concurrency ═══
── 8 tasks · I/O-bound: 5 ms of Python + 100 ms waiting for a reply · CPU-bound: 100 ms of Python (a teaching model)
                                             I/O-bound   CPU-bound
   one after another                           840 ms      800 ms
   8 threads, one baton (the GIL)              140 ms      800 ms
   asyncio, 1 thread, 8 tasks                  140 ms      800 ms
   4 processes on 4 cores (+50 ms start)       260 ms      250 ms
   8 threads, free-threaded build, 4 cores     110 ms      200 ms
   waiting needs no baton: threads and asyncio overlap the waits · Python work needs the baton: processes, or a free-threaded build
   (the GIL is released while waiting on I/O and inside many C extensions; free-threaded CPython exists since 3.13)

═══ queueing ═══
── one water table, 10 ms to serve a runner · M/M/1 approximation: time at the table = 10 ms ÷ (1 − utilisation)
   utilisation 50% → formula     20 ms · simulated (20,000 runners)     20 ms
   utilisation 80% → formula     50 ms · simulated (20,000 runners)     49 ms
   utilisation 90% → formula    100 ms · simulated (20,000 runners)    104 ms
   utilisation 95% → formula    200 ms · simulated (20,000 runners)    186 ms
   utilisation 99% → formula   1000 ms · simulated (20,000 runners)    519 ms
   near 100% busy, the wait explodes (and a short simulation has not caught up with it yet)
── Little's law L = λW: 80 runners/s × 50 ms = 4 runners at the table on average
── Amdahl's law: 90% of the relay can be shared out, 10% is one runner's leg that nobody can help with
   2 runners → 1.82× · 4 runners → 3.08× · 8 runners → 4.71× · 16 runners → 6.40× · endless runners → 10.00×
   keep utilisation well below 100% for low latency · speed up the serial part, it caps everything

═══ database ═══
── 10,000 laps, 500 runners · EXPLAIN QUERY PLAN for: SELECT lap_ms FROM laps WHERE runner_id = ?
   no index            → SCAN laps   (reads all 10,000 rows)
   index on runner_id  → SEARCH laps USING INDEX idx_laps_runner (runner_id=?)   (jumps to runner 42's 20 rows)
   index on (runner_id, lap_ms) → SEARCH laps USING COVERING INDEX idx_laps_runner_ms (runner_id=?)
   covering: every column the query needs is in the index, so the table itself is never read
   runners WHERE name = 'runner042'         → SEARCH runners USING COVERING INDEX idx_runners_name (name=?)
   runners WHERE lower(name) = 'runner042'  → SCAN runners
   runners WHERE name LIKE '%042'           → SCAN runners
   a function on the column or a leading % hides the index; read the plan before and after every change

═══ web ═══
── the sports-day results page on a phone: 150 ms round trip, 500 KB/s (a teaching model of one page load)
   as built: 30 KB HTML, 80 KB CSS, 900 KB JS, 1200 KB hero image   LCP  5020 ms (poor) ·   2210 KB
   + compress text (gzip/Brotli, text to about 25%)                 LCP  3505 ms (needs improvement) · 1452.5 KB
   + defer the script (no longer blocks the first paint)            LCP  3055 ms (needs improvement) · 1452.5 KB
   + resize the hero and serve AVIF (1200 → 200 KB)                 LCP  1055 ms (good) ·  452.5 KB
   repeat visit: files cached, the HTML revalidated (304)           LCP   450 ms (good) ·      0 KB
   (hashed files: Cache-Control: max-age=31536000, immutable · the HTML: Cache-Control: no-cache)
── CLS: an image with no width/height pushes the text down a quarter of the screen → 0.1875 (needs improvement) · with width/height set → 0.0 (good)
── INP: a tap runs 300 ms of script in one piece → 316 ms (needs improvement) · in 50 ms chunks that yield → 66 ms (good)
   Core Web Vitals (75th percentile of real visits): LCP ≤ 2.5 s · INP ≤ 200 ms (replaced FID in March 2024) · CLS ≤ 0.1

═══ budgets ═══
── a load test (simulated, open model): one server, 10 ms per request, capacity about 100 requests/s
    50 req/s → p50    14 ms · p95    60 ms · p99    98 ms
    70 req/s → p50    25 ms · p95   109 ms · p99   160 ms
    90 req/s → p50    64 ms · p95   280 ms · p99   411 ms
   110 req/s → p50  1716 ms · p95  3122 ms · p99  3187 ms
   the knee: past about 80% busy, p95 climbs fast · above capacity the queue only grows
── budget check, before the fixes: SQL queries per page 51 / 5 ❌ · LCP ms (simulated) 5020 / 2500 ❌ · page weight KB 2210 / 500 ❌ · p95 ms at 70 req/s 109 / 100 ❌ → FAIL (the CI job exits 1)
── budget check, after the fixes : SQL queries per page 1 / 5 ✅ · LCP ms (simulated) 1055 / 2500 ✅ · page weight KB 452.5 / 500 ✅ · p95 ms at 70 req/s 23 / 100 ✅ → PASS
   (after: the profiled fix is assumed to cut the service time from 10 ms to 5 ms — an input of the model)
   budget counts (queries, bytes, calls) in CI — they do not flake like timings; time things on a quiet, fixed machine
── the whole picture: measure → profile → fix the algorithm → memory, caching, I/O, concurrency → queues → queries → the page → budgets

✅ done — every runner timed, the relay is faster

🗺️ संपूर्ण चित्र — एक आकृती, पूर्ण कोर्स

संपूर्ण कोर्स एका कॅनव्हासवर. 4K आवृत्तीसाठी क्लिक करा.

⏱️ भाग 1 — मोजमाप (धडे 1–4)

एक git branch = एक कल्पना; branch 04 मध्ये धडे 01–04 आहेत. perf/ चा प्रत्येक run सारखाच असतो — खर्च simulated आणि seeded आहेत, आणि lab घड्याळाच्या वेळा नव्हे तर counts वाचते.

🔧 भाग 2 — नेहमीचे उपाय (धडे 5–8)

पुन्हा पुन्हा लागणारे उपाय: कमी धरून ठेवा, दोनदा गणना करू नका, कमी फेऱ्या करा, वाट पाहणे एकमेकांवर चढवा (overlap).

🏟️ भाग 3 — संपूर्ण system (धडे 9–12)

तुमच्या code भोवतीची service: queues, database, browser, आणि load tests व budgets वापरून ती वेगवान ठेवणे.

📐 धड्यांच्या आकृत्या — क्रमांक अनुसरा

प्रत्येक धडा एका क्रमांकित बॉक्स-आणि-बाण आकृतीत — स्वतंत्र पानावरही.