🖼️ See the drawing + lab 🏠 Course home 🌿 Branch on GitHub ✏️ View source

🖼️ आकृती आणि labThe drawing + lab पूर्ण पानावर उघडा ↗Open full page ↗

📌 धडा 09 — Database चे caching: कार्यालयासमोरचा सूचना फलक

📍 तुम्ही इथे आहात: 13 पैकी धडा 09 · मागील: lesson-08-serverless-scaling · पुढील: lesson-10-replicas-and-pools

📦 या ब्रँचमध्ये काय आहे

धडे 01–08, आणि database स्तरासाठीचे पहिले साधन: त्याच्यासमोर एक cache (Redis किंवा Valkey साठी ElastiCache). तुम्ही cache-aside, TTLs, write झाल्यावर काय करायचे, आणि लोकप्रिय key expire झाल्यावर stampede कसा थांबवायचा हे शिकता. scale/demo.py मधले cache() यातले प्रत्येक दाखवते.

🧒 5 वर्षांच्या मुलाला समजावल्यासारखे

प्रत्येक पालक कार्यालयाला एकच प्रश्न विचारतो: "3A वर्गाचे निकाल काय आहेत?" कार्यालयातला कारकून मागच्या खोलीतल्या मोठ्या नोंदवहीकडे जातो, 3A शोधतो, परत येतो आणि उत्तर देतो. पुन्हा. आणि पुन्हा. एका मिनिटात 6,000 वेळा.

म्हणून दीपिका कार्यालयाबाहेर एक सूचना फलक 📌 लावते. पहिल्यांदा कोणी विचारले की, कारकून नोंदवही एकदाच वाचतो आणि उत्तर फलकावर एका चिठ्ठीसह लावतो: "30 सेकंद चालेल". बाकी सगळे फलक वाचतात.

एके दिवशी 500 पालक फलकासमोर उभे असतानाच चिठ्ठीची मुदत संपते. सगळे 500 एकाच क्षणी कार्यालयात घुसतात, आणि कारकून पुरा पडत नाही. यालाच stampede म्हणतात.

पुढच्या वेळी, चिठ्ठीची मुदत संपलेली पाहणारा पहिला पालक म्हणतो: "मी आणतो — बाकीच्यांनी एक क्षण थांबा." नोंदवहीकडे एकच फेरी, आणि मग 500 आनंदी पालक. यालाच single-flight म्हणतात.

🗺️ आकृती

flowchart LR
    app["🍳 API"] -->|"1 GET results:3A"| rc["📌 Redis / Valkey<br/>ElastiCache"]
    rc -->|"hit"| app
    app -->|"2 miss: SELECT"| db["📒 database"]
    app -->|"3 SET results:3A EX 30"| rc
    w["✏️ teacher saves a grade"] -->|"UPDATE, then DEL results:3A"| db
    st["🐘 500 waiting on one expired key<br/>single-flight off → 500 DB reads<br/>on → 1"]

🗺️ रेखाटलेली आवृत्ती + एक lab: https://school-edh.pages.dev/scaling/lesson-diagrams.html#l09

❓ काय

Cache — database ने आधी दिलेली उत्तरे ठेवणारी जलद साठवण (memory). Amazon ElastiCache तुमच्यासाठी Redis OSS, Valkey (Redis चा open-source fork) किंवा Memcached चालवते; serverless पर्यायही आहे.
Cache-aside (lazy loading) — काम app करते:
1. cache वाचा; hit झाला तर काम संपले;
2. miss झाला तर database वाचा;
3. उत्तर cache मध्ये TTL सह साठवा. Write झाला की: database update करा, मग ती key delete करा (पुढचा read ती पुन्हा भरतो).
Write-through — प्रत्येक write एकाच वेळी database आणि cache दोन्हीकडे जातो. Cache नेहमी ताजा राहतो, पण प्रत्येक write ची किंमत दुप्पट होते आणि कोणी न वाचणारा data cache भरतो.
TTL — key किती वेळ जगते. जास्त TTL म्हणजे कमी database reads आणि जुना data. Write वर delete करत असलात तरी TTL नेहमी ठेवा, म्हणजे चुकलेला delete कालांतराने आपोआप सुधारतो.
Hit ratio — hits ÷ reads. Database ला फक्त misses दिसतात.
Cache stampede (thundering herd) — लोकप्रिय key expire होते आणि अनेक callers एकदम miss करतात; ते सगळे database ला query करतात.
Stampede थांबवणे:
- single-flight / lock — पहिला caller छोटा lock घेतो (SET key:lock NX EX 5) आणि key पुन्हा भरतो; बाकीचे एक क्षण थांबून नवीन value वाचतात;
- jittered TTLs — प्रत्येक TTL मध्ये थोडी random भर घाला, म्हणजे अनेक keys एकाच सेकंदाला expire होत नाहीत;
- लवकर refresh — hot key expire होण्याच्या थोडे आधीच पुन्हा भरा.

🤔 का

कारण database हा scale करायला सर्वात कठीण स्तर आहे — एका मिनिटात 20 प्रती जोडता येत नाहीत — आणि निकालाच्या दिवशी त्याचे बहुतेक reads एकच प्रश्न असतात. Cache ती उत्तरे memory मधून एका millisecond पेक्षा खूप कमी वेळात देतो, आणि database ला प्रत्येक key साठी प्रत्येक TTL मध्ये एकच read दिसतो.

🔧 कसे (या repo मध्ये)

scale/sim.py मधले CacheAside(ttl, single_flight=False) प्रत्येक key साठी एक timestamp ठेवते. entry ttl पेक्षा लहान वयाची असेपर्यंत read(key, t, concurrent=1) एक hit मोजते; नाहीतर ते database reads मोजते — single-flight बंद असेल तर concurrent इतके (थांबलेला प्रत्येक caller database कडे जातो), चालू असेल तर एक — आणि key refresh करते. cache() 60 सेकंद दर सेकंदाला 100 वेळा results:3A वाचते, आणि मग 500 थांबलेल्या callers असताना key expire करते.

🧪 करून पाहा

python3 scale/demo.py cache
python3 - <<'EOF'
import sys; sys.path.insert(0, "scale"); from sim import CacheAside
for ttl in (1, 5, 30, 300):
    c = CacheAside(ttl=ttl)
    for t in range(60):
        for _ in range(100): c.read("results:3A", t)
    print(f"TTL {ttl:>3} s → database reads {c.db_reads:>3} · hit ratio {c.hits / 6000:.1%}")
for waiting in (50, 500, 2000):
    off, on = CacheAside(30), CacheAside(30, single_flight=True)
    off.read("results:3A", 0, concurrent=waiting); on.read("results:3A", 0, concurrent=waiting)
    print(f"{waiting:>4} waiting on an expired key → DB reads {off.db_reads:>4} (single-flight off), {on.db_reads} (on)")
EOF
python3 scale/test_scale.py

✅ तपासा — तुम्हाला काय दिसायला हवे

cache हे छापते:

── 6,000 reads of results:3A in a minute, cache TTL 30 s → database reads: 2, hits: 5,998
   the key expires while 500 parents are waiting — single-flight off → database reads: 500
   the key expires while 500 parents are waiting — single-flight on  → database reads: 1

तुमचा snippet हे छापतो:

TTL   1 s → database reads  60 · hit ratio 99.0%
TTL   5 s → database reads  12 · hit ratio 99.8%
TTL  30 s → database reads   2 · hit ratio 100.0%
TTL 300 s → database reads   1 · hit ratio 100.0%
  50 waiting on an expired key → DB reads   50 (single-flight off), 1 (on)
 500 waiting on an expired key → DB reads  500 (single-flight off), 1 (on)
2000 waiting on an expired key → DB reads 2000 (single-flight off), 1 (on)

(100.0% म्हणजे 6,000 पैकी 5,998, पूर्णांकित.) Tests मध्ये ✅ L09 single-flight stops a stampede आहे.

🏁 तुम्ही आत्ताच काय सिद्ध केले

1 सेकंदाचा TTL सुद्धा 6,000 database reads चे 60 करतो. पण expire होण्याचा क्षण धोकादायक असतो: single-flight नसेल तर database ला प्रत्येक थांबलेल्या caller मागे एक read मिळतो — एकदम 2,000. Single-flight असेल तर एक — गर्दी कितीही मोठी असो.

⚠️ नेहमीच्या चुका

TTL नाही — एक delete चुकला की data कायमचा शिळा
आधी cache आणि मग database update करणे अशा चुकीच्या क्रमाने, किंवा delete करण्याऐवजी update करणे — दोन writers मिळून cache मध्ये जुनी value ठेवू शकतात
प्रत्येक user चा data shared key खाली cache करणे (results:katrina ऐवजी results)
प्रत्येक key ला सारखाच TTL, आणि deploy मुळे सगळ्या एकाच क्षणी set — त्या सगळ्या एकत्र expire होतात
cache लाच database समजणे: cache रिकामा होऊन restart झाला तरी database ला misses झेलून टिकता आले पाहिजे (सर्वात hot keys आधी warm करा)
app पेक्षा वेगळ्या Availability Zone किंवा Region मधला cache, जो प्रत्येक hit मध्ये network वेळ वाढवतो

🏭 प्रत्यक्ष वापरात

खऱ्या account वर — एक serverless Valkey cache:

aws elasticache create-serverless-cache --serverless-cache-name school-cache --engine valkey \
    --subnet-ids subnet-0aaa1111bbbb2222c subnet-0ddd3333eeee4444f \
    --security-group-ids sg-0123456789abcdef0
aws elasticache describe-serverless-caches --serverless-cache-name school-cache \
    --query 'ServerlessCaches[0].Endpoint'

Jittered TTL आणि lock सह cache-aside (Python, redis client — Valkey तोच protocol बोलतो):

import json, random, time, redis
r = redis.Redis(host="school-cache-abc123.serverless.aps1.cache.amazonaws.com", port=6379, ssl=True)

def results_for(cls):
    key = f"results:{cls}"
    for _ in range(50):                                   # wait up to ~2.5 s for another refill
        hit = r.get(key)
        if hit is not None:
            return json.loads(hit)
        if r.set(f"{key}:lock", "1", nx=True, ex=5):       # single-flight: only one caller refills
            try:
                rows = db_query("SELECT pupil, grade FROM results WHERE class = %s", cls)
                r.set(key, json.dumps(rows), ex=30 + random.randint(0, 5))   # jittered TTL
                return rows
            finally:
                r.delete(f"{key}:lock")
        time.sleep(0.05)
    return db_query("SELECT pupil, grade FROM results WHERE class = %s", cls)  # give up waiting

def save_grade(pupil, cls, grade):
    db_execute("UPDATE results SET grade = %s WHERE pupil = %s", grade, pupil)
    r.delete(f"results:{cls}")                            # on a write: delete the key

🏭 प्रत्यक्ष वापरात हे का महत्त्वाचे: cache चा hit rate आणि database चे read IOPS एकत्र पाहा. Hit rate घसरला की तो एका सेकंदाने database वर दिसतो.

⏭️ पुढे

सूचना फलक पुन्हा पुन्हा येणारे प्रश्न हाताळतो. बाकीचे अजूनही एकाच database कडे जातात. पुढे: वाचण्यासाठी नोंदवहीच्या प्रती, आणि connections वाटून देणारा proxy.

git checkout lesson-10-replicas-and-pools

📌 Lesson 09 — Caching the database: the notice board in front of the office

📍 You are here: Lesson 09 of 13 · Previous: lesson-08-serverless-scaling · Next: lesson-10-replicas-and-pools

📦 What's in this branch

Lessons 01–08, plus the first tool for the database tier: a cache in front of it (ElastiCache for Redis or Valkey). You learn cache-aside, TTLs, what to do on a write, and how to stop a stampede when a popular key expires. cache() in scale/demo.py shows each one.

🧒 Explain like I'm 5

Every parent asks the office the same question: "What are the results for class 3A?" The office clerk walks to the big register in the back room, finds 3A, walks back and answers. Again. And again. 6,000 times in a minute.

So Dipika puts up a notice board 📌 outside the office. The first time someone asks, the clerk reads the register once and pins the answer on the board with a note: "good for 30 seconds". Everyone else reads the board.

One day the note runs out while 500 parents are standing at the board. All 500 rush into the office at the same moment, and the clerk cannot keep up. That is a stampede.

Next time, the first parent who finds the note expired says: "I will fetch it — everyone else, wait one moment." One trip to the register, then 500 happy parents. That is single-flight.

🗺️ Diagram

flowchart LR
    app["🍳 API"] -->|"1 GET results:3A"| rc["📌 Redis / Valkey<br/>ElastiCache"]
    rc -->|"hit"| app
    app -->|"2 miss: SELECT"| db["📒 database"]
    app -->|"3 SET results:3A EX 30"| rc
    w["✏️ teacher saves a grade"] -->|"UPDATE, then DEL results:3A"| db
    st["🐘 500 waiting on one expired key<br/>single-flight off → 500 DB reads<br/>on → 1"]

🗺️ Drawn version + a lab: https://school-edh.pages.dev/scaling/lesson-diagrams.html#l09

❓ What

Cache — a fast store (memory) that keeps answers the database already gave. Amazon ElastiCache runs Redis OSS, Valkey (an open-source fork of Redis) or Memcached for you; there is also a serverless option.
Cache-aside (lazy loading) — the app does the work:
1. read the cache; on a hit, done;
2. on a miss, read the database;
3. store the answer in the cache with a TTL. On a write: update the database, then delete that key (the next read refills it).
Write-through — every write goes to the database and the cache at once. The cache is always fresh, but every write costs twice and data nobody reads fills the cache.
TTL — how long a key lives. A longer TTL means fewer database reads and older data. Always set one, even with deletes on write, so a missed delete fixes itself in time.
Hit ratio — hits ÷ reads. The database sees the misses only.
Cache stampede (thundering herd) — a popular key expires and many callers miss at once; all of them query the database.
Stopping a stampede:
- single-flight / a lock — the first caller takes a short lock (SET key:lock NX EX 5) and refills; the others wait a moment and read the new value;
- jittered TTLs — add a small random amount to each TTL so many keys do not expire in the same second;
- refresh early — refill a hot key a little before it expires.

🤔 Why

Because a database is the hardest tier to scale — you cannot add 20 copies in a minute — and most of its reads on results day are the same question. A cache answers those from memory in well under a millisecond, and the database sees one read per key per TTL.

🔧 How (in this repo)

CacheAside(ttl, single_flight=False) in scale/sim.py keeps a timestamp per key. read(key, t, concurrent=1) counts a hit while the entry is younger than ttl; otherwise it counts database reads — concurrent of them with single-flight off (every waiting caller goes to the database), one with it on — and refreshes the key. cache() reads results:3A 100 times a second for 60 seconds, then expires the key under 500 waiting callers.

🧪 Try it

python3 scale/demo.py cache
python3 - <<'EOF'
import sys; sys.path.insert(0, "scale"); from sim import CacheAside
for ttl in (1, 5, 30, 300):
    c = CacheAside(ttl=ttl)
    for t in range(60):
        for _ in range(100): c.read("results:3A", t)
    print(f"TTL {ttl:>3} s → database reads {c.db_reads:>3} · hit ratio {c.hits / 6000:.1%}")
for waiting in (50, 500, 2000):
    off, on = CacheAside(30), CacheAside(30, single_flight=True)
    off.read("results:3A", 0, concurrent=waiting); on.read("results:3A", 0, concurrent=waiting)
    print(f"{waiting:>4} waiting on an expired key → DB reads {off.db_reads:>4} (single-flight off), {on.db_reads} (on)")
EOF
python3 scale/test_scale.py

✅ Verify — what you should see

cache prints:

── 6,000 reads of results:3A in a minute, cache TTL 30 s → database reads: 2, hits: 5,998
   the key expires while 500 parents are waiting — single-flight off → database reads: 500
   the key expires while 500 parents are waiting — single-flight on  → database reads: 1

Your snippet prints:

TTL   1 s → database reads  60 · hit ratio 99.0%
TTL   5 s → database reads  12 · hit ratio 99.8%
TTL  30 s → database reads   2 · hit ratio 100.0%
TTL 300 s → database reads   1 · hit ratio 100.0%
  50 waiting on an expired key → DB reads   50 (single-flight off), 1 (on)
 500 waiting on an expired key → DB reads  500 (single-flight off), 1 (on)
2000 waiting on an expired key → DB reads 2000 (single-flight off), 1 (on)

(100.0% is 5,998 of 6,000, rounded.) The tests include ✅ L09 single-flight stops a stampede.

🏁 What you just proved

Even a 1-second TTL turns 6,000 database reads into 60. But the moment of expiry is dangerous: without single-flight, the database gets one read per waiting caller — 2,000 at once. With it, one — however big the crowd.

⚠️ Common mistakes

no TTL — a missed delete means stale data forever
updating the cache and then the database in the wrong order, or updating instead of deleting — two writers can leave the older value in the cache
caching per-user data under a shared key (results instead of results:katrina)
every key with the same TTL, set at the same moment by a deploy — they all expire together
treating the cache as the database: when the cache restarts empty, the database must still survive the misses (warm the hottest keys first)
a cache in a different Availability Zone or Region from the app, adding network time to every hit

🏭 In production

On a real account — a serverless Valkey cache:

aws elasticache create-serverless-cache --serverless-cache-name school-cache --engine valkey \
    --subnet-ids subnet-0aaa1111bbbb2222c subnet-0ddd3333eeee4444f \
    --security-group-ids sg-0123456789abcdef0
aws elasticache describe-serverless-caches --serverless-cache-name school-cache \
    --query 'ServerlessCaches[0].Endpoint'

Cache-aside with a jittered TTL and a lock (Python, redis client — Valkey speaks the same protocol):

import json, random, time, redis
r = redis.Redis(host="school-cache-abc123.serverless.aps1.cache.amazonaws.com", port=6379, ssl=True)

def results_for(cls):
    key = f"results:{cls}"
    for _ in range(50):                                   # wait up to ~2.5 s for another refill
        hit = r.get(key)
        if hit is not None:
            return json.loads(hit)
        if r.set(f"{key}:lock", "1", nx=True, ex=5):       # single-flight: only one caller refills
            try:
                rows = db_query("SELECT pupil, grade FROM results WHERE class = %s", cls)
                r.set(key, json.dumps(rows), ex=30 + random.randint(0, 5))   # jittered TTL
                return rows
            finally:
                r.delete(f"{key}:lock")
        time.sleep(0.05)
    return db_query("SELECT pupil, grade FROM results WHERE class = %s", cls)  # give up waiting

def save_grade(pupil, cls, grade):
    db_execute("UPDATE results SET grade = %s WHERE pupil = %s", grade, pupil)
    r.delete(f"results:{cls}")                            # on a write: delete the key

🏭 Why this matters in production: watch the cache's hit rate and the database's read IOPS together. A drop in hit rate shows up on the database a second later.

⏭️ Next

The notice board handles the repeated questions. The rest still go to one database. Next: copies of the register for reading, and a proxy that shares connections.

git checkout lesson-10-replicas-and-pools

← Previousserverless scaling Next →replicas and pools

This page is the lesson's README from the lesson-09-caching-the-database branch, shown here so the whole School stays on one site. Code files open on GitHub at the same branch.