🖼️ See the drawing + lab 🏠 Course home 🌿 Branch on GitHub ✏️ View source

🖼️ आकृती आणि labThe drawing + lab पूर्ण पानावर उघडा ↗Open full page ↗

🪄 धडा 08 — Serverless scaling: प्रत्येक पालकासाठी एक खिडकी

📍 तुम्ही इथे आहात: 13 पैकी धडा 08 · मागे: lesson-07-kubernetes-scaling · पुढे: lesson-09-caching-the-database

📦 या ब्रँचमध्ये काय आहे

धडे 01–07, आणि त्याशिवाय AWS Lambda: platform चालू असलेल्या प्रत्येक request साठी एक execution environment सुरू करतो. तुम्ही शिकता concurrency ≈ requests per second × average duration (स्थिर traffic साठी — अचानक गर्दीला headroom लागतो), cold starts, scaling rate, आणि दोन कळी — reserved आणि provisioned concurrency. scale/demo.py मधले serverless() निकालाची गर्दी चालवून दाखवते.

🧒 5 वर्षांच्या मुलाला समजावल्यासारखे

एका जादूच्या जत्रेची कल्पना करा: पालक आला की तिच्यासाठी एक नवी खिडकी उगवते. ती गेली की दुसरा पालक आला तर म्हणून खिडकी थोडा वेळ थांबते.

एका वेळी किती खिडक्या उभ्या असतात? दर सेकंदाला 2,000 पालक आले आणि प्रत्येक जण सरासरी 0.2 सेकंद थांबला, तर कोणत्याही क्षणी सुमारे 400 खिडक्या व्यस्त असतात. हा स्थिर आकडा. पालक घोळक्याने आले, किंवा काही जण खूप जास्त वेळ थांबले, तर थोडा वेळ जास्त खिडक्या व्यस्त असतात — म्हणून दीपिका 400 पेक्षा जास्तीचे नियोजन करते.

पहिल्यांदाच उगवणाऱ्या खिडकीला तयार व्हायला थोडा वेळ लागतो — हा cold start. आधीच उभी असलेली खिडकी warm असते आणि लगेच सेवा देते.

दीपिका दोन नियम जोडते:

"हा stall कधीच 300 पेक्षा जास्त खिडक्या वापरणार नाही" — जत्रेच्या बाकीच्या भागालाही खिडक्या लागतात. हे reserved concurrency. मर्यादेपलीकडच्या पालकांना "थोड्या वेळाने परत या" असे सांगितले जाते (throttled).
"निकाल लागण्याआधीच 400 खिडक्या तयार ठेवा" — set-up साठी कुणीच थांबत नाही. हे provisioned concurrency.

🗺️ आकृती

flowchart LR
    r["👪 2,000 req/s"] --> f["🪄 Lambda function<br/>200 ms each"]
    f --> c["🧮 concurrency ≈<br/>2,000 × 0.2 s = 400<br/>steady traffic; plan headroom"]
    c --> res["🔒 reserved 300<br/>runs 300 · throttles 500 req/s (429)"]
    c --> pro["🔥 provisioned 400<br/>pre-warmed · 0 cold starts"]
    q["🏛️ account: 1,000 per Region<br/>by default, raisable"]

🗺️ काढलेली आकृती + एक lab: https://school-edh.pages.dev/scaling/lesson-diagrams.html#l08

❓ काय

Execution environment — तुमच्या function ची एक चालू प्रत. ती एका वेळी एकच request हाताळते; एकाच वेळी जास्त requests म्हणजे जास्त environments.
Concurrency — एकाच वेळी वापरात असलेली environments. स्थिर traffic साठी ती सुमारे requests per second × सेकंदांतील सरासरी duration असते (धडा 02 मधला Little's law): 2,000 req/s × 0.2 s ≈ 400. हा एक अंदाज आहे. अचानक गर्दी (एकाच 100 ms मध्ये अनेक requests) आणि असमान durations (काही हळू calls) खरा peak त्याच्या वर ढकलतात, म्हणून या आकड्याच्या वर headroom ठेवून नियोजन करा.
Scaling rate — Lambda एकदम कोणत्याही आकारावर उडी मारत नाही. प्रत्येक function दर 10 सेकंदांना 1,000 environments पर्यंत जोडू शकते (सध्याच्या नियमासाठी Lambda docs तपासा). खूप तीव्र गर्दी वाढत असताना, quota च्या खाली असतानाही, throttle होऊ शकते.
Account quota — default नुसार एका Region मध्ये 1,000 concurrent executions, account मधली सगळी functions ते वाटून घेतात. Service Quotas मधून तुम्ही जास्त मागू शकता.
Throttling — मर्यादेपलीकडे synchronous calls ना 429 TooManyRequestsException मिळते; asynchronous events काही वेळ पुन्हा प्रयत्न (retry) केले जातात.
Cold start — नव्या environment ला पहिल्या request आधी code download करावा लागतो, runtime सुरू करावा लागतो आणि init code चालवावा लागतो. काही दहा milliseconds पासून काही सेकंदांपर्यंत, runtime, package size आणि तुमचा init काय करतो यावर अवलंबून.
Reserved concurrency — function चा maximum ठरवते आणि account quota मधला तेवढा भाग फक्त त्याच्यासाठी राखून ठेवते. ते account च्या बाकीच्या भागाचे संरक्षण करते, आणि function मागच्या database चेही संरक्षण करते (धडा 10). जादा शुल्क नाही.
Provisioned concurrency — N environments आधीच initialized आणि warm ठेवते. त्या N साठी cold start नाही. ते set असेपर्यंत तुम्ही पैसे भरता, म्हणून माहीत असलेल्या गर्दीसाठी ते schedule करा (Application Auto Scaling).
SnapStart — काही runtimes साठी, Lambda initialized environment चा snapshot घेतो आणि तिथूनच पुन्हा सुरू करतो, ज्यामुळे cold starts कमी होतात. कोणत्या runtimes साठी ते Lambda docs मध्ये तपासा.
15 मिनिटे — एक invocation जास्तीत जास्त इतका वेळ चालू शकते. जास्त लांब काम queue आणि workers कडे, किंवा Step Functions कडे जाते (धडा 12).

🤔 का

कारण Lambda बहुतेक requests साठी धडे 06 आणि 07 मधली warm-up ची समस्या काढून टाकतो: नवी environments instance ला boot व्हायला लागणाऱ्या वेळेपेक्षा खूप कमी वेळात सुरू होतात, आणि तुम्ही फक्त वापरलेल्या वेळेचेच पैसे भरता. पण तो मर्यादा दुसरीकडे हलवतो: concurrency quota, पहिल्या लाटेवर cold starts, आणि environments ची गर्दी, ज्यांपैकी प्रत्येक database connection उघडू शकते (धडा 10).

🔧 कसे (या repo मध्ये)

scale/sim.py मधले lambda_concurrency(rps, duration_ms) म्हणजे rps × duration_ms / 1000 — स्थिर traffic चा अंदाज, burst नाही आणि scaling-rate ची मर्यादा नाही. LambdaFleet(duration_ms, cold_ms, reserved, provisioned) एका वेळी एक सेकंद चालवते: त्याला ceil(concurrency) environments लागतात, त्यांपैकी जास्तीत जास्त reserved चालवते, warm असलेल्यांपलीकडच्या प्रत्येक environment ला cold start म्हणून मोजते, आणि चालवता न आलेल्या requests चा वाटा throttle करते. Warm environments पूर्ण run भर ठेवली जातात (खरा Lambda काही वेळाने रिकामी environments काढतो). cold_ms साठवले जाते पण हिशोबात वापरले जात नाही — model cold starts मोजते; त्यांचा वेळ जोडत नाही.

🧪 करून पाहा

python3 scale/demo.py serverless
python3 - <<'EOF'
import sys; sys.path.insert(0, "scale"); from sim import lambda_concurrency, LambdaFleet
print("5,000 req/s × 200 ms →", lambda_concurrency(5000, 200), "environments (default Region quota: 1,000)")
for label, f in (("no reserved limit", LambdaFleet(200, 600)), ("reserved 300", LambdaFleet(200, 600, reserved=300)),
                 ("provisioned 200", LambdaFleet(200, 600, provisioned=200)), ("provisioned 400", LambdaFleet(200, 600, provisioned=400))):
    rows = [f.second(r) for r in (100, 2000, 2000, 500)]
    print(f"{label:<18} cold starts {sum(r['cold_starts'] for r in rows):>3} · throttled {sum(r['throttled'] for r in rows):>5}")
EOF
python3 scale/test_scale.py

✅ तपासा — तुम्हाला काय दिसायला हवे

serverless हे print करते:

── Lambda: concurrency ≈ requests per second × average duration (steady traffic; bursts need more)
     100 req/s × 200 ms →    20 environments at once
    2000 req/s × 200 ms →   400 environments at once
    2000 req/s ×  50 ms →   100 environments at once
── the results spike, reserved concurrency 300, cold start ~600 ms:
   second 0:   100 req/s → needs  20, runs  20, cold starts  20, throttled     0
   second 1:  2000 req/s → needs 400, runs 300, cold starts 280, throttled   500
   second 2:  2000 req/s → needs 400, runs 300, cold starts   0, throttled   500
   second 3:   500 req/s → needs 100, runs 100, cold starts   0, throttled     0
── with 400 provisioned (pre-warmed) environments: cold starts at 2,000 req/s = 0

तुमचा snippet हे print करतो:

5,000 req/s × 200 ms → 1000.0 environments (default Region quota: 1,000)
no reserved limit  cold starts 400 · throttled     0
reserved 300       cold starts 300 · throttled  1000
provisioned 200    cold starts 200 · throttled     0
provisioned 400    cold starts   0 · throttled     0

आणि tests मध्ये ✅ L08 Lambda: reserved concurrency throttles the rest असते.

🏁 तुम्ही आत्ताच काय सिद्ध केले

Duration म्हणजे bill चा अर्धा भाग आणि concurrency चा अर्धा भाग: 200 ms ऐवजी 50 ms ला 400 ऐवजी 100 environments लागतात. 5,000 req/s × 200 ms ला हे एकच function संपूर्ण default account quota वापरेल. Reserved 300 गर्दीत 1,000 requests throttle करते; provisioned 400 प्रत्येक cold start काढून टाकते, आणि 200 अर्धे काढतात.

⚠️ नेहमीच्या चुका

Region मधली सगळी functions एकच quota वाटून घेतात हे विसरणे — एक व्यस्त function बाकीच्यांना throttle करते. महत्त्वाच्या functions साठी concurrency reserve करा
हळू database प्रत्येक call लांबवतो, त्यामुळे concurrency वाढत जाते (Little's law) आणि quota वर आदळते
reserved किंवा provisioned concurrency अगदी req/s × average duration इतकीच ठेवणे — अचानक गर्दी आणि हळू calls ना त्यावर headroom लागतो
1,000 environments प्रत्येकी एक database connection उघडतात (धडा 10 हे RDS Proxy ने दुरुस्त करतो)
प्रत्येक cold start वर जड init code (मोठ्या libraries, अनेक clients)
एका सकाळच्या गर्दीसाठी provisioned concurrency वर्षभर चालू ठेवणे
एकाच invocation मध्ये 15 मिनिटांपेक्षा लांब काम

🏭 प्रत्यक्ष वापरात

On a real account — account quota पाहा, एका function ला मर्यादा घाला, आणि निकालाच्या सकाळसाठी एक alias आधीच warm करा:

aws lambda get-account-settings --query 'AccountLimit.{total:ConcurrentExecutions,unreserved:UnreservedConcurrentExecutions}'
aws lambda put-function-concurrency --function-name results-api --reserved-concurrent-executions 300
aws lambda put-provisioned-concurrency-config --function-name results-api \
    --qualifier live --provisioned-concurrent-executions 400
aws lambda get-provisioned-concurrency-config --function-name results-api --qualifier live

Application Auto Scaling ने provisioned concurrency schedule करा (8:30 ला चालू, 12:00 ला बंद, IST):

aws application-autoscaling register-scalable-target --service-namespace lambda \
    --resource-id function:results-api:live \
    --scalable-dimension lambda:function:ProvisionedConcurrency --min-capacity 0 --max-capacity 400
aws application-autoscaling put-scheduled-action --service-namespace lambda \
    --resource-id function:results-api:live \
    --scalable-dimension lambda:function:ProvisionedConcurrency \
    --scheduled-action-name results-morning --schedule "cron(0 3 20 5 ? 2026)" \
    --scalable-target-action MinCapacity=400,MaxCapacity=400
aws application-autoscaling put-scheduled-action --service-namespace lambda \
    --resource-id function:results-api:live \
    --scalable-dimension lambda:function:ProvisionedConcurrency \
    --scheduled-action-name results-morning-over --schedule "cron(30 6 20 5 ? 2026)" \
    --scalable-target-action MinCapacity=0,MaxCapacity=0

🏭 प्रत्यक्ष वापरात हे का महत्त्वाचे: CloudWatch मध्ये function चे ConcurrentExecutions, Throttles आणि Duration (p99) पाहा. Traffic न वाढता concurrency वाढत असेल तर function मागचे काहीतरी हळू झाले आहे.

⏭️ पुढे

भाग 2 पूर्ण: API servers, pods किंवा Lambdas ने वाढू शकतो. आता त्यांपैकी प्रत्येक जण database ला विचारतो — आणि database सहज copy करता येत नाही. पहिली युक्ती: त्याच्यासमोर एक सूचना फलक.

git checkout lesson-09-caching-the-database

🪄 Lesson 08 — Serverless scaling: a counter for every parent

📍 You are here: Lesson 08 of 13 · Previous: lesson-07-kubernetes-scaling · Next: lesson-09-caching-the-database

📦 What's in this branch

Lessons 01–07, plus AWS Lambda: the platform starts an execution environment for each request in flight. You learn concurrency ≈ requests per second × average duration (for steady traffic — bursts need headroom), cold starts, the scaling rate, and the two dials — reserved and provisioned concurrency. serverless() in scale/demo.py plays the results spike.

🧒 Explain like I'm 5

Imagine a magic fair: when a parent arrives, a new counter appears for her. When she leaves, the counter waits a while in case another parent comes.

How many counters stand at once? If 2,000 parents arrive every second and each one stays 0.2 seconds on average, then about 400 counters are busy at any moment. That is the steady number. If parents arrive in clumps, or some stay much longer, more counters are busy for a while — so Dipika plans for more than 400.

A counter that appears for the first time needs a moment to set up — the cold start. A counter that is already standing is warm and serves at once.

Dipika adds two rules:

"This stall may never use more than 300 counters" — the rest of the fair needs counters too. That is reserved concurrency. Parents over the limit are told "come back in a moment" (throttled).
"Set up 400 counters before the results go up" — no one waits for set-up. That is provisioned concurrency.

🗺️ Diagram

flowchart LR
    r["👪 2,000 req/s"] --> f["🪄 Lambda function<br/>200 ms each"]
    f --> c["🧮 concurrency ≈<br/>2,000 × 0.2 s = 400<br/>steady traffic; plan headroom"]
    c --> res["🔒 reserved 300<br/>runs 300 · throttles 500 req/s (429)"]
    c --> pro["🔥 provisioned 400<br/>pre-warmed · 0 cold starts"]
    q["🏛️ account: 1,000 per Region<br/>by default, raisable"]

🗺️ Drawn version + a lab: https://school-edh.pages.dev/scaling/lesson-diagrams.html#l08

❓ What

Execution environment — one running copy of your function. It handles one request at a time; more requests at once means more environments.
Concurrency — environments in use at once. For steady traffic it is about requests per second × average duration in seconds (Little's law from lesson 02): 2,000 req/s × 0.2 s ≈ 400. It is an approximation. Bursts (many requests in the same 100 ms) and uneven durations (a few slow calls) push the real peak above it, so plan headroom above this number.
Scaling rate — Lambda does not jump to any size at once. Each function can add up to 1,000 environments every 10 seconds (check the Lambda docs for the current rule). A very sharp spike can be throttled while it grows, even below the quota.
Account quota — by default 1,000 concurrent executions per Region, shared by all functions in the account. You can ask for more through Service Quotas.
Throttling — over the limit, synchronous calls get 429 TooManyRequestsException; asynchronous events are retried for a while.
Cold start — a new environment must download the code, start the runtime and run the init code before the first request. From tens of milliseconds to seconds, depending on runtime, package size and what your init does.
Reserved concurrency — sets a function's maximum and keeps that much of the account quota for it alone. It protects the rest of the account, and protects a database behind the function (lesson 10). No extra charge.
Provisioned concurrency — keeps N environments initialized and warm ahead of time. No cold start for those N. You pay while it is set, so schedule it (Application Auto Scaling) for known spikes.
SnapStart — for some runtimes, Lambda snapshots an initialized environment and resumes from it, which cuts cold starts. Check the Lambda docs for which runtimes.
15 minutes — the most one invocation may run. Longer work goes to a queue and workers, or Step Functions (lesson 12).

🤔 Why

Because Lambda removes the warm-up problem of lessons 06 and 07 for most requests: new environments start in well under the time an instance takes to boot, and you pay only for the time you use. But it moves the limits: a concurrency quota, cold starts on the first wave, and a crowd of environments that can each open a database connection (lesson 10).

🔧 How (in this repo)

lambda_concurrency(rps, duration_ms) in scale/sim.py is rps × duration_ms / 1000 — the steady-traffic estimate, with no burst and no scaling-rate limit. LambdaFleet(duration_ms, cold_ms, reserved, provisioned) plays one second at a time: it needs ceil(concurrency) environments, runs at most reserved of them, counts every environment beyond the warm ones as a cold start, and throttles the share of requests it could not run. Warm environments are kept for the whole run (real Lambda removes idle environments after a while). cold_ms is stored but not used in the arithmetic — the model counts cold starts; it does not add their time.

🧪 Try it

python3 scale/demo.py serverless
python3 - <<'EOF'
import sys; sys.path.insert(0, "scale"); from sim import lambda_concurrency, LambdaFleet
print("5,000 req/s × 200 ms →", lambda_concurrency(5000, 200), "environments (default Region quota: 1,000)")
for label, f in (("no reserved limit", LambdaFleet(200, 600)), ("reserved 300", LambdaFleet(200, 600, reserved=300)),
                 ("provisioned 200", LambdaFleet(200, 600, provisioned=200)), ("provisioned 400", LambdaFleet(200, 600, provisioned=400))):
    rows = [f.second(r) for r in (100, 2000, 2000, 500)]
    print(f"{label:<18} cold starts {sum(r['cold_starts'] for r in rows):>3} · throttled {sum(r['throttled'] for r in rows):>5}")
EOF
python3 scale/test_scale.py

✅ Verify — what you should see

serverless prints:

── Lambda: concurrency ≈ requests per second × average duration (steady traffic; bursts need more)
     100 req/s × 200 ms →    20 environments at once
    2000 req/s × 200 ms →   400 environments at once
    2000 req/s ×  50 ms →   100 environments at once
── the results spike, reserved concurrency 300, cold start ~600 ms:
   second 0:   100 req/s → needs  20, runs  20, cold starts  20, throttled     0
   second 1:  2000 req/s → needs 400, runs 300, cold starts 280, throttled   500
   second 2:  2000 req/s → needs 400, runs 300, cold starts   0, throttled   500
   second 3:   500 req/s → needs 100, runs 100, cold starts   0, throttled     0
── with 400 provisioned (pre-warmed) environments: cold starts at 2,000 req/s = 0

Your snippet prints:

5,000 req/s × 200 ms → 1000.0 environments (default Region quota: 1,000)
no reserved limit  cold starts 400 · throttled     0
reserved 300       cold starts 300 · throttled  1000
provisioned 200    cold starts 200 · throttled     0
provisioned 400    cold starts   0 · throttled     0

and the tests include ✅ L08 Lambda: reserved concurrency throttles the rest.

🏁 What you just proved

Duration is half of the bill and half of the concurrency: 50 ms instead of 200 ms needs 100 environments instead of 400. At 5,000 req/s × 200 ms this one function would use the whole default account quota. Reserved 300 throttles 1,000 requests over the spike; provisioned 400 removes every cold start, and 200 removes half.

⚠️ Common mistakes

forgetting that all functions in the Region share one quota — one busy function throttles the others. Reserve concurrency for the important ones
a slow database making every call longer, so concurrency grows (Little's law) until it hits the quota
sizing reserved or provisioned concurrency to exactly req/s × average duration — bursts and slow calls need headroom above it
1,000 environments each opening a database connection (lesson 10 fixes this with RDS Proxy)
heavy init code (large libraries, many clients) on every cold start
provisioned concurrency left on all year for a one-morning spike
work longer than 15 minutes in one invocation

🏭 In production

On a real account — see the account quota, cap a function, and pre-warm an alias for results morning:

aws lambda get-account-settings --query 'AccountLimit.{total:ConcurrentExecutions,unreserved:UnreservedConcurrentExecutions}'
aws lambda put-function-concurrency --function-name results-api --reserved-concurrent-executions 300
aws lambda put-provisioned-concurrency-config --function-name results-api \
    --qualifier live --provisioned-concurrent-executions 400
aws lambda get-provisioned-concurrency-config --function-name results-api --qualifier live

Schedule provisioned concurrency with Application Auto Scaling (on at 8:30, off at 12:00, IST):

aws application-autoscaling register-scalable-target --service-namespace lambda \
    --resource-id function:results-api:live \
    --scalable-dimension lambda:function:ProvisionedConcurrency --min-capacity 0 --max-capacity 400
aws application-autoscaling put-scheduled-action --service-namespace lambda \
    --resource-id function:results-api:live \
    --scalable-dimension lambda:function:ProvisionedConcurrency \
    --scheduled-action-name results-morning --schedule "cron(0 3 20 5 ? 2026)" \
    --scalable-target-action MinCapacity=400,MaxCapacity=400
aws application-autoscaling put-scheduled-action --service-namespace lambda \
    --resource-id function:results-api:live \
    --scalable-dimension lambda:function:ProvisionedConcurrency \
    --scheduled-action-name results-morning-over --schedule "cron(30 6 20 5 ? 2026)" \
    --scalable-target-action MinCapacity=0,MaxCapacity=0

🏭 Why this matters in production: watch the function's ConcurrentExecutions, Throttles and Duration (p99) in CloudWatch. Concurrency rising with no rise in traffic means something behind the function got slower.

⏭️ Next

Part 2 is done: the API can grow by servers, pods or Lambdas. Now every one of them asks the database — and the database cannot simply be copied. First trick: a notice board in front of it.

git checkout lesson-09-caching-the-database

← Previouskubernetes scaling Next →caching the database

This page is the lesson's README from the lesson-08-serverless-scaling branch, shown here so the whole School stays on one site. Code files open on GitHub at the same branch.