🖼️ See the drawing + lab 🏠 Course home 🌿 Branch on GitHub ✏️ View source

🖼️ आकृती आणि labThe drawing + lab पूर्ण पानावर उघडा ↗Open full page ↗

🚦 धडा 07 — Throttling आणि usage plans: दारावरचा token bucket

📍 तुम्ही इथे आहात: 12 पैकी धडा 07 · मागे: lesson-06-auth · पुढे: lesson-08-caching

📦 या ब्रँचमध्ये काय आहे

धडे 01–06, आणि office गर्दी कशी थांबवते ते: token bucket (rate आणि burst), प्रत्येक stage साठी आणि प्रत्येक client साठी मर्यादा (API keys आणि दैनिक quota असलेले usage plans), आणि 429 Too Many Requests caller ला काय करायला सांगतो. apigw/demo.py मधले throttle() एकाच क्षणी आठ visitors पाठवते.

🧒 5 वर्षांच्या मुलाला समजावल्यासारखे

दारावर tokens ची बादली 🪣 ठेवलेली आहे. आत येण्यासाठी प्रत्येक visitor ने एक token घ्यायलाच हवा. बादलीत जास्तीत जास्त 5 tokens मावतात (हा burst). एक मदतनीस दर सेकंदाला 5 नवे tokens बादलीत टाकतो (हा rate) — पण बादलीत मावतील त्यापेक्षा जास्त कधीच नाही.

आठ visitors एकाच क्षणी येतात. पहिले पाच जण पाचही tokens घेतात आणि आत जातात. सहावा, सातवा आणि आठवा यांना बादली रिकामी दिसते: "कृपया थांबा आणि परत या" (429). एका सेकंदानंतर बादलीत पुन्हा पाच ताजे tokens असतात.

Partner शाळांना त्यांची स्वतःची छोटी बादली आणि त्यांच्या visitor pass वर दैनिक मर्यादा मिळते — दिवसाला 1,000 भेटी. 1,001 वी भेट उद्यापर्यंत नाकारली जाते.

बादली कोणालाही शिक्षा करत नाही. ती kitchen ला गर्दीत बुडण्यापासून वाचवते.

🗺️ आकृती

flowchart LR
    crowd["👥 8 requests<br/>same instant"] --> b["🪣 stage dev bucket<br/>burst 5 · refill 5/s"]
    b -->|"tokens 5→0"| ok["✅ 200 × 5"]
    b -->|"empty"| no["⏳ 429 × 3<br/>Too Many Requests"]
    key["🔑 x-api-key: key-partner-1"] --> plan["📋 usage plan partner-basic<br/>2 rps · burst 2 · 1,000/day"]
    plan -->|"over the rate"| r["429 Too Many Requests"]
    plan -->|"over the quota"| q["429 Limit Exceeded"]

🗺️ काढलेली आकृती + एक lab: https://school-edh.pages.dev/apigateway/lesson-diagrams.html#l07

❓ काय

Token bucket — API Gateway वापरतो तो algorithm: burst = बादलीचा आकार, rate = दर सेकंदाला भरले जाणारे tokens. Tokens असेपर्यंत requests चा burst चालतो; त्यानंतर requests rate इतक्याच वेगाने जातात.
Account-level limit — default नुसार दर सेकंदाला 10,000 requests आणि 5,000 चा burst, प्रत्येक account, प्रत्येक Region साठी, तुमच्या सगळ्या APIs मध्ये वाटलेला. हा soft quota आहे: Service Quotas मधून तुम्ही जास्त मागू शकता.
Stage आणि method limits — संपूर्ण stage साठी किंवा एका method साठी कमी मर्यादा ठेवा, उदा. सावकाश चालणाऱ्या POST /reports चे रक्षण करा.
Usage plans (REST APIs) — प्रत्येक client साठी throttle (rate, burst) आणि quota (दिवस, आठवडा किंवा महिन्याला किती requests), API keys ला जोडलेले. Client आपली key x-api-key header मध्ये पाठवतो.
क्रम — सगळ्यात नेमकी मर्यादा आधी लागू होते: client चा usage plan, मग method, मग stage, मग account. मर्यादा best-effort पद्धतीने लागू होतात.
429 — throttle झाल्यावर {"message": "Too Many Requests"}, quota संपल्यावर {"message": "Limit Exceeded"}. Callers नी exponential backoff आणि jitter सह retry करावे (AWS SDKs हे तुमच्यासाठी करतात).

🤔 का

कारण एकच बेलगाम client — mobile app मधला एखादा loop, partner चा batch job — संपूर्ण वाटलेली account limit भरून टाकू शकतो आणि Region मधला प्रत्येक API बंद पाडू शकतो. दारावरच्या मर्यादा kitchen ला जिवंत ठेवतात आणि क्षमता न्याय्यपणे वाटतात. Usage plans "एक partner" चे रूपांतर contract मध्ये मान्य करता येतील अशा आकड्यांमध्ये करतात: दर सेकंदाला इतके, दिवसाला इतके.

🔧 कसे (या repo मध्ये)

apigw/gateway.py मधला TokenBucket फक्त नऊ ओळींचा आहे: take(now) (now - t) * rate tokens जोडतो (जास्तीत जास्त burst) आणि शक्य असल्यास एक घेतो. प्रत्येक stage ला स्वतःची bucket मिळते (dev: 5 आणि 5; prod: defaults RATE = 10_000, BURST = 5_000). Gateway.usage_plan() प्रत्येक plan साठी bucket आणि quota बनवतो आणि API keys त्याला जोडतो. api_key=True असलेल्या routes साठी _handle() key तपासतो (अनोळखी → 403), quota (429 Limit Exceeded) आणि plan ची bucket (429 Too Many Requests) — stage bucket च्या आधी.

🧪 करून पाहा

python3 apigw/demo.py throttle
python3 - <<'EOF'
import sys; sys.path.insert(0, "apigw"); from demo import office, Clock
from gateway import TokenBucket
b = TokenBucket(rate=5, burst=5, now=0)
print("t=0.0 ", [b.take(0) for _ in range(6)])
print("t=0.2 ", [b.take(0.2) for _ in range(2)])
print("t=10  ", b.take(10), "tokens left:", b.tokens)
c = Clock(); gw, _ = office(c)
gw.usage_plan("trial", rate=100, burst=100, quota_per_day=3, keys=["key-trial"])
print([gw.handle("GET", "/partner/students/9", {"x-api-key": "key-trial"})[0] for _ in range(5)], gw.handle("GET", "/partner/students/9", {"x-api-key": "key-trial"})[2])
print("no key:", gw.handle("GET", "/partner/students/9")[0::2])
EOF

✅ तपासा — तुम्हाला काय दिसायला हवे

throttle आधी stage dev: rate 5/s, burst 5 — 8 requests in the same instant: छापते, मग 200 200 200 200 200 429 429 429, मग one second later: 200 200 200 200 200 429. Usage plan साठी: 200 200 429 429 · unknown key → 403.

तुमचा snippet t=0.0 [True, True, True, True, True, False] छापतो (पाच tokens, मग रिकामी), t=0.2 [True, False] (0.2 s × 5/s = एक नवा token), t=10 True tokens left: 4 (दहा शांत सेकंदांनंतरही बादलीत 5 पेक्षा जास्त tokens कधीच नसतात), मग trial quota 3 साठी [200, 200, 200, 429, 429] {'message': 'Limit Exceeded'}, आणि no key: (403, {'message': 'Forbidden'}).

🏁 तुम्ही आत्ताच काय सिद्ध केले

तुम्ही प्रत्येक 429 आधीच सांगू शकता: bucket मधले tokens मोजा, refill जोडा, आणि quota तपासा. पहिला क्षण burst ठरवतो; त्यानंतरचा प्रत्येक सेकंद rate ठरवतो.

⚠️ नेहमीच्या चुका

stage किंवा method limits नसणे, त्यामुळे एकच client संपूर्ण account चे 10,000 rps वापरू शकतो
429 ला लगेच घट्ट loop मध्ये retry करणे — त्याने गर्दी वाढते; थांबून, वाढत्या अंतराने retry करा
usage plan ची API key authentication आहे असे मानणे (धडा 06)
alarm नसलेला quota — partner ला सकाळी 11 वाजता कळते की ते उद्यापर्यंत blocked आहेत

🏭 प्रत्यक्ष वापरात

खऱ्या account वर — आधी stage limits, मग एका partner साठी key सह usage plan:

aws apigateway update-stage --rest-api-id abc123 --stage-name prod --patch-operations \
    op=replace,path=/*/*/throttling/rateLimit,value=500 \
    op=replace,path=/*/*/throttling/burstLimit,value=1000
aws apigateway create-usage-plan --name partner-basic \
    --throttle burstLimit=2,rateLimit=2 --quota limit=1000,period=DAY \
    --api-stages apiId=abc123,stage=prod
aws apigateway create-api-key --name partner-1 --enabled
aws apigateway create-usage-plan-key --usage-plan-id pl4n01 --key-id k3y001 --key-type API_KEY

Terraform:

resource "aws_api_gateway_usage_plan" "partner_basic" {
  name = "partner-basic"
  api_stages {
    api_id = aws_api_gateway_rest_api.school.id
    stage  = aws_api_gateway_stage.prod.stage_name
  }
  throttle_settings {
    rate_limit  = 2
    burst_limit = 2
  }
  quota_settings {
    limit  = 1000
    period = "DAY"
  }
}

🏭 Production मध्ये हे का महत्त्वाचे आहे: back end खरोखर किती सहन करू शकतो त्यापेक्षा कमी stage limits ठेवा (ते मोजा), प्रत्येक partner ला usage plan द्या, आणि 429s वर alarm लावा, म्हणजे customer तक्रार करण्याआधीच तुम्हाला मर्यादा दिसेल.

⏭️ पुढे

गर्दी नियंत्रणात आली. आता तोच प्रश्न kitchen ला त्रास न देता सोडवा: सूचना फलक — caching.

git checkout lesson-08-caching

🚦 Lesson 07 — Throttling & usage plans: a token bucket at the door

📍 You are here: Lesson 07 of 12 · Previous: lesson-06-auth · Next: lesson-08-caching

📦 What's in this branch

Lessons 01–06, plus how the office stops a crowd: the token bucket (rate and burst), limits per stage and per client (usage plans with API keys and a daily quota), and what 429 Too Many Requests asks the caller to do. throttle() in apigw/demo.py sends eight visitors at the same instant.

🧒 Explain like I'm 5

At the door stands a bucket of tokens 🪣. Every visitor must take one token to come in. The bucket holds at most 5 tokens (the burst). A helper drops 5 new tokens every second into it (the rate) — but never more than the bucket holds.

Eight visitors arrive at the same moment. The first five take the five tokens and walk in. Visitors six, seven and eight find the bucket empty: "please wait and come back" (429). One second later there are five fresh tokens again.

Partner schools get their own small bucket and a daily limit on their visitor pass — 1,000 visits a day. Visit 1,001 is refused until tomorrow.

The bucket does not punish anyone. It protects the kitchen from being flooded.

🗺️ Diagram

flowchart LR
    crowd["👥 8 requests<br/>same instant"] --> b["🪣 stage dev bucket<br/>burst 5 · refill 5/s"]
    b -->|"tokens 5→0"| ok["✅ 200 × 5"]
    b -->|"empty"| no["⏳ 429 × 3<br/>Too Many Requests"]
    key["🔑 x-api-key: key-partner-1"] --> plan["📋 usage plan partner-basic<br/>2 rps · burst 2 · 1,000/day"]
    plan -->|"over the rate"| r["429 Too Many Requests"]
    plan -->|"over the quota"| q["429 Limit Exceeded"]

🗺️ Drawn version + a lab: https://school-edh.pages.dev/apigateway/lesson-diagrams.html#l07

❓ What

Token bucket — the algorithm API Gateway uses: burst = bucket size, rate = tokens added per second. A burst of requests is fine while tokens last; after that, requests pass at the rate.
Account-level limit — by default 10,000 requests per second with a burst of 5,000, per account, per Region, shared by all your APIs. It is a soft quota: you can ask for more through Service Quotas.
Stage and method limits — set lower limits for a whole stage or one method, e.g. protect a slow POST /reports.
Usage plans (REST APIs) — per-client throttle (rate, burst) and quota (requests per day, week or month), attached to API keys. The client sends its key in the x-api-key header.
Order — the most specific limit applies first: the client's usage plan, then the method, then the stage, then the account. Limits are applied on a best-effort basis.
429 — {"message": "Too Many Requests"} when throttled, {"message": "Limit Exceeded"} when a quota is used up. Callers should retry with exponential backoff and jitter (AWS SDKs do this for you).

🤔 Why

Because one runaway client — a loop in a mobile app, a partner's batch job — can fill the shared account limit and take down every API in the Region. Limits at the door keep the kitchen alive and share capacity fairly. Usage plans turn "a partner" into numbers you can agree on in a contract: so many per second, so many per day.

🔧 How (in this repo)

TokenBucket in apigw/gateway.py is nine lines: take(now) adds (now - t) * rate tokens (at most burst) and takes one if it can. Each stage gets its own bucket (dev: 5 and 5; prod: the defaults RATE = 10_000, BURST = 5_000). Gateway.usage_plan() makes a per-plan bucket and a quota and maps API keys to it. For routes with api_key=True, _handle() checks the key (unknown → 403), the quota (429 Limit Exceeded) and the plan's bucket (429 Too Many Requests) — before the stage bucket.

🧪 Try it

python3 apigw/demo.py throttle
python3 - <<'EOF'
import sys; sys.path.insert(0, "apigw"); from demo import office, Clock
from gateway import TokenBucket
b = TokenBucket(rate=5, burst=5, now=0)
print("t=0.0 ", [b.take(0) for _ in range(6)])
print("t=0.2 ", [b.take(0.2) for _ in range(2)])
print("t=10  ", b.take(10), "tokens left:", b.tokens)
c = Clock(); gw, _ = office(c)
gw.usage_plan("trial", rate=100, burst=100, quota_per_day=3, keys=["key-trial"])
print([gw.handle("GET", "/partner/students/9", {"x-api-key": "key-trial"})[0] for _ in range(5)], gw.handle("GET", "/partner/students/9", {"x-api-key": "key-trial"})[2])
print("no key:", gw.handle("GET", "/partner/students/9")[0::2])
EOF

✅ Verify — what you should see

throttle prints stage dev: rate 5/s, burst 5 — 8 requests in the same instant: then 200 200 200 200 200 429 429 429, then one second later: 200 200 200 200 200 429. For the usage plan: 200 200 429 429 · unknown key → 403.

Your snippet prints t=0.0 [True, True, True, True, True, False] (five tokens, then empty), t=0.2 [True, False] (0.2 s × 5/s = one new token), t=10 True tokens left: 4 (the bucket never holds more than 5, even after ten quiet seconds), then [200, 200, 200, 429, 429] {'message': 'Limit Exceeded'} for the trial quota of 3, and no key: (403, {'message': 'Forbidden'}).

🏁 What you just proved

You can predict every 429: count the tokens in the bucket, add the refill, and check the quota. The burst decides the first moment; the rate decides every second after.

⚠️ Common mistakes

no stage or method limits, so one client can use the whole account's 10,000 rps
retrying a 429 at once in a tight loop — it makes the crowd bigger; back off
treating the usage plan's API key as authentication (lesson 06)
a quota with no alarm — the partner finds out at 11 a.m. that they are blocked until tomorrow

🏭 In production

On a real account — stage limits, then a usage plan for a partner with a key:

aws apigateway update-stage --rest-api-id abc123 --stage-name prod --patch-operations \
    op=replace,path=/*/*/throttling/rateLimit,value=500 \
    op=replace,path=/*/*/throttling/burstLimit,value=1000
aws apigateway create-usage-plan --name partner-basic \
    --throttle burstLimit=2,rateLimit=2 --quota limit=1000,period=DAY \
    --api-stages apiId=abc123,stage=prod
aws apigateway create-api-key --name partner-1 --enabled
aws apigateway create-usage-plan-key --usage-plan-id pl4n01 --key-id k3y001 --key-type API_KEY

Terraform:

resource "aws_api_gateway_usage_plan" "partner_basic" {
  name = "partner-basic"
  api_stages {
    api_id = aws_api_gateway_rest_api.school.id
    stage  = aws_api_gateway_stage.prod.stage_name
  }
  throttle_settings {
    rate_limit  = 2
    burst_limit = 2
  }
  quota_settings {
    limit  = 1000
    period = "DAY"
  }
}

🏭 Why this matters in production: set stage limits below what the back end can really handle (measure it), give every partner a usage plan, and alarm on 429s so you see a limit before a customer reports it.

⏭️ Next

The crowd is under control. Now answer the same question without bothering the kitchen: the notice board — caching.

git checkout lesson-08-caching

← Previousauth Next →caching

This page is the lesson's README from the lesson-07-throttling branch, shown here so the whole School stays on one site. Code files open on GitHub at the same branch.