🖼️ See the drawing + lab 🏠 Course home 🌿 Branch on GitHub ✏️ View source

🖼️ आकृती आणि labThe drawing + lab पूर्ण पानावर उघडा ↗Open full page ↗

🛡️ धडा 12 — Private APIs, WAF, मर्यादा आणि खर्च: कडक मर्यादा

📍 तुम्ही इथे आहात: 12 पैकी धडा 12 · मागे: lesson-11-monitoring

📦 या ब्रँचमध्ये काय आहे

धडे 01–11, आणि फक्त इच्छा करून न हलणाऱ्या मर्यादा: integration timeout (REST APIs वर default 29 s, HTTP APIs वर 30 s), 10 MB payload मर्यादा, interface VPC endpoint मागचे private APIs, REST stages समोरचा AWS WAF, आणि bill. apigw/demo.py मधले limits() 504 आणि 502 दाखवते, आणि नेमक्या मर्यादा तुम्ही स्वतः शोधता.

🧒 5 वर्षांच्या मुलाला समजावल्यासारखे

Front office चे काही नियम असे आहेत जे कोणीही वाकवू शकत नाही:

⏱️ Stopwatch: visitor kitchen साठी जास्तीत जास्त 29 सेकंद थांबतो. Kitchen अजून स्वयंपाक करत असेल, तर clerk म्हणतो "kitchen ने खूप वेळ घेतला" (504) — जेवण जवळजवळ तयार असले तरीही. लांब कामासाठी clerk त्याऐवजी visitor ला token क्रमांक देतो आणि म्हणतो "नंतर परत या".
📦 Parcel चा आकार: 10 MB पेक्षा मोठे कोणतेही parcel desk वरून पुढे जात नाही (413). मोठ्या files वेगळ्या माल-फाटकातून जातात (pre-signed S3 link).
🔥 Kitchen मध्ये आग: kitchen बिघडले तर clerk म्हणतो "आत काहीतरी बिघडले आहे" (502) — आणि visitor ला kitchen कधीच दाखवत नाही.

काही offices फक्त शाळेच्या आतच 🏫 असतात — एक खाजगी दार, ज्यापर्यंत आधीच campus वर असलेले लोकच पोहोचू शकतात. आणि office समोर एक पहारेकरी 🛡️ (WAF) उभा राहू शकतो, जो ओळखीच्या त्रासदायक लोकांना clerk पर्यंत पोहोचण्याआधीच परत पाठवतो.

🗺️ आकृती

flowchart LR
    c["🙋 caller"] --> waf["🛡️ AWS WAF<br/>SQL injection · IP sets · rate rules"]
    waf --> gw["🛎️ REST API stage"]
    gw -->|"body > 10 MB"| e413["413 Request too large"]
    gw -->|"kitchen > 29 s"| e504["504 Endpoint request timed out"]
    gw -->|"kitchen crashes"| e502["502 Internal server error"]
    vpc["🏫 VPC → interface endpoint<br/>com.amazonaws.region.execute-api"] -->|"resource policy allows this vpce"| priv["🔒 private API"]

🗺️ काढलेली आकृती + एक lab: https://school-edh.pages.dev/apigateway/lesson-diagrams.html#l12

❓ काय

Integration timeout — REST APIs: default 29 सेकंद. 2024 पासून तुम्ही (Service Quotas मधून) Regional आणि private REST APIs साठी जास्त मोठा timeout मागू शकता; त्याबदल्यात AWS तुमच्या account चा throttle quota कमी करू शकते. HTTP APIs: 30 सेकंद, ही कडक कमाल मर्यादा. Caller ला 504 मिळतो.
Payload — REST आणि HTTP API requests साठी 10 MB. Lambda integration वर Lambda ची स्वतःची synchronous payload मर्यादाही (6 MB) लागू होते. Model 413 उत्तर देते.
WebSocket — 10 मिनिटे निष्क्रिय राहिल्यावर किंवा एकूण 2 तासांनंतर office connection बंद करते.
लांब कामे — request स्वीकारा, job id सह 202 Accepted उत्तर द्या, काम background मध्ये करा (SQS, Step Functions), आणि client ला poll करू द्या किंवा push पाठवा (WebSocket, email). मोठ्या files — pre-signed S3 URL द्या; file कधीच API मधून जात नाही.
502 — back end अपयशी ठरला किंवा त्याने बिघडलेले उत्तर दिले (उदा. error फेकणारे Lambda proxy function).
Private API (REST) — endpoint type PRIVATE. फक्त execute-api साठीच्या interface VPC endpoint मधूनच (com.amazonaws.<region>.execute-api) पोहोचता येते, आणि तेही फक्त API ची resource policy परवानगी देत असेल तर (उदाहरणार्थ aws:SourceVpce ने).
AWS WAF — REST API stage ला web ACL जोडा: managed rule groups (SQL injection, ओळखीचे वाईट inputs), IP sets, भौगोलिक rules आणि rate-based rules. HTTP APIs ना WAF थेट जोडता येत नाही — समोर WAF सह CloudFront ठेवा.
खर्च (अंदाजे, us-east-1, पहिला tier): REST ~$3.50 प्रति दशलक्ष requests, HTTP ~$1.00 प्रति दशलक्ष, WebSocket ~$1.00 प्रति दशलक्ष messages अधिक connection minutes — त्याशिवाय cache तास (धडा 08), बाहेर जाणारे data transfer, CloudWatch Logs, आणि WAF (web ACL, rules आणि requests). तुमच्या Region साठी pricing page तपासा.

🤔 का

कारण या मर्यादा तुमचे design ठरवतात, तुमचे configuration नाही. 45 सेकंदांचा report default timeout असलेल्या REST API मागे नक्कीच अपयशी ठरेल, तुम्ही कितीही tune केले तरी — उपाय म्हणजे वेगळा आकार (आधी स्वीकारा, मग poll). Private API हा internal service प्रसिद्ध करण्याचा, internet चा कोणताही मार्ग न ठेवता, सगळ्यात सोपा मार्ग आहे. आणि bill प्रत्येक request सोबत वाढते, म्हणून खूप जास्त आणि स्थिर volume वर load balancer स्वस्त पडू शकतो.

🔧 कसे (या repo मध्ये)

apigw/gateway.py मध्ये मर्यादा constants म्हणून आहेत: TIMEOUT_S = 29 आणि MAX_PAYLOAD = 10 * 1024 * 1024. Gateway._handle() मध्ये MAX_PAYLOAD पेक्षा (JSON म्हणून) मोठ्या body ला 413 मिळतो; ज्या route ची latency 29 s पेक्षा जास्त आहे त्याला kitchen ला call न करताच 504 मिळतो; exception फेकणाऱ्या integration ला 502 Internal server error मिळतो. apigw/demo.py मध्ये /report ला 31 s लागतात आणि /broken RuntimeError फेकतो. Private endpoints, WAF आणि किमती limits() छापते — lab त्या चालवू शकत नाही.

🧪 करून पाहा

python3 apigw/demo.py limits
python3 - <<'EOF'
import sys; sys.path.insert(0, "apigw"); from demo import office
from gateway import Route, TIMEOUT_S, MAX_PAYLOAD
gw, _ = office()
gw.add(Route("GET", "/slow/{s}", lambda e: (200, {"ok": True}), latency=29),
       Route("GET", "/slower/{s}", lambda e: (200, {"ok": True}), latency=29.5),
       Route("POST", "/upload", lambda e: (201, {"size": len(e["body"]["data"])})))
print("29 s  :", gw.handle("GET", "/slow/x")[0::2])
print("29.5 s:", gw.handle("GET", "/slower/x")[0::2])
print("limit :", TIMEOUT_S, "s ·", MAX_PAYLOAD, "bytes")
print("1 MB  :", gw.handle("POST", "/upload", body={"data": "x" * 1_000_000})[0::2])
print("11 MB :", gw.handle("POST", "/upload", body={"data": "x" * 11_000_000})[0::2])
EOF
python3 apigw/test_apigw.py

आणि चालू office वर:

python3 apigw/demo.py serve &
sleep 1                                # give the office a second to open
curl -i localhost:8080/report
kill %1

✅ तपासा — तुम्हाला काय दिसायला हवे

limits छापते hard edges: integration timeout 29 s (REST default), payload 10 MB, GET /report → 504 {'message': 'Endpoint request timed out'}, GET /broken → 502 {'message': 'Internal server error'}, मग लांब कामे, private API, WAF यांच्या ओळी, आणि cost (example, us-east-1): REST ~$3.50 per million requests, HTTP ~$1.00 per million — plus cache hours and data out.

तुमचा snippet हे छापतो:

29 s  : (200, {'ok': True})
29.5 s: (504, {'message': 'Endpoint request timed out'})
limit : 29 s · 10485760 bytes
1 MB  : (201, {'size': 1000000})
11 MB : (413, {'message': 'Request too large'})

आणि tests 12/12 passed ने संपतात. चालू office {"message": "Endpoint request timed out"} सह HTTP/1.0 504 Gateway Timeout उत्तर देते.

🏁 तुम्ही आत्ताच काय सिद्ध केले

तुम्ही मर्यादा नेमक्या शोधल्या: 29 s पास होतो आणि 29.5 s नाही; 1 MB पास होतो आणि 11 MB नाही; बिघडलेले kitchen तपशील लपवणारा साधा 502 बनते.

⚠️ नेहमीच्या चुका

REST API मागे Lambda timeout 5 मिनिटे करणे आणि caller थांबेल अशी अपेक्षा ठेवणे — office तरीही 29 s ला थांबते (तुम्ही तो quota वाढवला नसेल तर)
pre-signed S3 URL ऐवजी API मधून files upload करणे
interface endpoint असलेला पण resource policy नसलेला private API — प्रत्येक call नाकारला जातो
VPC मध्ये execute-api endpoint साठी private DNS चालू करणे आणि मग लक्षात येणे की त्या VPC मधून public APIs त्यांच्या default नावांनी पोहोचता येत नाहीत (त्यांच्यासाठी custom domain वापरा)
custom domain च्या API वर WAF, पण default execute-api endpoint अजूनही उघडा
bill वरच्या "त्याशिवाय" गोष्टींकडे दुर्लक्ष: cache तास, logs आणि WAF यांचा खर्च अनेकदा requests पेक्षा जास्त असतो

🏭 प्रत्यक्ष वापरात

खऱ्या account वर — execute-api साठी interface endpoint, फक्त तोच स्वीकारणारा private REST API, आणि public stage वर WAF:

aws ec2 create-vpc-endpoint --vpc-id vpc-0123456789abcdef0 --vpc-endpoint-type Interface \
    --service-name com.amazonaws.ap-south-1.execute-api \
    --subnet-ids subnet-0aaa1111bbbb2222c --security-group-ids sg-0123456789abcdef0
aws apigateway create-rest-api --name school-internal --endpoint-configuration types=PRIVATE \
    --policy file://private-policy.json
aws wafv2 associate-web-acl \
    --web-acl-arn arn:aws:wafv2:ap-south-1:111122223333:regional/webacl/school-api-acl/1234abcd-12ab-34cd-56ef-1234567890ab \
    --resource-arn arn:aws:apigateway:ap-south-1::/restapis/abc123/stages/prod

private-policy.json — फक्त त्या एकाच endpoint मधून calls ना परवानगी:

{
  "Version": "2012-10-17",
  "Statement": [
    { "Effect": "Allow", "Principal": "*", "Action": "execute-api:Invoke",
      "Resource": "execute-api:/*" },
    { "Effect": "Deny", "Principal": "*", "Action": "execute-api:Invoke",
      "Resource": "execute-api:/*",
      "Condition": { "StringNotEquals": { "aws:SourceVpce": "vpce-0123456789abcdef0" } } }
  ]
}

🏭 Production मध्ये हे का महत्त्वाचे आहे: पहिल्या दिवसापासूनच लांब कामे accept-then-poll म्हणून design करा, files S3 मधून पाठवा, internal APIs private ठेवा, public REST stages समोर WAF ठेवा, आणि महिन्यातून एकदा संपूर्ण bill वाचा — फक्त requests ची ओळ नाही.

🎓 Front office आता तुमचे आहे

एक front office → offices चे तीन प्रकार → कोणता desk, कोणते kitchen → form तपासणी आणि स्वच्छ उत्तर → रंगीत तालीम आणि खरा प्रयोग → badge तपासणी → दारावरचा token bucket → सूचना फलक → browser आधी विचारतो → नावफलक → dashboard आणि stopwatch → कडक मर्यादा. तुम्ही फक्त API Gateway शिकला नाहीत — तुम्ही front office चालवू शकता. 🛎️🎓

शाळेतली पुढची दारे: API school office मागे काय बसते त्याचे design करते; IAM school SigV4 आणि IAM auth मागचे roles समजावते; VPC school private API ज्यात राहतो तो खाजगी campus बांधते.

git checkout main
python3 apigw/demo.py     # one last run, for fun

🛡️ Lesson 12 — Private APIs, WAF, limits & cost: the hard edges

📍 You are here: Lesson 12 of 12 · Previous: lesson-11-monitoring

📦 What's in this branch

Lessons 01–11, plus the edges you cannot move by wishing: the integration timeout (29 s by default on REST APIs, 30 s on HTTP APIs), the 10 MB payload limit, private APIs behind an interface VPC endpoint, AWS WAF in front of REST stages, and the bill. limits() in apigw/demo.py shows a 504 and a 502, and you find the exact edges yourself.

🧒 Explain like I'm 5

The front office has a few rules that nobody can bend:

⏱️ The stopwatch: a visitor waits at most 29 seconds for the kitchen. If the kitchen is still cooking, the clerk says "the kitchen took too long" (504) — even if the food is almost ready. For a long job, the clerk gives the visitor a ticket number instead and says "come back later".
📦 The parcel size: no parcel bigger than 10 MB passes the desk (413). Big files go through a separate loading gate (a pre-signed S3 link).
🔥 A kitchen fire: if the kitchen breaks, the clerk says "something went wrong inside" (502) — and never shows the visitor the kitchen.

Some offices are inside the school only 🏫 — a private door that only people already on the campus can reach. And a guard 🛡️ (WAF) can stand in front of the office, turning away known troublemakers before they even reach the clerk.

🗺️ Diagram

flowchart LR
    c["🙋 caller"] --> waf["🛡️ AWS WAF<br/>SQL injection · IP sets · rate rules"]
    waf --> gw["🛎️ REST API stage"]
    gw -->|"body > 10 MB"| e413["413 Request too large"]
    gw -->|"kitchen > 29 s"| e504["504 Endpoint request timed out"]
    gw -->|"kitchen crashes"| e502["502 Internal server error"]
    vpc["🏫 VPC → interface endpoint<br/>com.amazonaws.region.execute-api"] -->|"resource policy allows this vpce"| priv["🔒 private API"]

🗺️ Drawn version + a lab: https://school-edh.pages.dev/apigateway/lesson-diagrams.html#l12

❓ What

Integration timeout — REST APIs: 29 seconds by default. Since 2024 you can ask (through Service Quotas) for a longer timeout on Regional and private REST APIs; AWS may lower your account's throttle quota in return. HTTP APIs: 30 seconds, a hard maximum. The caller gets 504.
Payload — 10 MB for REST and HTTP API requests. A Lambda integration is also limited by Lambda's own synchronous payload limit (6 MB). The model answers 413.
WebSocket — the office closes a connection after 10 minutes idle or 2 hours in total.
Long jobs — accept the request, answer 202 Accepted with a job id, do the work in the background (SQS, Step Functions), and let the client poll or get a push (WebSocket, email). Big files — hand out a pre-signed S3 URL; the file never passes through the API.
502 — the back end failed or returned a malformed answer (e.g. a Lambda proxy function that threw an error).
Private API (REST) — endpoint type PRIVATE. Reachable only through an interface VPC endpoint for execute-api (com.amazonaws.<region>.execute-api), and only if the API's resource policy allows it (for example with aws:SourceVpce).
AWS WAF — attach a web ACL to a REST API stage: managed rule groups (SQL injection, known bad inputs), IP sets, geographic rules and rate-based rules. HTTP APIs cannot attach WAF directly — put CloudFront with WAF in front.
Cost (approximate, us-east-1, first tier): REST ~$3.50 per million requests, HTTP ~$1.00 per million, WebSocket ~$1.00 per million messages plus connection minutes — plus cache hours (lesson 08), data transfer out, CloudWatch Logs, and WAF (web ACL, rules and requests). Check the pricing page for your Region.

🤔 Why

Because these edges decide your design, not your configuration. A 45-second report will fail behind a REST API with the default timeout, however you tune it — the fix is a different shape (accept, then poll). A private API is the simplest way to publish an internal service without any internet path. And the bill grows with every request, so at very high steady volume a load balancer can be cheaper.

🔧 How (in this repo)

apigw/gateway.py has the edges as constants: TIMEOUT_S = 29 and MAX_PAYLOAD = 10 * 1024 * 1024. In Gateway._handle() a body larger than MAX_PAYLOAD (as JSON) gets 413; a route whose latency is more than 29 s gets 504 without calling the kitchen; an integration that raises an exception gets 502 Internal server error. In apigw/demo.py, /report takes 31 s and /broken raises RuntimeError. Private endpoints, WAF and prices are printed by limits() — the lab cannot run them.

🧪 Try it

python3 apigw/demo.py limits
python3 - <<'EOF'
import sys; sys.path.insert(0, "apigw"); from demo import office
from gateway import Route, TIMEOUT_S, MAX_PAYLOAD
gw, _ = office()
gw.add(Route("GET", "/slow/{s}", lambda e: (200, {"ok": True}), latency=29),
       Route("GET", "/slower/{s}", lambda e: (200, {"ok": True}), latency=29.5),
       Route("POST", "/upload", lambda e: (201, {"size": len(e["body"]["data"])})))
print("29 s  :", gw.handle("GET", "/slow/x")[0::2])
print("29.5 s:", gw.handle("GET", "/slower/x")[0::2])
print("limit :", TIMEOUT_S, "s ·", MAX_PAYLOAD, "bytes")
print("1 MB  :", gw.handle("POST", "/upload", body={"data": "x" * 1_000_000})[0::2])
print("11 MB :", gw.handle("POST", "/upload", body={"data": "x" * 11_000_000})[0::2])
EOF
python3 apigw/test_apigw.py

And on the live office:

python3 apigw/demo.py serve &
sleep 1                                # give the office a second to open
curl -i localhost:8080/report
kill %1

✅ Verify — what you should see

limits prints hard edges: integration timeout 29 s (REST default), payload 10 MB, GET /report → 504 {'message': 'Endpoint request timed out'}, GET /broken → 502 {'message': 'Internal server error'}, then the lines on long jobs, the private API, WAF and cost (example, us-east-1): REST ~$3.50 per million requests, HTTP ~$1.00 per million — plus cache hours and data out.

Your snippet prints:

29 s  : (200, {'ok': True})
29.5 s: (504, {'message': 'Endpoint request timed out'})
limit : 29 s · 10485760 bytes
1 MB  : (201, {'size': 1000000})
11 MB : (413, {'message': 'Request too large'})

and the tests end with 12/12 passed. The live office answers HTTP/1.0 504 Gateway Timeout with {"message": "Endpoint request timed out"}.

🏁 What you just proved

You found the edges exactly: 29 s passes and 29.5 s does not; 1 MB passes and 11 MB does not; a crashing kitchen becomes a plain 502 that hides the details.

⚠️ Common mistakes

raising the Lambda timeout to 5 minutes behind a REST API and expecting the caller to wait — the office still stops at 29 s (unless you raised that quota)
uploading files through the API instead of a pre-signed S3 URL
a private API with an interface endpoint but no resource policy — every call is refused
turning on private DNS for the execute-api endpoint in a VPC and then finding that public APIs are no longer reachable by their default names from that VPC (use a custom domain for those)
WAF on the custom domain's API but the default execute-api endpoint still open
ignoring the "plus" items on the bill: cache hours, logs and WAF often cost more than the requests

🏭 In production

On a real account — an interface endpoint for execute-api, a private REST API that only accepts it, and WAF on a public stage:

aws ec2 create-vpc-endpoint --vpc-id vpc-0123456789abcdef0 --vpc-endpoint-type Interface \
    --service-name com.amazonaws.ap-south-1.execute-api \
    --subnet-ids subnet-0aaa1111bbbb2222c --security-group-ids sg-0123456789abcdef0
aws apigateway create-rest-api --name school-internal --endpoint-configuration types=PRIVATE \
    --policy file://private-policy.json
aws wafv2 associate-web-acl \
    --web-acl-arn arn:aws:wafv2:ap-south-1:111122223333:regional/webacl/school-api-acl/1234abcd-12ab-34cd-56ef-1234567890ab \
    --resource-arn arn:aws:apigateway:ap-south-1::/restapis/abc123/stages/prod

private-policy.json — allow calls only through that one endpoint:

{
  "Version": "2012-10-17",
  "Statement": [
    { "Effect": "Allow", "Principal": "*", "Action": "execute-api:Invoke",
      "Resource": "execute-api:/*" },
    { "Effect": "Deny", "Principal": "*", "Action": "execute-api:Invoke",
      "Resource": "execute-api:/*",
      "Condition": { "StringNotEquals": { "aws:SourceVpce": "vpce-0123456789abcdef0" } } }
  ]
}

🏭 Why this matters in production: design long jobs as accept-then-poll from day one, send files through S3, keep internal APIs private, put WAF in front of public REST stages, and read the whole bill once a month — not only the request line.

🎓 The front office is yours

One front office → three kinds of office → which desk, which kitchen → the form check and the clean answer → rehearsal and the real show → the badge check → the token bucket at the door → the notice board → the browser asks first → the name plate → the dashboard and the stopwatch → the hard edges. You didn't just learn API Gateway — you can run the front office. 🛎️🎓

Next doors in the school: the API school designs what sits behind the office; the IAM school explains SigV4 and the roles behind IAM auth; the VPC school builds the private campus a private API lives in.

git checkout main
python3 apigw/demo.py     # one last run, for fun

← Previousmonitoring Finished! Take the quiz →check what stuck

This page is the lesson's README from the lesson-12-private-waf-limits branch, shown here so the whole School stays on one site. Code files open on GitHub at the same branch.