🖼️ See the drawing + lab 🏠 Course home 🌿 Branch on GitHub ✏️ View source

🖼️ आकृती आणि labThe drawing + lab पूर्ण पानावर उघडा ↗Open full page ↗

📬 धडा 12 — Queues + संपूर्ण आराखडा: गर्दी gate वरच झेला

📍 तुम्ही इथे आहात: 13 पैकी धडा 12 · मागील: lesson-11-partitioning · पुढील: lesson-13-surviving-failure

📦 या ब्रँचमध्ये काय आहे

धडे 01–11, आणि शेवटचे साधन: request आणि हळू काम यांच्यामध्ये एक queue (Amazon SQS), backlog नुसार scale होणाऱ्या workers सह — आणि मग scale केलेल्या web app चा संपूर्ण आराखडा. scale/demo.py मधले queues() प्रमाणपत्रांच्या requests ची एक गर्दी queue मधून पाठवते, आणि अपयशी काम सुरक्षितपणे परत मिळवता येण्यासाठी काय लागते ते सांगते: at-least-once delivery, visibility timeout, DLQ आणि idempotent workers.

🧒 5 वर्षांच्या मुलाला समजावल्यासारखे

निकालानंतर प्रत्येक पालकाला छापलेले प्रमाणपत्र हवे असते. छपाईला वेळ लागतो. जर प्रत्येक पालक छपाई होईपर्यंत खिडकीवर थांबला, तर रांग कधीच पुढे सरकत नाही.

म्हणून दीपिका tokens 🎟️ वापरते. पालक प्रमाणपत्र मागतो, त्याला लगेच token मिळतो, आणि तो जत्रेची मजा घ्यायला जातो. Token एका पेटीत जातो. मागच्या खोलीत छपाई करणारे पेटीतून एकेक token घेतात आणि छापतात.

तीन सेकंद दर सेकंदाला 800 पालक मागणी करतात. 3 छपाई करणाऱ्यांसह पेटीत 1,500 tokens जमतात आणि दहा सेकंदांनंतरही ती रिकामी होत नाही. म्हणून दीपिका एक नियम जोडते: "पेटीतल्या प्रत्येक 100 tokens मागे एक छपाई करणारा, जास्तीत जास्त 10". पेटी भरली की छपाई करणारे वाढवले जातात; रिकामी झाली की ते घरी जातात. पेटी गर्दी झेलते: requests खिडकीवर गर्दी करण्याऐवजी पेटीत थांबतात.

प्रमाणपत्र छापताना मध्येच छपाई अडकली, तर काही वेळाने token पेटीत परत जातो, आणि दुसरा छपाई करणारा पुन्हा प्रयत्न करतो. कधी कधी तोच token दोनदा दिला जातो, म्हणून प्रत्येक छपाई करणारा आधी तपासतो "हे प्रमाणपत्र आधीच छापले आहे का?". पुन्हा पुन्हा अपयशी ठरणारा token एका बाजूच्या पेटीत जातो, म्हणजे कोणीतरी तो पाहू शकेल. Tokens पेटीत कायम राहत नाहीत: काही दिवसांनंतर जुना token फेकून दिला जातो.

🗺️ आकृती

flowchart LR
    u["👪 parents"] --> api["🍳 API<br/>202 Accepted + token"]
    api -->|"SendMessage"| q["📬 SQS queue<br/>backlog"]
    q -->|"ReceiveMessage"| w["🖨️ workers<br/>1 per 100 waiting, max 10"]
    w -->|"DeleteMessage when done"| q
    q -.->|"failed too many times"| dlq["🧯 dead-letter queue"]
    m["📊 ApproximateNumberOfMessagesVisible"] -.->|"scale workers"| w

🗺️ रेखाटलेली आवृत्ती + एक lab: https://school-edh.pages.dev/scaling/lesson-diagrams.html#l12

❓ काय

Queue — काम स्वीकारणारा भाग आणि ते करणारा भाग यांच्यामधला buffer. API लगेच उत्तर देते (202 Accepted, नंतर तपासण्यासाठी एका id सह); workers काम आपल्या वेगाने करतात.
Amazon SQS — AWS ची managed queue:
- Standard queue — खूप जास्त throughput, at-least-once delivery (एखादा message दोनदा येऊ शकतो) आणि शक्य तितका क्रम;
- FIFO queue (नाव .fifo ने संपते) — message group मध्ये क्रम राखला जातो आणि duplicates काढले जातात, पण throughput कमी.
At-least-once delivery — standard queue प्रत्येक message एक किंवा अधिक वेळा पोहोचवते. तोच message दोन workers पर्यंत, किंवा एकाच worker कडे दोनदा पोहोचू शकतो. त्याची तयारी ठेवा.
Visibility timeout — worker ला message मिळाला की SQS तो इतका वेळ लपवते (default 30 सेकंद). Worker ने तो वेळेत delete केला तर काम झाले; worker crash झाला किंवा खूप हळू असला, तर message दुसऱ्या worker साठी पुन्हा दिसू लागतो. कामाला लागणाऱ्या वेळेपेक्षा तो जास्त ठेवा.
Retries — delete न झालेला message परत येतो आणि पुन्हा प्रयत्न होतो. प्रत्येक receive त्याच्या receive count मध्ये 1 ची भर घालतो.
Dead-letter queue (DLQ) — redrive policy सांगते: maxReceiveCount receives नंतर message DLQ मध्ये हलवा. मग एक खराब (poison) message workers ना अडवत नाही, आणि एखादी व्यक्ती तो पाहून, दुरुस्तीनंतर त्याला source queue मध्ये परत redrive करू शकते. DLQ ला source queue पेक्षा जास्त retention द्या, म्हणजे बाजूला ठेवलेले messages कोणी पाहण्याआधी expire होत नाहीत.
Message retention — message queue मध्ये default 4 दिवस राहतो, आणि तुम्ही तो 1 मिनिटापासून 14 दिवसांपर्यंत ठेवू शकता. त्यानंतर तो delete होतो, काम झाले असो वा नसो. सर्वात जुन्या message चे वय पाहा (ApproximateAgeOfOldestMessage).
Idempotent worker — तोच message दोनदा केला तरी परिणाम एकदा केल्यासारखाच (आधी तपासा "प्रमाणपत्र आधीच बनले आहे का?"). At-least-once delivery मुळे हे आवश्यक आहे.
सगळे एकत्र: queue गर्दी झेलते; workers messages asynchronously हाताळतात. Retries, DLQ आणि idempotent consumers असतील तर अपयशी काम सुरक्षितपणे परत मिळवता येते.
Backlog नुसार scale — workers साठीचा संकेत म्हणजे ApproximateNumberOfMessagesVisible, किंवा त्याहून चांगला प्रत्येक worker मागचा backlog (messages ÷ workers), एक worker स्वीकारार्ह वेळेत किती संपवू शकतो त्याच्याशी तुलना करून. Kubernetes साठी हे KEDA करते; EC2 Auto Scaling group backlog-per-instance metric वर target-track करू शकतो; SQS event source असलेले Lambda function आपले pollers आपोआप scale करते.

🤔 का

कारण काही काम एखाद्या पानाला लागावा त्यापेक्षा हळू असते, आणि गर्दी servers जोडायला लागणाऱ्या वेळेपेक्षा कमी काळ टिकते. Queue "3 सेकंदांत 2,400 requests" चे "पुढच्या काही सेकंदांत पूर्ण झालेली 2,400 कामे" करते. पुढचा भाग जलद राहतो, आणि मागचा भाग त्याला झेपेल अशा वेगाने चालतो. ही जादू नाही: message तरीही expire होऊ शकतो, किंवा प्रत्येक वेळी अपयशी ठरू शकतो. Retries, DLQ आणि idempotent workers असतील तर ते अपयशी काम crash मध्ये हरवण्याऐवजी जपले जाते आणि सुरक्षितपणे परत मिळवता येते.

🔧 कसे (या repo मध्ये)

scale/sim.py मधले drain(arrivals, per_worker, workers_for) एकेक सेकंद चालवते: ते workers_for(backlog) ला किती workers चालवायचे ते विचारते — या सेकंदाचे arrivals येण्याआधीच्या backlog वरून ठरवलेले, जसे खऱ्या autoscaler ला metric थोडा उशिरा दिसतो — arrivals जोडते, आणि प्रत्येक worker ला per_worker कामे पूर्ण करू देते. प्रत्येक row म्हणजे (second, arrived, workers, done, backlog). queues() ठरलेल्या 3 workers ची तुलना "प्रत्येक 100 थांबलेल्यांमागे एक worker, किमान 1, जास्तीत जास्त 10" शी करते.

🧪 करून पाहा

python3 scale/demo.py queues
python3 - <<'EOF'
import sys; sys.path.insert(0, "scale"); from sim import drain
arrivals = [50, 800, 800, 800, 200, 50, 50, 50, 50, 50]
for name, rule in (("fixed 3", lambda b: 3), ("fixed 8", lambda b: 8), ("1 per 100 waiting, max 10", lambda b: min(10, max(1, -(-b // 100)))),
                   ("1 per 100 waiting, max 20", lambda b: min(20, max(1, -(-b // 100)))), ("1 per 50 waiting, max 20", lambda b: min(20, max(1, -(-b // 50))))):
    rows = drain(arrivals, 100, rule)
    print(f"{name:<26} peak backlog {max(r[4] for r in rows):>5} · most workers {max(r[2] for r in rows):>2} · left after 10 s {rows[-1][4]:>4}")
for s, a, w, d, b in drain(arrivals, 100, lambda b: min(10, max(1, -(-b // 100)))):
    print(f"second {s}: arrived {a:>3}, workers {w:>2}, done {d:>4}, waiting {b:>4}")
EOF
python3 scale/test_scale.py

✅ तपासा — तुम्हाला काय दिसायला हवे

queues हे छापते:

   fixed 3 workers                                      peak backlog  1500 · left after 10 s  150
   scale on backlog (1 worker per 100 waiting, max 10)  peak backlog   800 · left after 10 s    0
   the queue (SQS) absorbs the burst; workers process messages asynchronously, at their own pace
   SQS delivers at least once: a visibility timeout, retries, a DLQ and idempotent workers make failed work safe to recover

तुमचा snippet हे छापतो:

fixed 3                    peak backlog  1500 · most workers  3 · left after 10 s  150
fixed 8                    peak backlog     0 · most workers  8 · left after 10 s    0
1 per 100 waiting, max 10  peak backlog   800 · most workers  8 · left after 10 s    0
1 per 100 waiting, max 20  peak backlog   800 · most workers  8 · left after 10 s    0
1 per 50 waiting, max 20   peak backlog   700 · most workers 14 · left after 10 s    0
second 0: arrived  50, workers  1, done   50, waiting    0
second 1: arrived 800, workers  1, done  100, waiting  700
second 2: arrived 800, workers  7, done  700, waiting  800
second 3: arrived 800, workers  8, done  800, waiting  800
second 4: arrived 200, workers  8, done  800, waiting  200
second 5: arrived  50, workers  2, done  200, waiting   50
second 6: arrived  50, workers  1, done  100, waiting    0
second 7: arrived  50, workers  1, done   50, waiting    0
second 8: arrived  50, workers  1, done   50, waiting    0
second 9: arrived  50, workers  1, done   50, waiting    0

Tests ✅ L12 a queue absorbs the burst and drains to zero आणि 13/13 passed ने संपतात.

🏁 तुम्ही आत्ताच काय सिद्ध केले

Model मध्ये प्रत्येक request queue मध्ये थांबली आणि नंतर पूर्ण झाली. Backlog नुसार scaling ने गर्दी सेकंद 6 पर्यंत संपवली आणि परत 1 worker वर आले; ठरलेले 3 workers 10 सेकंदांनंतरही 150 ने मागे होते. ठरलेले 8 workers backlog तयारच होऊ देत नाहीत — पण दिवसभर 8 workers चे पैसे मोजतात. कमाल 10 वरून 20 केल्याने काहीच बदलले नाही, कारण नियमाने कधीच 8 पेक्षा जास्त मागितले नाहीत; जास्त उत्सुक नियमाने (प्रत्येक 50 मागे 1) 14 पर्यंत workers वापरले आणि peak थोडे कमी ठेवले. आणि गर्दीचा पहिला सेकंद नेहमीच हळू असतो: autoscaler ला backlog तो तयार झाल्यानंतरच दिसतो.

⚠️ नेहमीच्या चुका

user ला आत्ताच दिसायला हवे अशा कामासाठी queue (पानाचे उत्तर) — queues थोड्या वेळाने पूर्ण होऊ शकणाऱ्या कामासाठी असतात, आणि UI ने तसे सांगितले पाहिजे ("तुमचे प्रमाणपत्र तयार होत आहे")
कामापेक्षा कमी visibility timeout — काम चालू असतानाच message परत येतो, आणि ते दोनदा होते
idempotent नसलेले workers — at-least-once delivery मुळे दोन प्रमाणपत्रे जातात
dead-letter queue नाही — एक बिघडलेला message expire होईपर्यंत पुन्हा पुन्हा प्रयत्नात राहतो, आणि प्रत्येक वेळी एक worker वापरतो
दिवसेंदिवस वाढणारा backlog — retention काळापेक्षा (default 4 दिवस) जुने messages delete होतात, काम झाले असो वा नसो
backlog ऐवजी CPU नुसार workers scale करणे
मागचा database झेलू शकतो त्यापेक्षा जलद scale होणारे workers (धडा 10)

🏭 प्रत्यक्ष वापरात

खऱ्या account वर — dead-letter queue सह एक queue (queue messages 4 दिवस ठेवते, DLQ 14 दिवस; 5 receives नंतर message DLQ मध्ये जातो):

aws sqs create-queue --queue-name certificates-dlq --attributes '{"MessageRetentionPeriod": "1209600"}'
aws sqs create-queue --queue-name certificates --attributes '{
  "VisibilityTimeout": "120",
  "MessageRetentionPeriod": "345600",
  "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:ap-south-1:111122223333:certificates-dlq\",\"maxReceiveCount\":\"5\"}"
}'
aws sqs get-queue-attributes --attribute-names ApproximateNumberOfMessagesVisible \
    --queue-url https://sqs.ap-south-1.amazonaws.com/111122223333/certificates

Kubernetes वर KEDA — प्रत्येक 100 थांबलेल्या messages मागे सुमारे एक worker pod, 1 ते 10:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata: { name: certificate-workers }
spec:
  scaleTargetRef: { name: certificate-worker }
  minReplicaCount: 1
  maxReplicaCount: 10
  triggers:
    - type: aws-sqs-queue
      authenticationRef: { name: keda-aws }
      metadata:
        queueURL: https://sqs.ap-south-1.amazonaws.com/111122223333/certificates
        queueLength: "100"
        awsRegion: ap-south-1

🏗️ संपूर्ण आराखडा — scale केलेली जत्रा

flowchart LR
    u["👪 parents"] --> cf["🌍 CloudFront<br/>UI from S3 · micro-cache · shield"]
    cf -->|"/api"| lb["⚖️ ALB or API Gateway"]
    lb --> api["🍳 stateless API<br/>ASG · pods + HPA/Karpenter · Lambda"]
    api --> rc["📌 Redis / Valkey"]
    api --> px["🛎️ RDS Proxy"]
    px --> pri["📒 primary"]
    px --> rep["📚 replicas"]
    api --> ddb["🗂️ DynamoDB<br/>keys that spread"]
    api --> q["📬 SQS"] --> wk["🖨️ workers<br/>scale on backlog"]

स्तर	कशाने scale होतो	काय पाहायचे
UI	CloudFront edges, versioned files, गर्दीच्या पानांवर `s-maxage`	cache hit rate, origin requests
API	load balancer मागे stateless प्रती: ASG, HPA + Karpenter, किंवा Lambda	p99 latency, 5xx, concurrency, Pending pods
Reads	Redis cache-aside, read replicas	hit rate, `ReplicaLag`
Connections	RDS Proxy / एक pool	`DatabaseConnections` विरुद्ध `max_connections`
Writes	पसरणाऱ्या key सह partitions	throttled requests, hot keys
हळू काम	SQS + backlog नुसार workers	`ApproximateNumberOfMessagesVisible`, DLQ depth

🏭 प्रत्यक्ष वापरात हे का महत्त्वाचे: प्रत्येक स्तर स्वतःच्या संकेतानुसार scale होतो. मोठ्या दिवसाआधी संपूर्ण मार्गाचा load-test करा (धडा 02), माहीत असलेला spike schedule करा (धडे 06, 08), आणि API स्तर आता जे पाठवेल ते database स्तर — cache, replicas, proxy, partitions, queue — झेलू शकतो याची खात्री करा.

⏭️ पुढे

जत्रा आता स्तरानुसार वाढू शकते. पण मोठ्या जत्रेत जास्त भाग असतात, आणि भाग बिघडतात: एखादी हळू service, अडकलेला database, हरवलेली इमारत. शेवटचा धडा: बिघाडातून टिकणे — timeouts, सभ्य retries, breaker switch आणि तीन इमारती.

git checkout lesson-13-surviving-failure

📬 Lesson 12 — Queues + the whole blueprint: take the burst at the gate

📍 You are here: Lesson 12 of 13 · Previous: lesson-11-partitioning · Next: lesson-13-surviving-failure

📦 What's in this branch

Lessons 01–11, plus the last tool: a queue (Amazon SQS) between the request and the slow work, with workers that scale on the backlog — and then the whole blueprint of a scaled web app. queues() in scale/demo.py sends a burst of certificate requests through a queue, and names what makes failed work safe to recover: at-least-once delivery, the visibility timeout, a DLQ and idempotent workers.

🧒 Explain like I'm 5

After the results, every parent wants a printed certificate. Printing takes time. If each parent waits at the counter while it prints, the queue never moves.

So Dipika uses tokens 🎟️. A parent asks for a certificate, gets a token at once, and goes to enjoy the fair. The token goes into a box. In the back room, printers take tokens from the box, one by one, and print.

800 parents a second ask for three seconds. With 3 printers, the box fills up with 1,500 tokens and is still not empty ten seconds later. So Dipika adds a rule: "one printer for every 100 tokens in the box, up to 10". When the box fills, printers are added; when it empties, they go home. The box absorbs the rush: requests wait in it instead of crowding the counter.

If a printer jams in the middle of a certificate, the token goes back into the box after a while, and another printer tries again. Sometimes the same token is handed out twice, so each printer first checks "is this certificate already printed?". A token that fails again and again goes into a side box so someone can look at it. Tokens do not stay in the box forever: after some days, an old token is thrown away.

🗺️ Diagram

flowchart LR
    u["👪 parents"] --> api["🍳 API<br/>202 Accepted + token"]
    api -->|"SendMessage"| q["📬 SQS queue<br/>backlog"]
    q -->|"ReceiveMessage"| w["🖨️ workers<br/>1 per 100 waiting, max 10"]
    w -->|"DeleteMessage when done"| q
    q -.->|"failed too many times"| dlq["🧯 dead-letter queue"]
    m["📊 ApproximateNumberOfMessagesVisible"] -.->|"scale workers"| w

🗺️ Drawn version + a lab: https://school-edh.pages.dev/scaling/lesson-diagrams.html#l12

❓ What

Queue — a buffer between the part that accepts work and the part that does it. The API answers at once (202 Accepted, with an id to check later); workers do the work at their own pace.
Amazon SQS — AWS's managed queue:
- Standard queue — very high throughput, at-least-once delivery (a message can arrive twice) and best-effort order;
- FIFO queue (name ends in .fifo) — order kept within a message group and duplicates removed, with lower throughput.
At-least-once delivery — a standard queue delivers every message one or more times. The same message can reach two workers, or the same worker twice. Plan for it.
Visibility timeout — when a worker receives a message, SQS hides it for this long (default 30 seconds). If the worker deletes it in time, it is done; if the worker crashes or is too slow, the message appears again for another worker. Set it longer than the work takes.
Retries — a message that was not deleted comes back and is tried again. Each receive adds 1 to its receive count.
Dead-letter queue (DLQ) — the redrive policy says: after maxReceiveCount receives, move the message to the DLQ. One bad (poison) message then stops blocking the workers, and a person can look at it and redrive it back to the source queue after a fix. Give the DLQ a longer retention than the source queue, so parked messages do not expire before anyone looks.
Message retention — a message stays in a queue for 4 days by default, and you can set it from 1 minute up to 14 days. After that it is deleted, whether it was done or not. Watch the age of the oldest message (ApproximateAgeOfOldestMessage).
Idempotent worker — doing the same message twice has the same effect as once (check "certificate already made?" first). Needed because of at-least-once delivery.
Together: the queue absorbs the burst; workers process messages asynchronously. With retries, a DLQ and idempotent consumers, failed work can be recovered safely.
Scale on backlog — the signal for workers is ApproximateNumberOfMessagesVisible, or better the backlog per worker (messages ÷ workers) against what one worker can clear in an acceptable time. KEDA does this for Kubernetes; an EC2 Auto Scaling group can target-track a backlog-per-instance metric; a Lambda function with an SQS event source scales its pollers by itself.

🤔 Why

Because some work is slower than a page should be, and bursts are shorter than the time it takes to add servers. A queue turns "2,400 requests in 3 seconds" into "2,400 jobs done over the next few seconds". The front stays fast, and the back runs at a pace it can survive. It is not magic: a message can still expire, or fail every time. With retries, a DLQ and idempotent workers, that failed work is kept and can be recovered safely, instead of being lost in a crash.

🔧 How (in this repo)

drain(arrivals, per_worker, workers_for) in scale/sim.py plays one second at a time: it asks workers_for(backlog) how many workers to run — decided from the backlog before this second's arrivals, as a real autoscaler sees a metric a little late — adds the arrivals, and lets each worker finish per_worker jobs. Each row is (second, arrived, workers, done, backlog). queues() compares a fixed 3 workers with "one worker per 100 waiting, at least 1, at most 10".

🧪 Try it

python3 scale/demo.py queues
python3 - <<'EOF'
import sys; sys.path.insert(0, "scale"); from sim import drain
arrivals = [50, 800, 800, 800, 200, 50, 50, 50, 50, 50]
for name, rule in (("fixed 3", lambda b: 3), ("fixed 8", lambda b: 8), ("1 per 100 waiting, max 10", lambda b: min(10, max(1, -(-b // 100)))),
                   ("1 per 100 waiting, max 20", lambda b: min(20, max(1, -(-b // 100)))), ("1 per 50 waiting, max 20", lambda b: min(20, max(1, -(-b // 50))))):
    rows = drain(arrivals, 100, rule)
    print(f"{name:<26} peak backlog {max(r[4] for r in rows):>5} · most workers {max(r[2] for r in rows):>2} · left after 10 s {rows[-1][4]:>4}")
for s, a, w, d, b in drain(arrivals, 100, lambda b: min(10, max(1, -(-b // 100)))):
    print(f"second {s}: arrived {a:>3}, workers {w:>2}, done {d:>4}, waiting {b:>4}")
EOF
python3 scale/test_scale.py

✅ Verify — what you should see

queues prints:

   fixed 3 workers                                      peak backlog  1500 · left after 10 s  150
   scale on backlog (1 worker per 100 waiting, max 10)  peak backlog   800 · left after 10 s    0
   the queue (SQS) absorbs the burst; workers process messages asynchronously, at their own pace
   SQS delivers at least once: a visibility timeout, retries, a DLQ and idempotent workers make failed work safe to recover

Your snippet prints:

fixed 3                    peak backlog  1500 · most workers  3 · left after 10 s  150
fixed 8                    peak backlog     0 · most workers  8 · left after 10 s    0
1 per 100 waiting, max 10  peak backlog   800 · most workers  8 · left after 10 s    0
1 per 100 waiting, max 20  peak backlog   800 · most workers  8 · left after 10 s    0
1 per 50 waiting, max 20   peak backlog   700 · most workers 14 · left after 10 s    0
second 0: arrived  50, workers  1, done   50, waiting    0
second 1: arrived 800, workers  1, done  100, waiting  700
second 2: arrived 800, workers  7, done  700, waiting  800
second 3: arrived 800, workers  8, done  800, waiting  800
second 4: arrived 200, workers  8, done  800, waiting  200
second 5: arrived  50, workers  2, done  200, waiting   50
second 6: arrived  50, workers  1, done  100, waiting    0
second 7: arrived  50, workers  1, done   50, waiting    0
second 8: arrived  50, workers  1, done   50, waiting    0
second 9: arrived  50, workers  1, done   50, waiting    0

The tests end with ✅ L12 a queue absorbs the burst and drains to zero and 13/13 passed.

🏁 What you just proved

In the model every request waited in the queue and was done later. Scaling on backlog cleared the burst by second 6 and went back to 1 worker; a fixed 3 was still 150 behind after 10 seconds. A fixed 8 never lets a backlog form — but pays for 8 workers all day. Raising the maximum from 10 to 20 changed nothing, because the rule never asked for more than 8; a more eager rule (1 per 50) used up to 14 workers and kept the peak a little lower. And the first second of the burst is always slow: the autoscaler only sees the backlog after it forms.

⚠️ Common mistakes

a queue for work the user must see now (the answer to the page) — queues are for work that can finish a little later, and the UI must say so ("your certificate is being prepared")
a visibility timeout shorter than the work — the message comes back while it is still being worked on, and it is done twice
workers that are not idempotent — at-least-once delivery sends two certificates
no dead-letter queue — one broken message is retried until it expires, and uses a worker each time
a backlog that grows for days — messages older than the retention period (4 days by default) are deleted, done or not
scaling workers on CPU instead of backlog
workers that scale faster than the database behind them can take (lesson 10)

🏭 In production

On a real account — a queue with a dead-letter queue (the queue keeps messages 4 days, the DLQ 14 days; after 5 receives a message moves to the DLQ):

aws sqs create-queue --queue-name certificates-dlq --attributes '{"MessageRetentionPeriod": "1209600"}'
aws sqs create-queue --queue-name certificates --attributes '{
  "VisibilityTimeout": "120",
  "MessageRetentionPeriod": "345600",
  "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:ap-south-1:111122223333:certificates-dlq\",\"maxReceiveCount\":\"5\"}"
}'
aws sqs get-queue-attributes --attribute-names ApproximateNumberOfMessagesVisible \
    --queue-url https://sqs.ap-south-1.amazonaws.com/111122223333/certificates

KEDA on Kubernetes — about one worker pod per 100 waiting messages, from 1 to 10:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata: { name: certificate-workers }
spec:
  scaleTargetRef: { name: certificate-worker }
  minReplicaCount: 1
  maxReplicaCount: 10
  triggers:
    - type: aws-sqs-queue
      authenticationRef: { name: keda-aws }
      metadata:
        queueURL: https://sqs.ap-south-1.amazonaws.com/111122223333/certificates
        queueLength: "100"
        awsRegion: ap-south-1

🏗️ The whole blueprint — the scaled fair

flowchart LR
    u["👪 parents"] --> cf["🌍 CloudFront<br/>UI from S3 · micro-cache · shield"]
    cf -->|"/api"| lb["⚖️ ALB or API Gateway"]
    lb --> api["🍳 stateless API<br/>ASG · pods + HPA/Karpenter · Lambda"]
    api --> rc["📌 Redis / Valkey"]
    api --> px["🛎️ RDS Proxy"]
    px --> pri["📒 primary"]
    px --> rep["📚 replicas"]
    api --> ddb["🗂️ DynamoDB<br/>keys that spread"]
    api --> q["📬 SQS"] --> wk["🖨️ workers<br/>scale on backlog"]

Tier	Scales by	Watch
UI	CloudFront edges, versioned files, `s-maxage` on busy pages	cache hit rate, origin requests
API	stateless copies behind a load balancer: ASG, HPA + Karpenter, or Lambda	p99 latency, 5xx, concurrency, Pending pods
Reads	Redis cache-aside, read replicas	hit rate, `ReplicaLag`
Connections	RDS Proxy / a pool	`DatabaseConnections` vs `max_connections`
Writes	partitions with a key that spreads	throttled requests, hot keys
Slow work	SQS + workers on backlog	`ApproximateNumberOfMessagesVisible`, DLQ depth

🏭 Why this matters in production: every tier scales on its own signal. Before a big day, load-test the whole path (lesson 02), schedule the known spike (lessons 06, 08), and make sure the database tier — cache, replicas, proxy, partitions, queue — can take what the API tier will now send it.

⏭️ Next

The fair can now grow tier by tier. But a bigger fair has more parts, and parts break: a slow service, a stuck database, a lost building. Last lesson: surviving failure — timeouts, polite retries, a breaker switch and three buildings.

git checkout lesson-13-surviving-failure

← Previouspartitioning Next →surviving failure

This page is the lesson's README from the lesson-12-queues-and-blueprint branch, shown here so the whole School stays on one site. Code files open on GitHub at the same branch.