🩺 धडा 16 — debugging playbook: परिचारिकेचा triage chart

📍 तुम्ही इथे आहात: धडा 16 / 26 — भाग 3 सुरू: ते खरोखर चालवणे · मागील: lesson-15-multi-az-scaling · पुढील: lesson-17-rbac

📦 या ब्रँचमध्ये काय आहे

आधीचे सगळे, आणि तुम्ही आयुष्यभर दर आठवड्याला वापराल असा धडा: प्रत्येक नेहमीच्या pod आजारासाठी triage chart, आणि ते सगळे ओळखणारा पाच tools चा सराव.

🧒 5 वर्षांच्या मुलाला समजावल्यासारखे

शाळेची परिचारिका 🩺 घाबरत नाही आणि अंदाज लावत नाही. कोणीही आले तरी ती तोच सराव करते: chart पाहा, ठरलेले प्रश्न विचारा, मग उपचार करा. Kubernetes debugging अगदी तसेच आहे. पाच प्रश्न, क्रमाने:

kubectl describe pod X — रुग्णाचा chart: तळाशी असलेले Events म्हणजे प्रत्येक निदानाचा 90%. ते आधी वाचा, नेहमी.
kubectl logs X — बेशुद्ध पडण्याआधी रुग्ण काय म्हणाला. --previous = मागच्या वेळी बेशुद्ध पडण्याआधी काय म्हणाला (crash loops साठी फार महत्त्वाचे!). -f = थेट ऐका.
kubectl get events -w — कॉरिडॉरमधल्या गप्पा: खोलीत घडणारे सगळे, क्रमाने.
kubectl exec -it X -- sh — रुग्णाला आतून तपासा.
kubectl port-forward X 8080:3000 — reception टाळून रुग्णाला थेट बोलवा (Service/Ingress ला बगल देऊन नेमके कुठे बिघडते ते वेगळे करा).

🗺️ आकृती

flowchart TB
    sick["🤒 pod is sick"] --> desc["1 describe → read EVENTS"]
    desc --> p["Pending 🪑<br/>no desk fits:<br/>requests too big? taints?<br/>cluster full? (L08, L15, L20)"]
    desc --> i["ImagePullBackOff 🍱<br/>wrong lunchbox label:<br/>typo? tag missing? no ECR<br/>permission? (L01, AWS L05)"]
    desc --> c["CrashLoopBackOff 💥<br/>starts, dies, repeats:<br/>logs --previous!<br/>bad config? (L06)"]
    desc --> o["OOMKilled 🫃<br/>ate over the limit:<br/>raise limit or fix leak (L08)"]
    desc --> s["Service silent ☎️<br/>selector ≠ labels?<br/>kubectl get endpointslices (L04)"]

❓ काय (chart, लक्षणानुसार)

लक्षण	अर्थ	पहिला प्रश्न
`Pending`	scheduler ला बाक सापडला नाही	`describe`: कोणती अट अपयशी — resources, taints, volume zone?
`ImagePullBackOff`	डबा आणता येत नाही	नेमकी image string? tag आहे का? registry auth (node ची IAM टोपी)?
`CrashLoopBackOff`	सुरू होतो मग मरतो, पुन्हा पुन्हा	`logs --previous`; मग env/config, मग command
`OOMKilled` (exit 137)	memory limit गाठली	`describe` मध्ये Last State दिसते; limit वाढवा किंवा leak शोधा
`Running` पण `Ready` नाही	readiness probe अपयशी	`/readyz` खरोखर निरोगी आहे का? DB पोहोचते का? (L07)
Service काहीच परत देत नाही	selector/labels जुळत नाहीत	`get endpointslices` — रिकामी यादी = reception ला कोणीच नाही
Node `NotReady`	बाकच आजारी आहे	ही EC2 ची गोष्ट: `describe node`, मग AWS कोर्स L12

🤔 का

आधीच्या प्रत्येक धड्याने एक यंत्रणा शिकवली; incidents म्हणजे त्या यंत्रणा एकेक करून अपयशी होणे. तज्ज्ञ जास्त हुशार नसतात — ते फक्त अंदाज लावण्याऐवजी नेहमी तोच सराव करतात. Events → logs → exec. याच क्रमाने.

🧪 करून पाहा — दवाखान्याचा सराव (सुरक्षित, स्वतः ओढवून घेतलेला)

# patient 1: ImagePullBackOff
kubectl -n school run sick1 --image=nginxdemos/hello:no-such-tag
kubectl -n school describe pod sick1 | tail -5        # Events tell you instantly

# patient 2: CrashLoopBackOff
kubectl -n school run sick2 --image=busybox -- sh -c "echo I die now; exit 1"
kubectl -n school logs sick2 --previous 2>/dev/null || kubectl -n school logs sick2

# patient 3: Pending (impossible request)
kubectl -n school run sick3 --image=nginx --overrides='{"spec":{"containers":[{"name":"sick3","image":"nginx","resources":{"requests":{"cpu":"100"}}}]}}'
kubectl -n school describe pod sick3 | grep -A3 Events

# discharge everyone:
kubectl -n school delete pod sick1 sick2 sick3

🔧 या धड्यासाठी kubectl

kubectl describe · kubectl get events · kubectl logs --previous · kubectl exec

🔗 यावर आधारित: 🪪 AWS L05 (ImagePullBackOff बहुधा ECR permission मुळे असतो)

✅ तपासा — तुम्हाला काय दिसले पाहिजे

दवाखान्यातल्या प्रत्येक आजारी pod चे निदान describe → events → logs (--previous) → exec मधून होते, डोकावून न पाहता.

🧹 साफसफाई

local cluster: kubectl delete -f k8s/ (किंवा तुम्ही apply केलेली file) — EKS वर चालू राहिलेल्या कशाचेही तासाप्रमाणे बिल येते

⚠️ नेहमीच्या चुका

events आधी logs वाचणे — ते कधीच सुरू का झाले नाही ते events सांगतात
CrashLoopBackOff वर --previous विसरणे

⏭️ पुढे

हे commands चालवायची परवानगी मुळात कोणाला आहे — आणि तुमचा CI robot cluster-admin का नसावा: RBAC.

git checkout lesson-17-rbac

🩺 Lesson 16 — The debugging playbook: the nurse's triage chart

📍 You are here: Lesson 16 of 26 — Part 3 begins: running it for real · Previous: lesson-15-multi-az-scaling · Next: lesson-17-rbac

📦 What's in this branch

Everything before, plus the lesson you'll use weekly forever: the triage chart for every classic pod sickness, and the five-tool drill that diagnoses all of them.

🧒 Explain like I'm 5

The school nurse 🩺 doesn't panic and doesn't guess. Whatever walks in, she runs the same drill: look at the chart, ask the standard questions, then treat. Kubernetes debugging is exactly that. The five questions, in order:

kubectl describe pod X — the patient's chart: Events at the bottom are 90% of every diagnosis. Read them first, always.
kubectl logs X — what the patient said before fainting. --previous = what they said before the last faint (crucial for crash loops!). -f = listen live.
kubectl get events -w — the hallway gossip: everything happening in the room, in order.
kubectl exec -it X -- sh — examine the patient from inside.
kubectl port-forward X 8080:3000 — call the patient directly, skipping reception (bypasses Service/Ingress to isolate WHERE it breaks).

🗺️ Diagram

flowchart TB
    sick["🤒 pod is sick"] --> desc["1 describe → read EVENTS"]
    desc --> p["Pending 🪑<br/>no desk fits:<br/>requests too big? taints?<br/>cluster full? (L08, L15, L20)"]
    desc --> i["ImagePullBackOff 🍱<br/>wrong lunchbox label:<br/>typo? tag missing? no ECR<br/>permission? (L01, AWS L05)"]
    desc --> c["CrashLoopBackOff 💥<br/>starts, dies, repeats:<br/>logs --previous!<br/>bad config? (L06)"]
    desc --> o["OOMKilled 🫃<br/>ate over the limit:<br/>raise limit or fix leak (L08)"]
    desc --> s["Service silent ☎️<br/>selector ≠ labels?<br/>kubectl get endpointslices (L04)"]

❓ What (the chart, symptom by symptom)

Symptom	Meaning	First question
`Pending`	scheduler found no desk	`describe`: which condition failed — resources, taints, volume zone?
`ImagePullBackOff`	can't fetch the lunchbox	exact image string? tag exists? registry auth (node's IAM hat)?
`CrashLoopBackOff`	starts then dies, repeatedly	`logs --previous`; then env/config, then command
`OOMKilled` (exit 137)	memory limit hit	`describe` shows Last State; raise limit or find the leak
`Running` but not `Ready`	readiness probe failing	is `/readyz` actually healthy? DB reachable? (L07)
Service returns nothing	selector/labels mismatch	`get endpointslices` — empty list = reception has nobody
Node `NotReady`	the desk itself is sick	it's an EC2 story: `describe node`, then AWS course L12

🤔 Why

Every earlier lesson taught a mechanism; incidents are those mechanisms failing one at a time. The pros aren't smarter — they just always run the same drill instead of guessing. Events → logs → exec. In that order.

🧪 Try it — infirmary drill (safe, self-inflicted)

# patient 1: ImagePullBackOff
kubectl -n school run sick1 --image=nginxdemos/hello:no-such-tag
kubectl -n school describe pod sick1 | tail -5        # Events tell you instantly

# patient 2: CrashLoopBackOff
kubectl -n school run sick2 --image=busybox -- sh -c "echo I die now; exit 1"
kubectl -n school logs sick2 --previous 2>/dev/null || kubectl -n school logs sick2

# patient 3: Pending (impossible request)
kubectl -n school run sick3 --image=nginx --overrides='{"spec":{"containers":[{"name":"sick3","image":"nginx","resources":{"requests":{"cpu":"100"}}}]}}'
kubectl -n school describe pod sick3 | grep -A3 Events

# discharge everyone:
kubectl -n school delete pod sick1 sick2 sick3

🔧 kubectl for this lesson

kubectl describe · kubectl get events · kubectl logs --previous · kubectl exec

🔗 Builds on: 🪪 AWS L05 (ImagePullBackOff is usually an ECR permission)

✅ Verify — what you should see

each sick pod in the infirmary gets a diagnosis from describe → events → logs (--previous) → exec, without peeking.

🧹 Clean up

local cluster: kubectl delete -f k8s/ (or the file you applied) — on EKS, anything left running bills by the hour

⚠️ Common mistakes

reading logs before events — events tell you WHY it never started
forgetting --previous on a CrashLoopBackOff

⏭️ Next

Who is even ALLOWED to run these commands — and why your CI robot shouldn't be cluster-admin: RBAC.

git checkout lesson-17-rbac

← Previousmulti az scaling Next →rbac

This page is the lesson's README from the lesson-16-debugging branch, shown here so the whole School stays on one site. Code files open on GitHub at the same branch.