🖼️ See the drawing + lab 🏠 Course home 🌿 Branch on GitHub ✏️ View source

🖼️ आकृती आणि labThe drawing + lab पूर्ण पानावर उघडा ↗Open full page ↗

🔎 धडा 11 — Flow logs & Reachability Analyzer: A ला B पर्यंत का पोहोचता येत नाही?

📍 तुम्ही इथे आहात: 12 पैकी धडा 11 · मागचा: lesson-10-transit-gateway · पुढचा: lesson-12-production-design

📦 या ब्रँचमध्ये काय आहे

धडे 01–10, आणि गुप्तहेराचे काम: vpc/demo.py मधले troubleshoot() तीन flow-log records वाचते, मग Campus.walk() ने web → db :5432 hop by hop चालून पाहते — जसे Reachability Analyzer करतो — आणि नेमका अडथळा सांगते.

🧒 5 वर्षांच्या मुलाला समजावल्यासारखे

एक विद्यार्थिनी म्हणते: "मी परीक्षा कार्यालयाला पत्र पाठवले आणि कुणीच उत्तर दिले नाही!" 📨

काळजीवाहिकेकडे प्रत्येक desk वर एक नोंदवही 📓 असते: पत्र कोणी पाठवले, कोणाला, कोणत्या खोलीत, किती पाने — आणि शेवटी एक शब्द: ACCEPT किंवा REJECT. नोंदवही दाखवते की पत्र परीक्षा कार्यालयात नाकारले गेले.

पण ते नाकारले कोणी? म्हणून काळजीवाहिका पत्र घेऊन मार्गावरून चालत जाते: विद्यार्थिनीची स्वतःची पाहुण्यांची यादी — ठीक. विद्यार्थिनीच्या wing चे दार — ठीक. Corridor ची पाटी — ठीक. परीक्षा wing चे दार — ठीक. परीक्षा कार्यालयाची पाहुण्यांची यादी — विद्यार्थिनीचे नाव त्यात नाही. तोच अडथळा आहे. फक्त ती एक गोष्ट दुरुस्त करा, सगळे नाही.

🗺️ आकृती

flowchart LR
    log["📓 flow log<br/>10.20.0.10 → 10.20.96.15:5432 REJECT"]
    s1["🧍 sg-web out ✓"]
    n1["🚦 acl-open 100 ✓"]
    rt["🪧 rt-public local ✓"]
    n2["🚦 acl-data 110 ✓"]
    s2["⛔ sg-db in<br/>THE BLOCKER"]
    log --> s1 --> n1 --> rt --> n2 --> s2

🗺️ रेखाटलेली आवृत्ती + एक lab: https://school-edh.pages.dev/vpc/lesson-diagrams.html#l11

❓ काय

VPC flow logs — VPC, subnet किंवा एका network interface साठी IP traffic च्या नोंदी, CloudWatch Logs किंवा S3 (किंवा Amazon Data Firehose) कडे पाठवल्या जातात. त्या एका कालखंडात (1 किंवा 10 मिनिटे) एकत्र केल्या जातात, real time नसतात, आणि त्यांत packet चा मजकूर कधीच नसतो. Default format मध्ये हे fields, या क्रमाने असतात: version account-id interface-id srcaddr dstaddr srcport dstport protocol packets bytes start end action log-status.
action — ACCEPT किंवा REJECT: security groups आणि network ACLs नी traffic ला परवानगी दिली की नाही. Protocol 6 म्हणजे TCP, 17 म्हणजे UDP.
NACL ची खूण ओळखणे — inbound ACCEPT आणि त्यानंतर उत्तरासाठी outbound REJECT याचा अर्थ सहसा ephemeral-port rule नसलेला stateless NACL (धडा 06). Security groups असे कधीच करत नाहीत — त्यांना उत्तर लक्षात असते.
ज्याची नोंद होत नाही — उदाहरणार्थ Amazon DNS resolver कडे जाणारे traffic, instance metadata service 169.254.169.254 कडे जाणारे traffic, आणि DHCP.
Reachability Analyzer — तुम्ही source आणि destination (instances, network interfaces, gateways…) आणि एक port देता. तो तुमची configuration वाचतो (कोणतेही packets पाठवत नाही), आणि उत्तर देतो reachable — hop-by-hop मार्गासह — किंवा not reachable — अडवणाऱ्या component सह: एक security group, एक NACL rule, एक गहाळ route. प्रत्येक analysis साठी खर्च (साधारण $0.10 — pricing page तपासा).
"Refused" विरुद्ध "timeout" — timeout network कडे बोट दाखवतो (SG, NACL, route); refusal चा अर्थ packet पोहोचला आणि कुणीच ऐकत नव्हते.

🤔 का

कारण "network बिघडले आहे" हे निदान नाही. Packet थांबवला जाऊ शकतो अशा किमान पाच जागा आहेत, आणि "सुरक्षित राहण्यासाठी" त्या सगळ्या बदलल्या तर अशी भोके पडतात जी तुम्ही कधीच बंद करणार नाही. Flow logs तुम्हाला सांगतात की ते नाकारले गेले; चालून पाहणे सांगते कुठे.

🔧 कसे (या repo मध्ये)

vpc/demo.py मधले troubleshoot() प्रत्येक log ओळ spaces वर तोडते आणि fields 3, 4, 6 आणि 12 (srcaddr, dstaddr, dstport, action) छापते. मग ते campus().walk("web", "10.20.96.15", "tcp", 5432) बोलावते. walk() पहिल्या अपयशी तपासणीवर थांबते, म्हणून शेवटची पायरी हाच अडथळा — vpc/test_vpc.py मधली test L11 ती ⛔ sg-db ने सुरू होते हे तपासते.

🧪 करून पाहा

python3 vpc/demo.py troubleshoot
python3 - <<'EOF'
import sys; sys.path.insert(0, "vpc"); from network import SG
from demo import campus
FIELDS = "version account-id interface-id srcaddr dstaddr srcport dstport protocol packets bytes start end action log-status".split()
line = "2 111122223333 eni-0web 10.20.0.10 10.20.96.15 49152 5432 6 3 180 1790000000 1790000060 REJECT OK"
rec = dict(zip(FIELDS, line.split()))
print({k: rec[k] for k in ("interface-id", "srcaddr", "dstaddr", "dstport", "protocol", "action")})
c = campus()
print("before:", c.walk("web", "10.20.96.15", "tcp", 5432)[1][-1])
c.sgs["sg-db"] = SG("sg-db", [("tcp", (5432, 5432), "10.20.48.0/20"), ("tcp", (5432, 5432), "10.20.0.10/32")])
ok, steps = c.walk("web", "10.20.96.15", "tcp", 5432)
print("after: ", ok, steps[-1])
print("no host:", c.walk("app", "10.20.96.99", "tcp", 5432)[1][-1])
EOF

✅ तपासा — तुम्हाला काय दिसायला हवे

troubleshoot हे छापते:

   10.20.48.25    → 10.20.96.15   :5432  ACCEPT
   10.20.0.10     → 10.20.96.15   :5432  REJECT
   198.51.100.9   → 10.20.0.10    :22    REJECT
── why is web → db :5432 rejected? walk it like Reachability Analyzer:
   🧍 sg-web out ✓
   🚦 acl-open rule 100 ALLOW
   🪧 rt-public: 10.20.0.0/16 → local
   🚦 acl-data rule 110 ALLOW (destination subnet, in)
   ⛔ sg-db in — not on the guest list

तुमचा snippet हे छापतो {'interface-id': 'eni-0web', 'srcaddr': '10.20.0.10', 'dstaddr': '10.20.96.15', 'dstport': '5432', 'protocol': '6', 'action': 'REJECT'}, मग before: ⛔ sg-db in — not on the guest list, after: True 🧍 sg-db in ✓ (एक /32 rule, बाकी काहीच बदलले नाही), आणि no host: ❓ no host at that address.

🏁 तुम्ही आत्ताच काय सिद्ध केले

तुम्ही flow-log ची ओळ field by field वाचू शकता, मार्ग hop by hop चालून पाहू शकता, आणि अडवणारा नेमका एकच component दुरुस्त करू शकता.

⚠️ नेहमीच्या चुका

गरज पडेपर्यंत flow logs नाहीत — तुम्ही ते चालू केल्याच्या क्षणापासूनच नोंद होते
SG 0.0.0.0/0 साठी उघडून REJECT "दुरुस्त" करणे
REJECT चा अर्थ "app बंद आहे" असा घेणे — ते नेहमी SG किंवा NACL असते
flow logs काही मिनिटे उशिरा येतात हे विसरणे — fix चालला नाही असे ठरवण्याआधी थांबा
NACL गुंतलेला असताना फक्त request ची दिशा तपासणे

🏭 प्रत्यक्ष वापरात

प्रत्येक VPC वर flow logs, S3 कडे (मोठ्या प्रमाणासाठी स्वस्त) किंवा CloudWatch Logs कडे (query करायला जलद), हा एक सामान्य security baseline आहे. Ticket मध्ये "A ला B पर्यंत पोहोचता येत नाही" असे आले की Reachability Analyzer हे पहिले tool असते.

खऱ्या account वर (AWS CLI आणि credentials लागतात; IDs उदाहरणादाखल आहेत):

aws ec2 create-flow-logs --resource-type VPC --resource-ids vpc-0123456789abcdef0 \
    --traffic-type ALL --log-destination-type s3 --log-destination arn:aws:s3:::school-flow-logs
aws ec2 create-network-insights-path --source i-0web0000000000000 --destination i-0db00000000000000 \
    --protocol tcp --destination-port 5432
aws ec2 start-network-insights-analysis --network-insights-path-id nip-0123456789abcdef0
aws ec2 describe-network-insights-analyses --network-insights-analysis-ids nia-0123456789abcdef0 \
    --query 'NetworkInsightsAnalyses[0].[NetworkPathFound,Explanations]'

⏭️ पुढे

तुम्ही campus बांधू शकता, जोडू शकता आणि debug करू शकता. शेवटची पायरी: टिकणारा campus बांधा. Production VPC design.

git checkout lesson-12-production-design

🔎 Lesson 11 — Flow logs & Reachability Analyzer: why can't A reach B?

📍 You are here: Lesson 11 of 12 · Previous: lesson-10-transit-gateway · Next: lesson-12-production-design

📦 What's in this branch

Lessons 01–10, plus the detective work: troubleshoot() in vpc/demo.py reads three flow-log records, then walks web → db :5432 hop by hop with Campus.walk() — the way Reachability Analyzer does — and names the exact blocker.

🧒 Explain like I'm 5

A pupil says: "I sent a letter to the exam office and nobody answered!" 📨

The caretaker has a logbook 📓 at every desk: who sent a letter, to whom, which room, how many pages — and one word at the end: ACCEPT or REJECT. The logbook shows the letter was rejected at the exam office.

But who rejected it? So the caretaker walks the path with the letter: the pupil's own guest list — fine. The pupil's wing door — fine. The corridor sign — fine. The exam wing's door — fine. The exam office's guest list — the pupil is not on it. That is the blocker. Fix that one thing, not everything.

🗺️ Diagram

flowchart LR
    log["📓 flow log<br/>10.20.0.10 → 10.20.96.15:5432 REJECT"]
    s1["🧍 sg-web out ✓"]
    n1["🚦 acl-open 100 ✓"]
    rt["🪧 rt-public local ✓"]
    n2["🚦 acl-data 110 ✓"]
    s2["⛔ sg-db in<br/>THE BLOCKER"]
    log --> s1 --> n1 --> rt --> n2 --> s2

🗺️ Drawn version + a lab: https://school-edh.pages.dev/vpc/lesson-diagrams.html#l11

❓ What

VPC flow logs — records of IP traffic for a VPC, a subnet or one network interface, delivered to CloudWatch Logs or S3 (or Amazon Data Firehose). They are aggregated over a window (1 or 10 minutes), not real time, and they never hold packet contents. The default format has these fields, in order: version account-id interface-id srcaddr dstaddr srcport dstport protocol packets bytes start end action log-status.
action — ACCEPT or REJECT: whether security groups and network ACLs allowed the traffic. Protocol 6 is TCP, 17 is UDP.
Reading the NACL signature — an inbound ACCEPT followed by an outbound REJECT for the reply usually means a stateless NACL without the ephemeral-port rule (lesson 06). Security groups never do that — they remember the reply.
Not logged — for example traffic to the Amazon DNS resolver, to the instance metadata service 169.254.169.254, and DHCP.
Reachability Analyzer — you give a source and a destination (instances, network interfaces, gateways…) and a port. It reads your configuration (it sends no packets), and answers reachable — with the hop-by-hop path — or not reachable — with the component that blocks it: a security group, a NACL rule, a missing route. Charged per analysis (approximately $0.10 — check the pricing page).
"Refused" vs "timeout" — a timeout points at the network (SG, NACL, route); a refusal means the packet arrived and nothing listened.

🤔 Why

Because "the network is broken" is not a diagnosis. There are at least five places a packet can be stopped, and changing all of them "to be safe" opens holes you will never close. Flow logs tell you that it was rejected; the walk tells you where.

🔧 How (in this repo)

troubleshoot() in vpc/demo.py splits each log line on spaces and prints fields 3, 4, 6 and 12 (srcaddr, dstaddr, dstport, action). Then it calls campus().walk("web", "10.20.96.15", "tcp", 5432). walk() stops at the first failing check, so the last step is the blocker — test L11 in vpc/test_vpc.py checks it starts with ⛔ sg-db.

🧪 Try it

python3 vpc/demo.py troubleshoot
python3 - <<'EOF'
import sys; sys.path.insert(0, "vpc"); from network import SG
from demo import campus
FIELDS = "version account-id interface-id srcaddr dstaddr srcport dstport protocol packets bytes start end action log-status".split()
line = "2 111122223333 eni-0web 10.20.0.10 10.20.96.15 49152 5432 6 3 180 1790000000 1790000060 REJECT OK"
rec = dict(zip(FIELDS, line.split()))
print({k: rec[k] for k in ("interface-id", "srcaddr", "dstaddr", "dstport", "protocol", "action")})
c = campus()
print("before:", c.walk("web", "10.20.96.15", "tcp", 5432)[1][-1])
c.sgs["sg-db"] = SG("sg-db", [("tcp", (5432, 5432), "10.20.48.0/20"), ("tcp", (5432, 5432), "10.20.0.10/32")])
ok, steps = c.walk("web", "10.20.96.15", "tcp", 5432)
print("after: ", ok, steps[-1])
print("no host:", c.walk("app", "10.20.96.99", "tcp", 5432)[1][-1])
EOF

✅ Verify — what you should see

troubleshoot prints:

   10.20.48.25    → 10.20.96.15   :5432  ACCEPT
   10.20.0.10     → 10.20.96.15   :5432  REJECT
   198.51.100.9   → 10.20.0.10    :22    REJECT
── why is web → db :5432 rejected? walk it like Reachability Analyzer:
   🧍 sg-web out ✓
   🚦 acl-open rule 100 ALLOW
   🪧 rt-public: 10.20.0.0/16 → local
   🚦 acl-data rule 110 ALLOW (destination subnet, in)
   ⛔ sg-db in — not on the guest list

Your snippet prints {'interface-id': 'eni-0web', 'srcaddr': '10.20.0.10', 'dstaddr': '10.20.96.15', 'dstport': '5432', 'protocol': '6', 'action': 'REJECT'}, then before: ⛔ sg-db in — not on the guest list, after: True 🧍 sg-db in ✓ (one /32 rule, nothing else changed), and no host: ❓ no host at that address.

🏁 What you just proved

You can read a flow-log line field by field, walk a path hop by hop, and fix exactly the one component that blocks it.

⚠️ Common mistakes

no flow logs until the day you need them — they only record from the moment you turn them on
"fixing" a REJECT by opening the SG to 0.0.0.0/0
reading REJECT as "the app is down" — it is always an SG or a NACL
forgetting that flow logs lag by minutes — wait before deciding the fix did not work
checking only the request direction when a NACL is involved

🏭 In production

Flow logs on every VPC, to S3 (cheaper for volume) or CloudWatch Logs (quicker to query), are a standard security baseline. Reachability Analyzer is the first tool when a ticket says "A cannot reach B".

On a real account (needs the AWS CLI and credentials; IDs are examples):

aws ec2 create-flow-logs --resource-type VPC --resource-ids vpc-0123456789abcdef0 \
    --traffic-type ALL --log-destination-type s3 --log-destination arn:aws:s3:::school-flow-logs
aws ec2 create-network-insights-path --source i-0web0000000000000 --destination i-0db00000000000000 \
    --protocol tcp --destination-port 5432
aws ec2 start-network-insights-analysis --network-insights-path-id nip-0123456789abcdef0
aws ec2 describe-network-insights-analyses --network-insights-analysis-ids nia-0123456789abcdef0 \
    --query 'NetworkInsightsAnalyses[0].[NetworkPathFound,Explanations]'

⏭️ Next

You can build, connect and debug a campus. Last step: build one that lasts. Production VPC design.

git checkout lesson-12-production-design

← Previoustransit gateway Next →production design

This page is the lesson's README from the lesson-11-troubleshooting branch, shown here so the whole School stays on one site. Code files open on GitHub at the same branch.