🖼️ See the drawing + lab 🏠 Course home 🌿 Branch on GitHub ✏️ View source

🖼️ आकृती आणि labThe drawing + lab पूर्ण पानावर उघडा ↗Open full page ↗

🕸️ धडा 06 — Dependency graph: gates आधी खोल्या

📍 तुम्ही इथे आहात: धडा 06 / 12 · मागे: lesson-05-state · पुढे: lesson-07-modules

📦 या ब्रँचमध्ये काय आहे

धडे 01–05, आणि कंत्राटदार क्रम कसा ठरवतो ते. प्रत्येक reference (gate च्या आतील school_room.lab.id) एक edge आहे: gate आधी lab अस्तित्वात हवी. depends_on आराखड्याला न दिसणारी edge जोडतो. ज्यांच्यामध्ये कोणताही मार्ग नाही अशा गोष्टी एका मर्यादेपर्यंत एकाच वेळी बांधल्या जातात (-parallelism, default ने 10). Destroy graph उलटा चालवतो. आणि loop — A ला B हवा, B ला A हवा — नाकारला जातो. iac/demo.py मधील graph(); iac/engine.py मधील _inst_deps(), graph() आणि apply().

🧒 5 वर्षांच्या मुलाला समजावल्यासारखे

आराखडा सांगतो: एक lab, एक library, lab मध्ये जाणारे एक मुख्य gate, library मध्ये जाणारे एक मागचे gate, आणि lab मध्ये 4 lockers.

अजून न बांधलेल्या खोलीला gate लावता येत नाही. 🚪 म्हणून कंत्राटदार बाण काढतो: lab → मुख्य gate, lab → प्रत्येक locker, library → मागचे gate. आणि आणखी एक बाण जो आराखड्याला स्वतःहून दिसत नाही: शाळा म्हणते "मुख्य gate तयार झाल्यानंतरच मागचे gate उघडा" — म्हणून ऐश्वर्या ते depends_on ने लिहून ठेवते.

कंत्राटदाराकडे 10 बांधकाम करणारे आहेत. 👷👷👷

पायरी 1: lab आणि library — आधी काहीच यायला नको. दोन्ही एकदम.
पायरी 2: मुख्य gate आणि चारही lockers — lab तयार आहे. 5 एकदम.
पायरी 3: मागचे gate — ते मुख्य gate ची वाट पाहत होते.

आठ नाही, तीन पायऱ्या. Campus पाडायचा असेल तर उलटे जा: आधी gates आणि lockers, शेवटी खोल्या.

🗺️ आकृती

flowchart LR
    lab["school_room.lab"] --> gm["school_gate.main"]
    lab --> l["school_locker.row[0..3]"]
    lib["school_room.library"] --> gb["school_gate.back"]
    gm -.->|"depends_on"| gb
    s1["step 1: 2 rooms"] --- s2["step 2: 1 gate + 4 lockers"] --- s3["step 3: back gate"]

🗺️ रेखाटलेली आवृत्ती + एक lab: https://school-edh.pages.dev/terraform/lesson-diagrams.html#l06

❓ काय

Dependency graph — एक directed graph: nodes म्हणजे resources, A → B edge चा अर्थ "B आधी A पूर्ण व्हायला हवा". Terraform तो config आणि state वरून बांधते.
Implicit dependency — reference मुळे तयार होते. room_id = school_room.lab.id मुळे gate lab वर अवलंबून राहते. बहुतेक क्रम असाच ठरतो, आणि हाच पसंतीचा मार्ग आहे.
Explicit dependency — depends_on = [school_gate.main], references मधून न दिसणाऱ्या क्रमासाठी (service वापरायला सुरुवात करण्याआधी अस्तित्वात हवी असलेली permission). क्वचितच वापरा: त्यामुळे plans अधिक सावध (conservative) होतात.
Directed acyclic graph (DAG) — graph मध्ये cycles नसायलाच हवेत. Terraform Error: Cycle: … दाखवते आणि loop मधील nodes ची नावे सांगते.
Parallel walk — ज्यांच्या dependencies पूर्ण झाल्या आहेत अशा प्रत्येक node वर Terraform काम करते, एका वेळी -parallelism=N पर्यंत (default 10).
Destroy चा क्रम — उलटा: एखाद्या node वर अवलंबून असलेले सगळे गेल्यानंतरच तो destroy होतो.

🤔 का

कारण क्रमामुळेच संपूर्ण system साठी एक apply सुरक्षित ठरतो: servers आधी network, service आधी ती वापरत असलेला role. Parallelism मुळे ते जलद होते. आणि cycle check कधीच बांधता न येणारे design थांबवतो — Terraform अर्धवट fail होण्याऐवजी plan च्या वेळीच तुम्हाला सांगते.

🔧 कसे (या repo मध्ये)

iac/engine.py मधील expand() प्रत्येक block चे references आणि depends_on गोळा करते. _inst_deps() त्यांचे instance-level edges मध्ये रूपांतर करते (school_room.lab चा reference त्याच्या सर्व instances ना लागू होतो). graph() म्हणजे Kahn's algorithm: ते वारंवार ज्यांच्या dependencies पूर्ण झाल्या आहेत असे सर्व nodes घेते; एकही तयार नसेल पण nodes उरले असतील तर ते loop शोधते आणि CycleError raise करते. apply(..., parallelism=N) प्रत्येक step ला N पर्यंत तयार operations चालवते — प्रत्येक operation एक step घेते असे शिकवण्यासाठीचे model. Deletes त्यांच्यावर अवलंबून असलेल्या resources ची वाट पाहतात.

🧪 करून पाहा

python3 iac/demo.py graph
python3 - <<'EOF'
import sys; sys.path.insert(0, "iac")
from cloud import FakeCloud
from engine import plan, apply
from demo import GRAPH
for par in (1, 2, 3, 4, 5, 10):
    _, steps, _ = apply(plan(GRAPH, None, FakeCloud()), FakeCloud(), parallelism=par)
    print(f"parallelism {par:>2} → {len(steps)} steps {[len(s) for s in steps]}")
flat = {"resource": {"school_room": {"lab": {"name": "lab"}},
        "school_locker": {"row": {"count": 6, "room_id": "${school_room.lab.id}", "label": "${count.index}"}}}}
_, steps, _ = apply(plan(flat, None, FakeCloud()), FakeCloud())
print("1 room + 6 lockers →", [len(s) for s in steps])
EOF

✅ तपासा — तुम्हाला काय दिसायला हवे

graph असे print करते:

── parallelism 10: 3 steps for 8 resources → [2, 5, 1] per step
   step 1: school_room.lab, school_room.library
   step 2: school_gate.main, school_locker.row[0], school_locker.row[1], school_locker.row[2], school_locker.row[3]
   step 3: school_gate.back
── destroy runs the graph backwards:
   step 1: school_locker.row[0], school_locker.row[1], school_locker.row[2], school_locker.row[3], school_gate.back
   step 2: school_room.library, school_gate.main
   step 3: school_room.lab
── a gate that needs a locker that needs the gate → Error: Cycle: school_gate.g, school_locker.l

तुमचा snippet असे print करतो:

parallelism  1 → 8 steps [1, 1, 1, 1, 1, 1, 1, 1]
parallelism  2 → 4 steps [2, 2, 2, 2]
parallelism  3 → 3 steps [2, 3, 3]
parallelism  4 → 3 steps [2, 4, 2]
parallelism  5 → 3 steps [2, 5, 1]
parallelism 10 → 3 steps [2, 5, 1]
1 room + 6 lockers → [1, 6]

🏁 तुम्ही आत्ताच काय सिद्ध केले

तुम्ही blocks कोणत्या क्रमाने लिहिले त्यावरून नाही, graph क्रम ठरवतो. पुरेसे बांधकाम करणारे असताना campus ला 3 steps लागल्या — सर्वात लांब साखळीची लांबी (lab → मुख्य gate → मागचे gate) — आणि 5 पेक्षा जास्त बांधकाम करणाऱ्यांचा उपयोग झाला नाही, कारण step 2 मध्ये कधीच 5 पेक्षा जास्त गोष्टी तयार नसतात. 1 बांधकाम करणाऱ्यासह 8 steps लागल्या. Destroy ने तोच graph उलटा चालवला, आणि काहीही बांधण्याआधीच loop नाकारला गेला.

⚠️ नेहमीच्या चुका

"सुरक्षित राहावे म्हणून" सगळीकडे depends_on जोडणे — plans हळू होतात, आणि मूल्ये जास्त काळ unknown राहतात
reference ऐवजी id किंवा नाव hard-code करून खरी dependency लपवणे
एकमेकांचा id लागणाऱ्या दोन resources मधील cycle — एकाचे दोन resources मध्ये विभाजन करा (उदाहरणार्थ security group आणि वेगळा rule)
-parallelism खूप जास्त ठेवणे आणि cloud API च्या rate limits ला धडकणे
वेगवेगळ्या state files मधील resources चा क्रम लागेल अशी अपेक्षा ठेवणे — graph state च्या सीमेवर थांबतो

🏭 प्रत्यक्ष वापरात

खऱ्या account वर — खरा graph काढा आणि walk नियंत्रित करा:

terraform graph | dot -Tsvg > graph.svg     # needs Graphviz; open graph.svg in a browser
terraform apply -parallelism=5              # fewer concurrent API calls

References मधून न दिसणारी explicit dependency:

resource "aws_iam_role_policy" "read_timetables" {
  role   = aws_iam_role.app.id
  policy = data.aws_iam_policy_document.read.json
}

resource "aws_instance" "app" {
  ami                  = var.ami
  instance_type        = "t3.small"
  iam_instance_profile = aws_iam_instance_profile.app.name
  depends_on           = [aws_iam_role_policy.read_timetables]   # the app reads S3 at boot
}

क्लासिक cycle — एकमेकांना परवानगी देणारे दोन security groups — rules वेगळ्या aws_vpc_security_group_ingress_rule resources मध्ये हलवून तोडला जातो, म्हणजे प्रत्येक group आधी तयार होऊ शकतो आणि rules नंतर.

🏭 प्रत्यक्ष वापरात हे का महत्त्वाचे: apply अर्ध्यावर fail झाला की काय बांधले गेले आणि काय नाही ते graph सांगतो. Error वाचा, दुरुस्त करा, आणि पुन्हा plan करा — Terraform सुरुवातीपासून नाही, state पासून पुढे चालू ठेवते.

⏭️ पुढे

तीच खोली + gate + lockers पुन्हा पुन्हा येत राहतात. पुढे: ते एकदाच module म्हणून लिहा आणि प्रत्येक wing साठी बांधा.

git checkout lesson-07-modules

🕸️ Lesson 06 — The dependency graph: rooms before gates

📍 You are here: Lesson 06 of 12 · Previous: lesson-05-state · Next: lesson-07-modules

📦 What's in this branch

Lessons 01–05, plus the way the contractor decides the order. Every reference (school_room.lab.id inside the gate) is an edge: the lab must exist before the gate. depends_on adds an edge the plans cannot see. Things with no path between them are built at the same time, up to a limit (-parallelism, 10 by default). Destroy runs the graph backwards. And a loop — A needs B, B needs A — is refused. graph() in iac/demo.py; _inst_deps(), graph() and apply() in iac/engine.py.

🧒 Explain like I'm 5

The plans say: a lab, a library, a main gate into the lab, a back gate into the library, and 4 lockers in the lab.

You can't hang a gate on a room that isn't built yet. 🚪 So the contractor draws arrows: lab → main gate, lab → each locker, library → back gate. And one more arrow that the plans can't see by themselves: the school says "open the back gate only after the main gate exists" — so Aishwarya writes it down with depends_on.

The contractor has 10 builders. 👷👷👷

Step 1: the lab and the library — nothing needs to come first. Both at once.
Step 2: the main gate and all 4 lockers — the lab is ready. 5 at once.
Step 3: the back gate — it waited for the main gate.

Three steps, not eight. To take the campus down, go backwards: gates and lockers first, rooms last.

🗺️ Diagram

flowchart LR
    lab["school_room.lab"] --> gm["school_gate.main"]
    lab --> l["school_locker.row[0..3]"]
    lib["school_room.library"] --> gb["school_gate.back"]
    gm -.->|"depends_on"| gb
    s1["step 1: 2 rooms"] --- s2["step 2: 1 gate + 4 lockers"] --- s3["step 3: back gate"]

🗺️ Drawn version + a lab: https://school-edh.pages.dev/terraform/lesson-diagrams.html#l06

❓ What

Dependency graph — a directed graph: nodes are resources, an edge A → B means "A must be done before B". Terraform builds it from the config and the state.
Implicit dependency — created by a reference. room_id = school_room.lab.id makes the gate depend on the lab. This is how most ordering happens, and it is the preferred way.
Explicit dependency — depends_on = [school_gate.main], for an ordering the references do not show (a permission that must exist before a service starts using it). Use it rarely: it makes plans more conservative.
Directed acyclic graph (DAG) — the graph must have no cycles. Terraform reports Error: Cycle: … and names the nodes in the loop.
Parallel walk — Terraform works on every node whose dependencies are done, up to -parallelism=N at a time (default 10).
Destroy order — the reverse: a node is destroyed only after everything that depends on it is gone.

🤔 Why

Because the order is what makes one apply safe for a whole system: the network before the servers, the role before the service that uses it. Parallelism is what makes it fast. And the cycle check is what stops a design that can never be built — Terraform tells you at plan time instead of failing halfway.

🔧 How (in this repo)

expand() in iac/engine.py collects each block's references and depends_on. _inst_deps() turns them into instance-level edges (a reference to school_room.lab covers all its instances). graph() is Kahn's algorithm: it repeatedly takes every node whose dependencies are done; if none is ready but nodes are left, it finds the loop and raises CycleError. apply(..., parallelism=N) runs up to N ready operations per step — a teaching model where every operation takes one step. Deletes wait for the resources that depended on them.

🧪 Try it

python3 iac/demo.py graph
python3 - <<'EOF'
import sys; sys.path.insert(0, "iac")
from cloud import FakeCloud
from engine import plan, apply
from demo import GRAPH
for par in (1, 2, 3, 4, 5, 10):
    _, steps, _ = apply(plan(GRAPH, None, FakeCloud()), FakeCloud(), parallelism=par)
    print(f"parallelism {par:>2} → {len(steps)} steps {[len(s) for s in steps]}")
flat = {"resource": {"school_room": {"lab": {"name": "lab"}},
        "school_locker": {"row": {"count": 6, "room_id": "${school_room.lab.id}", "label": "${count.index}"}}}}
_, steps, _ = apply(plan(flat, None, FakeCloud()), FakeCloud())
print("1 room + 6 lockers →", [len(s) for s in steps])
EOF

✅ Verify — what you should see

graph prints:

── parallelism 10: 3 steps for 8 resources → [2, 5, 1] per step
   step 1: school_room.lab, school_room.library
   step 2: school_gate.main, school_locker.row[0], school_locker.row[1], school_locker.row[2], school_locker.row[3]
   step 3: school_gate.back
── destroy runs the graph backwards:
   step 1: school_locker.row[0], school_locker.row[1], school_locker.row[2], school_locker.row[3], school_gate.back
   step 2: school_room.library, school_gate.main
   step 3: school_room.lab
── a gate that needs a locker that needs the gate → Error: Cycle: school_gate.g, school_locker.l

Your snippet prints:

parallelism  1 → 8 steps [1, 1, 1, 1, 1, 1, 1, 1]
parallelism  2 → 4 steps [2, 2, 2, 2]
parallelism  3 → 3 steps [2, 3, 3]
parallelism  4 → 3 steps [2, 4, 2]
parallelism  5 → 3 steps [2, 5, 1]
parallelism 10 → 3 steps [2, 5, 1]
1 room + 6 lockers → [1, 6]

🏁 What you just proved

The graph, not the order you wrote the blocks in, decides the order. With enough builders the campus took 3 steps — the length of the longest chain (lab → main gate → back gate) — and more than 5 builders did not help, because step 2 never has more than 5 things ready. With 1 builder it took 8 steps. Destroy ran the same graph backwards, and the loop was refused before anything was built.

⚠️ Common mistakes

adding depends_on everywhere "to be safe" — slower plans, and values become unknown for longer
hiding a real dependency by hard-coding an id or a name instead of a reference
a cycle between two resources that each need the other's id — split one into two resources (for example a security group and a separate rule)
setting -parallelism very high and hitting the cloud API's rate limits
expecting resources in different state files to be ordered — the graph stops at the state boundary

🏭 In production

On a real account — draw the real graph and control the walk:

terraform graph | dot -Tsvg > graph.svg     # needs Graphviz; open graph.svg in a browser
terraform apply -parallelism=5              # fewer concurrent API calls

An explicit dependency that references cannot show:

resource "aws_iam_role_policy" "read_timetables" {
  role   = aws_iam_role.app.id
  policy = data.aws_iam_policy_document.read.json
}

resource "aws_instance" "app" {
  ami                  = var.ami
  instance_type        = "t3.small"
  iam_instance_profile = aws_iam_instance_profile.app.name
  depends_on           = [aws_iam_role_policy.read_timetables]   # the app reads S3 at boot
}

The classic cycle — two security groups that allow each other — is broken by moving the rules into separate aws_vpc_security_group_ingress_rule resources, so each group can be created first and the rules afterwards.

🏭 Why this matters in production: when an apply fails halfway, the graph tells you what was built and what was not. Read the error, fix it, and plan again — Terraform continues from the state, not from the start.

⏭️ Next

The same room + gate + lockers keeps coming back. Next: write it once as a module and build it for every wing.

git checkout lesson-07-modules

← Previousstate Next →modules

This page is the lesson's README from the lesson-06-dependency-graph branch, shown here so the whole School stays on one site. Code files open on GitHub at the same branch.