🖼️ See the drawing + lab 🏠 Course home 🌿 Branch on GitHub ✏️ View source

🖼️ आकृती आणि labThe drawing + lab पूर्ण पानावर उघडा ↗Open full page ↗

🚚 धडा 07 — CI/CD चा बांधलेला रस्ता: production कडे जाणारा एकच सामायिक रस्ता

📍 तुम्ही इथे आहात: 12 पैकी धडा 07 · मागे: lesson-06-preview-environments · पुढे: lesson-08-secrets-config

📦 या ब्रँचमध्ये काय आहे

धडे 01–06, आणि पुन्हा वापरता येणारे, version असलेले pipeline templates. प्रत्येक service ने build-test-scan-deploy पायऱ्यांची स्वतःची copy बाळगण्याऐवजी, ती कार्यालयाचा template एका version वर बोलावते. Version reference कसा सोडवला जातो — हलणारा major tag (@v2), नेमके version (@v2.0.0) किंवा paste केलेली copy — यावरून ठरते की दुरुस्ती service पर्यंत आपोआप पोहोचते की नाही. idp/selfserve.py मधले PIPELINES, resolve_ref() आणि pipeline(), idp/demo.py मधले pipelines().

🧒 5 वर्षांच्या मुलाला समजावल्यासारखे

विद्यार्थी येण्याआधी प्रत्येक वर्गखोलीला सकाळचा तोच दिनक्रम लागतो: कुलूप उघडा, आपत्कालीन दार तपासा, दिवे लावा, खिडक्या उघडा. 🚪

सुरुवातीला प्रत्येक शिक्षिकेने आपले दिनक्रम-कार्ड स्वतः लिहिले. मग सुविधा कार्यालयाने एकच दिनक्रम-कार्ड लिहिले, त्याला क्रमांक दिला (version 2), आणि ते शिक्षक-कक्षात लावले. आता शिक्षिका म्हणतात "मी कार्यालयाचे कार्ड, version 2, पाळते."

कार्यालय एक पायरी जोडते — "धुराचा alarm तपासा" (version 2.1) — तेव्हा काहीतरी मनोरंजक घडते:

कतरिना म्हणाली होती "नवीनतम version 2 कार्ड". तिला काहीही न करता दुसऱ्या दिवशी सकाळी धुराच्या alarm ची पायरी मिळते. ✅
ऐश्वर्या म्हणाली होती "नेमके कार्ड 2.0". ती स्वतः बदलायचे ठरवेपर्यंत जुनेच कार्ड ठेवते. ⏸️
ग्रंथालय अजूनही कार्ड 1 पाळते. त्यात आपत्कालीन दाराची तपासणीच नाही. ⚠️
fees कार्यालयाने वर्षांपूर्वी कार्ड 1 ची photocopy काढली आणि ती photocopy पाळते. कार्यालयाने काहीही बदलले तरी ते तिच्यापर्यंत कधीच पोहोचणार नाही. 📄

🗺️ आकृती

flowchart LR
    subgraph office["🏢 the office's pipeline releases"]
      v1["v1.0.0<br/>4 steps · 14 min"]
      v2["v2.0.0<br/>+ SBOM + image scan<br/>6 steps · 17 min"]
      v21["v2.1.0<br/>+ dependency cache<br/>7 steps · 15 min"]
    end
    er["exam-results @v2"] --> v21
    tt["timetable @v2.0.0"] --> v2
    ls["library-search @v1"] --> v1
    fe["fees: pasted copy of v1"] -.-> cp["📄 v1.0.0 copy<br/>never changes"]

🗺️ काढलेली आकृती + एक lab: https://school-edh.pages.dev/platform-engineering/lesson-diagrams.html#l07

❓ काय

Paved road pipeline — platform team बनवते आणि सांभाळते अशी CI/CD pipeline, संस्थेच्या defaults सह: tests, image build, SBOM (software bill of materials, image च्या आत काय आहे त्याची यादी), माहीत असलेल्या vulnerabilities साठी image scan, deploy.
Reusable pipeline / template — pipeline एकाच ठिकाणी राहते; services तिचा संदर्भ देतात. GitHub Actions यांना reusable workflows म्हणते (on: workflow_call, बोलावले जाते uses: org/repo/.github/workflows/file.yml@ref ने); GitLab CI include: (आणि CI/CD components) वापरते; Jenkins मध्ये shared libraries असतात.
Semantic versioning — MAJOR.MINOR.PATCH. Minor किंवा patch release ने बोलावणाऱ्यांना मोडू नये; major release मोडू शकते.
Moving major tag — v2 हा एक Git tag आहे जो प्रकाशक नवीनतम v2.x.y कडे हलवतो. हा एक संकेत आहे (GitHub Actions साठी सामान्य), Git त्याची सक्ती करत नाही. बोलावणाऱ्यांना दुरुस्त्या आपोआप मिळतात — आणि नव्या release मधली कोणतीही चूकसुद्धा.
Exact pin — @v2.0.0 किंवा, आणखी सुरक्षित, पूर्ण commit SHA: बोलावणारा reference बदलेपर्यंत नेमका तोच code चालवतो. GitHub चे security मार्गदर्शन third-party actions पूर्ण commit SHA ला pin करण्याची शिफारस करते, कारण tag हलवता येतो.
Copy-paste drift — paste केलेली copy म्हणजे परतीचा दुवा नसलेला fork. तिला security च्या दुरुस्त्यांसह कोणत्याच दुरुस्त्या मिळत नाहीत.

🤔 का

कारण एकमेकांकडून copy केलेल्या शंभर pipelines शंभर थोड्या वेगळ्या pipelines बनतात. नवा नियम आला ("प्रत्येक image scan झालीच पाहिजे") की कोणाला तरी शंभर files बदलाव्या लागतात. सामायिक template असेल तर कार्यालय बदल एकदाच करते, तो release करते, आणि template पाळणाऱ्या प्रत्येक service ला तो मिळतो — moving tags साठी लगेच, exact pins साठी पुढच्या bump ला. Version reference म्हणजे केव्हा यावरचे team चे नियंत्रण.

🔧 कसे (या repo मध्ये)

idp/selfserve.py मधल्या PIPELINES मध्ये तीन releases (step, minutes) च्या यादी म्हणून आहेत — शिकवण्यासाठीचे आकडे. resolve_ref(ref) reference ला release मध्ये बदलते: exact version स्वतःलाच परत देते; v2 सारखा prefix जुळणारी नवीनतम release परत देतो (आकडे म्हणून तुलना करून, म्हणजे v2.10.0 हे v2.9.0 वर मात करेल); copy paste केलेली v1.0.0 परत देते. pipeline(ref) release, तिच्या पायऱ्या आणि तिची एकूण मिनिटे परत देते. idp/demo.py मधले pipelines() चार services आणि प्रत्येक खरोखर काय चालवते ते दाखवते.

🧪 करून पाहा

python3 idp/demo.py pipelines
python3 - <<'EOF'
import sys; sys.path.insert(0, "idp"); import selfserve
from selfserve import PIPELINES, resolve_ref, pipeline
PIPELINES["v2.2.0"] = PIPELINES["v2.1.0"] + [("smoke test", 2)]      # the office ships a new minor release
PIPELINES["v3.0.0"] = [("checkout", 1), ("test", 3), ("build image", 4), ("sign image", 1), ("deploy", 3)]
for ref in ("v2", "v2.0.0", "v3", "v1", "copy"):
    version, steps, minutes = pipeline(ref)
    print(f"@{ref:<7} → {version:<21} {minutes:>2} min · last step: {steps[-1]}")
print("unknown ref:", resolve_ref("v9"))
EOF

✅ तपासा — तुम्हाला काय दिसायला हवे

pipelines हे छापते:

   exam-results    uses @v2      → runs v2.1.0                7 steps, 15 min  (scans images)
   timetable       uses @v2.0.0  → runs v2.0.0                6 steps, 17 min  (scans images)
   library-search  uses @v1      → runs v1.0.0                4 steps, 14 min
   fees            uses @copy    → runs v1.0.0 (pasted copy)  4 steps, 14 min

तुमचा snippet हे छापतो:

@v2      → v2.2.0                17 min · last step: smoke test
@v2.0.0  → v2.0.0                17 min · last step: deploy
@v3      → v3.0.0                12 min · last step: deploy
@v1      → v1.0.0                14 min · last step: deploy
@copy    → v1.0.0 (pasted copy)  14 min · last step: deploy

🏁 तुम्ही आत्ताच काय सिद्ध केले

एका नव्या release (v2.2.0) ने @v2 काय चालवते ते बदलले — एक smoke test आली — पण @v2.0.0 जागीच राहिले. Major release v3.0.0 कोणी v3 मागेपर्यंत कोणापर्यंतच पोहोचली नाही; major version चा हाच तर उद्देश आहे. दोन services (library-search, fees) कोणताच image scan चालवत नाहीत. त्यातली एक एकच reference @v2 करून ते दुरुस्त करू शकते; दुसरीला आधी आपली copy हटवावी लागेल. Demo मध्ये, v2.1.0 च्या cache मुळे जास्त पायऱ्या असूनही pipeline v2.0.0 पेक्षा 2 मिनिटे जलद झाली — platform release फक्त नियम जोडत नाही, वेळही परत देऊ शकते.

⚠️ नेहमीच्या चुका

template ची "फक्त या एकदाच" copy करणे — त्या copy ला दुरुस्त्यांबद्दल पुन्हा कधीच कळत नाही
बोलावणाऱ्यांना मोडणाऱ्या release कडे major tag हलवणे (त्यासाठी नवे major हवे होते)
exact versions pin करणे आणि कधीच bump न करणे — आज सुरक्षित, उद्या वर्षांनी मागे
इतके inputs असलेला template की प्रत्येक service चे बोलावणे तरीही वेगळेच असते
changelog नाही, त्यामुळे bump ने काय बदलेल ते teams ना कळत नाही

🏭 प्रत्यक्ष वापरात

खऱ्या repo वर GitHub Actions. कार्यालयाचा reusable workflow, school/paved-road मध्ये:

# .github/workflows/python.yml
on:
  workflow_call:
    inputs:
      image-name: {required: true, type: string}
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pip install -r requirements.txt && pytest
      - run: docker build -t "${{ inputs.image-name }}:${{ github.sha }}" .

Service तो बोलावते — moving major tag ने, किंवा exact pin साठी पूर्ण commit SHA ने:

jobs:
  ci:
    uses: school/paved-road/.github/workflows/python.yml@v2
    with:
      image-name: exam-results

GitLab CI हेच include ने करते:

include:
  - project: school/paved-road
    ref: v2.1.0
    file: /templates/python.yml

Dependabot किंवा Renovate हे references bump करणारे pull requests उघडू शकतात, म्हणजे exact pins गुपचूप मागे पडत नाहीत.

🏭 प्रत्यक्ष वापरात हे का महत्त्वाचे: प्रत्येक service कोणते pipeline version चालवते याची यादी करा (धडा 10 ते scorecard check मध्ये बदलतो). Copies आणि सर्वात जुने pins इथेच पुढची security finding येऊन पडेल.

⏭️ पुढे

Pipelines आणि services ना passwords आणि settings लागतात. पुढे: कार्यालयाचे किल्ल्यांचे कपाट — service म्हणून secrets आणि config.

git checkout lesson-08-secrets-config

🚚 Lesson 07 — The paved CI/CD road: one shared road to production

📍 You are here: Lesson 07 of 12 · Previous: lesson-06-preview-environments · Next: lesson-08-secrets-config

📦 What's in this branch

Lessons 01–06, plus reusable, versioned pipeline templates. Instead of each service carrying its own copy of the build-test-scan-deploy steps, it calls the office's template at a version. How a version reference is resolved — a moving major tag (@v2), an exact version (@v2.0.0) or a pasted copy — decides whether a fix reaches the service by itself. PIPELINES, resolve_ref() and pipeline() in idp/selfserve.py, pipelines() in idp/demo.py.

🧒 Explain like I'm 5

Every classroom needs the same morning routine before students come in: unlock, check the fire exit, switch on the lights, open the windows. 🚪

At first each teacher wrote her own routine card. Then the facilities office wrote one routine card, numbered it (version 2), and pinned it in the staff room. Teachers now say "I follow the office's card, version 2."

When the office adds a step — "check the smoke alarm" (version 2.1) — something interesting happens:

Katrina said "the newest version 2 card". She gets the smoke-alarm step next morning, without doing anything. ✅
Aishwarya said "exactly card 2.0". She keeps the old card until she decides to switch. ⏸️
The library still follows card 1. It has no fire-exit check at all. ⚠️
The fees office photocopied card 1 years ago and follows the photocopy. Nothing the office changes will ever reach it. 📄

🗺️ Diagram

flowchart LR
    subgraph office["🏢 the office's pipeline releases"]
      v1["v1.0.0<br/>4 steps · 14 min"]
      v2["v2.0.0<br/>+ SBOM + image scan<br/>6 steps · 17 min"]
      v21["v2.1.0<br/>+ dependency cache<br/>7 steps · 15 min"]
    end
    er["exam-results @v2"] --> v21
    tt["timetable @v2.0.0"] --> v2
    ls["library-search @v1"] --> v1
    fe["fees: pasted copy of v1"] -.-> cp["📄 v1.0.0 copy<br/>never changes"]

🗺️ Drawn version + a lab: https://school-edh.pages.dev/platform-engineering/lesson-diagrams.html#l07

❓ What

Paved road pipeline — a CI/CD pipeline the platform team builds and maintains, with the organisation's defaults: tests, image build, SBOM (software bill of materials, a list of what is inside the image), image scan for known vulnerabilities, deploy.
Reusable pipeline / template — the pipeline lives in one place; services reference it. GitHub Actions calls these reusable workflows (on: workflow_call, called with uses: org/repo/.github/workflows/file.yml@ref); GitLab CI uses include: (and CI/CD components); Jenkins has shared libraries.
Semantic versioning — MAJOR.MINOR.PATCH. A minor or patch release should not break callers; a major release may.
Moving major tag — v2 is a Git tag the publisher moves to the newest v2.x.y. It is a convention (common for GitHub Actions), not something Git enforces. Callers get fixes automatically — and also any mistake in a new release.
Exact pin — @v2.0.0 or, safer still, a full commit SHA: the caller runs exactly that code until it changes the reference. GitHub's security guidance recommends pinning third-party actions to a full commit SHA, because a tag can be moved.
Copy-paste drift — a pasted copy is a fork with no link back. It gets no fixes, including security ones.

🤔 Why

Because a hundred pipelines copied from each other become a hundred slightly different pipelines. When a new rule arrives ("every image must be scanned"), someone must edit a hundred files. With a shared template, the office makes the change once, releases it, and every service that follows the template gets it — immediately for moving tags, at the next bump for exact pins. The version reference is the team's control over when.

🔧 How (in this repo)

PIPELINES in idp/selfserve.py holds three releases as lists of (step, minutes) — teaching numbers. resolve_ref(ref) turns a reference into a release: an exact version returns itself; a prefix like v2 returns the newest matching release (compared as numbers, so v2.10.0 would beat v2.9.0); copy returns the pasted v1.0.0. pipeline(ref) returns the release, its steps and its total minutes. pipelines() in idp/demo.py shows four services and what each really runs.

🧪 Try it

python3 idp/demo.py pipelines
python3 - <<'EOF'
import sys; sys.path.insert(0, "idp"); import selfserve
from selfserve import PIPELINES, resolve_ref, pipeline
PIPELINES["v2.2.0"] = PIPELINES["v2.1.0"] + [("smoke test", 2)]      # the office ships a new minor release
PIPELINES["v3.0.0"] = [("checkout", 1), ("test", 3), ("build image", 4), ("sign image", 1), ("deploy", 3)]
for ref in ("v2", "v2.0.0", "v3", "v1", "copy"):
    version, steps, minutes = pipeline(ref)
    print(f"@{ref:<7} → {version:<21} {minutes:>2} min · last step: {steps[-1]}")
print("unknown ref:", resolve_ref("v9"))
EOF

✅ Verify — what you should see

pipelines prints:

   exam-results    uses @v2      → runs v2.1.0                7 steps, 15 min  (scans images)
   timetable       uses @v2.0.0  → runs v2.0.0                6 steps, 17 min  (scans images)
   library-search  uses @v1      → runs v1.0.0                4 steps, 14 min
   fees            uses @copy    → runs v1.0.0 (pasted copy)  4 steps, 14 min

Your snippet prints:

@v2      → v2.2.0                17 min · last step: smoke test
@v2.0.0  → v2.0.0                17 min · last step: deploy
@v3      → v3.0.0                12 min · last step: deploy
@v1      → v1.0.0                14 min · last step: deploy
@copy    → v1.0.0 (pasted copy)  14 min · last step: deploy

🏁 What you just proved

One new release (v2.2.0) changed what @v2 runs — a smoke test appeared — while @v2.0.0 stayed put. The major release v3.0.0 reached nobody until a caller asks for v3; that is the point of a major version. Two services (library-search, fees) run no image scan. One of them can fix that by changing one reference to @v2; the other must first delete its copy. In the demo, v2.1.0's cache made the pipeline 2 minutes faster than v2.0.0 even with more steps — a platform release can give time back, not only add rules.

⚠️ Common mistakes

copying the template "just this once" — the copy never hears about fixes again
moving a major tag to a release that breaks callers (that needed a new major)
pinning exact versions and never bumping them — safe today, years behind tomorrow
a template with so many inputs that every service's call is different anyway
no changelog, so teams cannot tell what a bump will change

🏭 In production

GitHub Actions, on a real repo. The office's reusable workflow, in school/paved-road:

# .github/workflows/python.yml
on:
  workflow_call:
    inputs:
      image-name: {required: true, type: string}
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pip install -r requirements.txt && pytest
      - run: docker build -t "${{ inputs.image-name }}:${{ github.sha }}" .

A service calls it — a moving major tag, or a full commit SHA for an exact pin:

jobs:
  ci:
    uses: school/paved-road/.github/workflows/python.yml@v2
    with:
      image-name: exam-results

GitLab CI does the same with include:

include:
  - project: school/paved-road
    ref: v2.1.0
    file: /templates/python.yml

Dependabot or Renovate can open pull requests that bump these references, so exact pins do not quietly fall behind.

🏭 Why this matters in production: list which pipeline version every service runs (lesson 10 turns that into a scorecard check). The copies and the oldest pins are where the next security finding will land.

⏭️ Next

Pipelines and services need passwords and settings. Next: the office's key cabinet — secrets and config as a service.

git checkout lesson-08-secrets-config

← Previouspreview environments Next →secrets config

This page is the lesson's README from the lesson-07-paved-pipelines branch, shown here so the whole School stays on one site. Code files open on GitHub at the same branch.