🖼️ See the drawing + lab 🏠 Course home 🌿 Branch on GitHub ✏️ View source

🖼️ आकृती आणि labThe drawing + lab पूर्ण पानावर उघडा ↗Open full page ↗

📊 धडा 11 — Platform चे मोजमाप: कार्यालयाचा उपयोग होतोय का?

📍 तुम्ही इथे आहात: 12 पैकी धडा 11 · मागील: lesson-10-scorecards · पुढील: lesson-12-developer-portal

📦 या ब्रँचमध्ये काय आहे

धडे 01–10, आणि platform मदत करतो की नाही हे कसे मोजायचे. Deployments च्या यादीतून मोजलेल्या चार DORA keys, नव्या team साठी पहिल्या deploy पर्यंतचा वेळ, आकड्यांतून सुटणाऱ्या भागांसाठी SPACE framework — आणि ध्येय बनवलेला metric कसा अर्थहीन होतो याचे प्रात्यक्षिक. dora() idp/guard.py मध्ये, measure() idp/demo.py मध्ये.

🧒 5 वर्षांच्या मुलाला समजावल्यासारखे

मुख्याध्यापिका दीपिकाला विचारतात: "तू kits, catalogue आणि किल्ल्यांचे कपाट बनवलेस. त्याचा उपयोग झाला का?"

दीपिका science विभागाने आपल्या वर्गखोल्यांमध्ये केलेले बदल पाहते, kits च्या आधीचे चार आठवडे आणि नंतरचे चार आठवडे:

Science ने किती वेळा काहीतरी बदलले? आधी आठवड्याला 1, नंतर आठवड्याला 3.
"कल्पना लिहून ठेवली" पासून "बदल चालू झाला" पर्यंत किती वेळ? आधी 150 तास, नंतर 5 तास.
बदलामुळे किती वेळा काहीतरी बिघडले? आधी निम्म्या वेळा, नंतर 8%.
काहीतरी बिघडले तर ते दुरुस्त करायला किती वेळ? आधी 23 तास, नंतर 2 तास.

मग मुख्याध्यापिका म्हणतात: "छान! आजपासून प्रत्येक विभागाने आठवड्याला 9 बदल करायलाच हवेत." 📈 पुढच्या आठवड्यात maths विभाग 9 सांगतो... तोच बदल तीन वेळा मोजून. आकडा वाढला. विद्यार्थ्यांसाठी काहीच सुधारले नाही.

म्हणून दीपिका शिक्षिकांना विचारतेही: "तुमचे काम सोपे झाले का? तुम्ही कमी वेळा अडकता का?" आकडे आणि उत्तरे मिळून सत्य सांगतात. फक्त आकडे वाकवता येतात.

🗺️ आकृती

flowchart LR
    d["📦 deployments<br/>commit hour · deploy hour ·<br/>failed? · restored hour"] --> k["🔑 four DORA keys"]
    k --> f["deploys/week<br/>1.0 → 3.0"]
    k --> lt["lead time<br/>150 h → 5 h"]
    k --> cfr["change failure rate<br/>50% → 8%"]
    k --> rt["recovery time<br/>23 h → 2 h"]
    g["⚠️ count re-runs:<br/>9.0/week, nothing better"] -.-> f
    sp["🗣️ SPACE: ask people too"] --> k

🗺️ काढलेली आवृत्ती + एक lab: https://school-edh.pages.dev/platform-engineering/lesson-diagrams.html#l11

❓ काय

DORA — DevOps Research and Assessment program (आता Google Cloud चा भाग), दरवर्षीच्या State of DevOps reports आणि Accelerate (Forsgren, Humble, Kim, 2018) या पुस्तकासाठी प्रसिद्ध. त्याचे संशोधन software delivery च्या चार मोजमापांचा संस्थेच्या कामगिरीशी संबंध जोडते:
1. Deployment frequency — team production मध्ये किती वेळा deploy करते.
2. Lead time for changes — बदल commit झाल्यापासून तो production मध्ये चालू होईपर्यंतचा वेळ.
3. Change failure rate — दुरुस्ती लागणारे failure घडवणाऱ्या deployments चा वाटा (rollback, hotfix).
4. Failed deployment recovery time — अशा failure मधून सावरायला किती वेळ लागतो. जुन्या reports मध्ये याला time to restore service म्हणत (अनेकदा थोडक्यात MTTR). पहिली दोन throughput मोजतात, शेवटची दोन stability. अलीकडच्या DORA reports मध्ये पाचव्या मोजमापाचीही (rework rate) चर्चा झाली आहे; नेमका संच किती ते सध्याच्या report मध्ये पाहा.
Time to first deploy (onboarding time) — नव्या team किंवा developer ला पहिला बदल production मध्ये न्यायला किती वेळ लागतो. सोनेरी वाटेचे (धडा 03) थेट मोजमाप.
SPACE — एक framework (Forsgren, Storey, Maddila, Zimmermann, Houck आणि Butler, 2021) जो सांगतो की developer productivity ला पाच बाजू आहेत: Satisfaction and well-being, Performance, Activity, Communication and collaboration, Efficiency and flow. कोणताही एक आकडा त्या सगळ्या झाकत नाही; अनेक आकडे वापरा, आणि लोकांची स्वतःची उत्तरेही घ्या.
Goodhart's law — "जेव्हा एखादे मोजमाप ध्येय बनते, तेव्हा ते चांगले मोजमाप राहत नाही." Deployment frequency ला ध्येय बनवले की re-runs मोजणे, दिखाव्यासाठी बदल तुकडे करणे किंवा सुरक्षेच्या पायऱ्या वगळणे सुरू होते.
Weaponizing metrics — teams ना क्रमवारी लावण्यासाठी, बक्षीस देण्यासाठी किंवा शिक्षा करण्यासाठी team metrics वापरणे. DORA चे स्वतःचे मार्गदर्शन असे आहे की team ला तिच्या स्वतःच्या भूतकाळाच्या तुलनेत सुधारायला मदत करण्यासाठी ते वापरा, वेगवेगळे संदर्भ असलेल्या teams ची तुलना करण्यासाठी नाही.

🤔 का

कारण platform साठी लोकांची एक team लागते, आणि त्यातून काय मिळाले हे कोणीतरी विचारणारच. "जलद वाटते" हे budget meeting मध्ये टिकत नाही; खऱ्या deployments मधले, आधी आणि नंतरची तुलना केलेले चार आकडे अनेकदा टिकतात. पण तेच आकडे एकदा ध्येय बनले की ते ज्या वागणुकीचे निरीक्षण करायचे होते तीच बदलतात. म्हणून platform चा परिणाम teams वर मोजा, आकडे teams सोबत वाटा, आणि त्यांना लोकांना विचारण्याची जोड द्या.

🔧 कसे (या repo मध्ये)

idp/guard.py मधील dora(deploys, weeks) deployments (commit_hour, deploy_hour, failed, restored_hour) म्हणून घेतो. तो दर आठवड्याचे deployments, तासांमधला median lead time, पूर्ण टक्क्यांत change failure rate, आणि failed झालेल्यांचा median recovery time परत देतो. (DORA चे surveys ranges विचारतात; स्वतःच्या data चा सारांश काढण्याचा median हा एक योग्य मार्ग आहे.) idp/demo.py मधील measure() science चे आधीचे 4 आठवडे (BEFORE, 4 deploys) आणि नंतरचे (AFTER, 12 deploys) यांची तुलना करतो, धडा 01 च्या रांगेवरून पहिल्या deploy पर्यंतच्या वेळेचा अंदाज काढतो, आणि मग metric शी कसा खेळ होतो हे दाखवण्यासाठी प्रत्येक deploy तीन वेळा मोजतो.

🧪 करून पाहा

python3 idp/demo.py measure
python3 - <<'EOF'
import sys; sys.path.insert(0, "idp"); from guard import dora
week = [(0, 30, False, None), (24, 40, True, 46), (48, 60, False, None), (72, 80, False, None), (96, 130, True, 131)]
print("one week:", dora(week, 1))
big = [(0, 160, True, 190)]
print("one big batch:", dora(big, 1))
print("re-runs counted:", dora(week * 2, 1)["per_week"], "deploys/week · lead", dora(week * 2, 1)["lead_h"], "h")
EOF

✅ तपासा — तुम्हाला काय दिसायला हवे

measure हे छापते:

── the four DORA keys for the science department, 4 weeks each (teaching data):
   before (tickets)     1.0 deploys/week · lead time 150.0 h · change failure rate 50% · recovery 23.0 h
   after (paved path)   3.0 deploys/week · lead time   5.0 h · change failure rate  8% · recovery 2.0 h
── time to first deploy for a new team: 4 tickets in a row (repo, pipeline, database, DNS) × 2.0 days ≈ 8.0 days
   on the golden path: scaffold + self-service database + one pipeline run of 15 min — the same day
── weaponize the metric ('deploy more!'): count every re-run of the deploy job → 9.0 deploys/week, lead time 5.0 h

तुमचा snippet हे छापतो:

one week: {'per_week': 5.0, 'lead_h': 16.0, 'cfr': 40, 'recover_h': 3.5}
one big batch: {'per_week': 1.0, 'lead_h': 160.0, 'cfr': 100, 'recover_h': 30.0}
re-runs counted: 10.0 deploys/week · lead 16.0 h

🏁 तुम्ही आत्ताच काय सिद्ध केले

Demo मध्ये चारही keys एकत्र हलतात: लहान, वारंवार बदलांसोबत कमी failure rate आणि जलद recovery आले — DORA चे संशोधन सांगते तो नमुना, नियम नाही. एक मोठा batch एकाच वेळी चारही बाबतीत सर्वात वाईट ठरतो. Re-runs मोजल्याने deployment frequency दुप्पट किंवा तिप्पट झाली, तर lead time अगदी तसाच राहिला — कोणीतरी आकड्याशी खेळत असल्याची खूण. पहिल्या deploy पर्यंतचा वेळ tickets च्या सुमारे 8 दिवसांवरून (धडा 01 ची सरासरी वाट × 4 विनंत्या) त्याच दिवसावर आला, कारण सोनेरी वाट (03), self-service (05) आणि paved pipeline (07) यांनी ते चार tickets बदलले.

📏 शिकवण्यासाठीचा नमुना: आधी/नंतरचा data स्पष्टतेसाठी बनवलेला आहे, आणि 4 आठवडे हा लहान नमुना आहे. Tickets चा अंदाज असे गृहीत धरतो की 4 विनंत्या एकामागून एक केल्या जातात. खऱ्या मोजमापांना जास्त मोठा कालावधी लागतो आणि "failure" म्हणून काय मोजायचे याची काळजी घ्यावी लागते.

⚠️ नेहमीच्या चुका

deployment frequency वरून teams ची क्रमवारी लावणे — वेगवेगळ्या products ची लय वेगळी असते
DORA चा एखादा आकडा ध्येय म्हणून ठेवणे, आणि मग तो अर्थहीन झाल्यावर आश्चर्य वाटणे
फक्त activity (commits, lines, बंद केलेले tickets) मोजणे आणि त्याला productivity म्हणणे
फक्त systems मोजणे आणि developers ना कधीच न विचारणे (SPACE मधील S)
baseline नाही — platform आल्यानंतरच मोजणे, त्यामुळे तुलना करायला काहीच नसते

🏭 प्रत्यक्ष वापरात

चार keys पैकी बहुतेक तुमच्याकडे आधीच असलेल्या data वरून मोजता येतात. उदाहरणार्थ, deployment frequency आणि lead time Git आणि तुमच्या deploy records वरून — खऱ्या repo वर GitHub CLI ने (deployments तिथे नोंदलेले असायला हवेत, उदाहरणार्थ तुमच्या pipeline ने):

gh api "repos/school/exam-results/deployments?environment=production&per_page=100" \
  --jq '.[] | [.sha, .created_at] | @tsv' > deploys.tsv
while IFS=$'\t' read -r sha deployed; do
  committed=$(git show -s --format=%cI "$sha")
  echo "$sha committed $committed deployed $deployed"
done < deploys.tsv

लोकांच्या बाजूसाठी, लांब वार्षिक survey पेक्षा लहान, नियमित survey जास्त चांगला चालतो — उदाहरणार्थ 1–5 मापावर तीन प्रश्न: "How easy was it to ship a change this month?", "How often were you blocked waiting for another team?", "How confident are you when you deploy?" — आणि एक मोकळ्या उत्तराचा प्रश्न. DevEx framework (Noda, Storey, Forsgren आणि Greiler, 2023) feedback loops, cognitive load आणि flow भोवती survey प्रश्न सुचवतो.

🏭 प्रत्यक्ष वापरात हे का महत्त्वाचे: प्रत्येक team ला तिचा स्वतःचा कल दाखवा, क्रमवारी नाही. Platform चा परिणाम "paved path वरच्या teams विरुद्ध आधी" असा सांगा, आणि नेहमी developers काय म्हणतात त्याच्या शेजारी.

⏭️ पुढे

कार्यालयाचे आता अनेक भाग आहेत. शेवटचा धडा त्यांना एका स्वागत कक्षा मागे आणतो: developer portal — आणि संपूर्ण चित्र काढतो.

git checkout lesson-12-developer-portal

📊 Lesson 11 — Measuring the platform: is the office helping?

📍 You are here: Lesson 11 of 12 · Previous: lesson-10-scorecards · Next: lesson-12-developer-portal

📦 What's in this branch

Lessons 01–10, plus how to measure whether the platform helps. The four DORA keys computed from a list of deployments, time to first deploy for a new team, the SPACE framework for the parts numbers miss — and a demonstration of how a metric turned into a target stops meaning anything. dora() in idp/guard.py, measure() in idp/demo.py.

🧒 Explain like I'm 5

The head teacher asks Dipika: "You built kits, a catalogue and a key cabinet. Did it help?"

Dipika looks at the science department's changes to its classrooms, four weeks before the kits and four weeks after:

How often did science change something? 1 a week before, 3 a week after.
How long from "idea written down" to "change is live"? 150 hours before, 5 hours after.
How often did a change break something? Half the time before, 8% after.
When something broke, how long to fix it? 23 hours before, 2 hours after.

Then the head teacher says: "Great! From now on every department must make 9 changes a week." 📈 The next week the maths department reports 9... by counting the same change three times. The number went up. Nothing got better for the students.

So Dipika also asks the teachers: "Is your work easier? Do you get stuck less?" The numbers and the answers together tell the truth. Numbers alone can be bent.

🗺️ Diagram

flowchart LR
    d["📦 deployments<br/>commit hour · deploy hour ·<br/>failed? · restored hour"] --> k["🔑 four DORA keys"]
    k --> f["deploys/week<br/>1.0 → 3.0"]
    k --> lt["lead time<br/>150 h → 5 h"]
    k --> cfr["change failure rate<br/>50% → 8%"]
    k --> rt["recovery time<br/>23 h → 2 h"]
    g["⚠️ count re-runs:<br/>9.0/week, nothing better"] -.-> f
    sp["🗣️ SPACE: ask people too"] --> k

🗺️ Drawn version + a lab: https://school-edh.pages.dev/platform-engineering/lesson-diagrams.html#l11

❓ What

DORA — the DevOps Research and Assessment program (now part of Google Cloud), known for the yearly State of DevOps reports and the book Accelerate (Forsgren, Humble, Kim, 2018). Its research links four software delivery measures to organisational performance:
1. Deployment frequency — how often a team deploys to production.
2. Lead time for changes — time from a change being committed to it running in production.
3. Change failure rate — the share of deployments that cause a failure needing a fix (a rollback, a hotfix).
4. Failed deployment recovery time — how long it takes to recover from such a failure. Older reports called it time to restore service (often shortened to MTTR). The first two measure throughput, the last two stability. Recent DORA reports have also discussed a fifth measure (rework rate); check the current report for the exact set.
Time to first deploy (onboarding time) — how long a new team or developer takes to get a first change into production. A direct measure of the golden path (lesson 03).
SPACE — a framework (Forsgren, Storey, Maddila, Zimmermann, Houck and Butler, 2021) that says developer productivity has five dimensions: Satisfaction and well-being, Performance, Activity, Communication and collaboration, Efficiency and flow. No single number covers them; use several, and include people's own answers.
Goodhart's law — "when a measure becomes a target, it ceases to be a good measure." Deployment frequency made into a target invites counting re-runs, splitting changes for show, or skipping safety steps.
Weaponizing metrics — using team metrics to rank, reward or punish teams. DORA's own guidance is to use them to help a team improve against its own past, not to compare teams with different contexts.

🤔 Why

Because a platform costs a team of people, and someone will ask what it bought. "It feels faster" does not survive a budget meeting; four numbers from real deployments, compared before and after, often do. But the same numbers, once they become targets, change the behaviour they were meant to observe. So measure the platform's effect on teams, share the numbers with the teams, and pair them with asking people.

🔧 How (in this repo)

dora(deploys, weeks) in idp/guard.py takes deployments as (commit_hour, deploy_hour, failed, restored_hour). It returns deployments per week, the median lead time in hours, the change failure rate as a whole percent, and the median recovery time of the failed ones. (DORA's surveys ask for ranges; medians are one reasonable way to summarise your own data.) measure() in idp/demo.py compares science's 4 weeks before (BEFORE, 4 deploys) and after (AFTER, 12 deploys), estimates time to first deploy from lesson 01's queue, and then counts each deploy three times to show the metric being gamed.

🧪 Try it

python3 idp/demo.py measure
python3 - <<'EOF'
import sys; sys.path.insert(0, "idp"); from guard import dora
week = [(0, 30, False, None), (24, 40, True, 46), (48, 60, False, None), (72, 80, False, None), (96, 130, True, 131)]
print("one week:", dora(week, 1))
big = [(0, 160, True, 190)]
print("one big batch:", dora(big, 1))
print("re-runs counted:", dora(week * 2, 1)["per_week"], "deploys/week · lead", dora(week * 2, 1)["lead_h"], "h")
EOF

✅ Verify — what you should see

measure prints:

── the four DORA keys for the science department, 4 weeks each (teaching data):
   before (tickets)     1.0 deploys/week · lead time 150.0 h · change failure rate 50% · recovery 23.0 h
   after (paved path)   3.0 deploys/week · lead time   5.0 h · change failure rate  8% · recovery 2.0 h
── time to first deploy for a new team: 4 tickets in a row (repo, pipeline, database, DNS) × 2.0 days ≈ 8.0 days
   on the golden path: scaffold + self-service database + one pipeline run of 15 min — the same day
── weaponize the metric ('deploy more!'): count every re-run of the deploy job → 9.0 deploys/week, lead time 5.0 h

Your snippet prints:

one week: {'per_week': 5.0, 'lead_h': 16.0, 'cfr': 40, 'recover_h': 3.5}
one big batch: {'per_week': 1.0, 'lead_h': 160.0, 'cfr': 100, 'recover_h': 30.0}
re-runs counted: 10.0 deploys/week · lead 16.0 h

🏁 What you just proved

The four keys move together in the demo: smaller, more frequent changes came with a lower failure rate and faster recovery — the pattern DORA's research reports, not a law. One big batch is the worst of all four at once. Counting re-runs doubled or tripled deployment frequency while lead time stayed exactly the same — a sign someone is playing the number. Time to first deploy fell from about 8 days of tickets (lesson 01's average wait × 4 requests) to the same day, because the golden path (03), self-service (05) and the paved pipeline (07) replaced the four tickets.

📏 Teaching model: the before/after data is invented to be clear, and 4 weeks is a small sample. The ticket estimate assumes the 4 requests are made one after another. Real measurements need longer periods and care about what counts as a "failure".

⚠️ Common mistakes

ranking teams by deployment frequency — different products have different rhythms
setting a DORA number as a target, then being surprised when it stops meaning anything
measuring only activity (commits, lines, tickets closed) and calling it productivity
measuring only systems and never asking developers (the S in SPACE)
no baseline — measuring only after the platform exists, so there is nothing to compare

🏭 In production

Most of the four keys can be computed from data you already have. For example, deployment frequency and lead time from Git and your deploy records — on a real repo with the GitHub CLI (deployments must be recorded there, for example by your pipeline):

gh api "repos/school/exam-results/deployments?environment=production&per_page=100" \
  --jq '.[] | [.sha, .created_at] | @tsv' > deploys.tsv
while IFS=$'\t' read -r sha deployed; do
  committed=$(git show -s --format=%cI "$sha")
  echo "$sha committed $committed deployed $deployed"
done < deploys.tsv

For the people side, a short, regular survey works better than a long yearly one — for example three questions on a 1–5 scale: "How easy was it to ship a change this month?", "How often were you blocked waiting for another team?", "How confident are you when you deploy?" — plus one free-text question. The DevEx framework (Noda, Storey, Forsgren and Greiler, 2023) suggests survey questions around feedback loops, cognitive load and flow.

🏭 Why this matters in production: show each team its own trend, not a ranking. Report the platform's effect as "teams on the paved path vs before", and always next to what developers say.

⏭️ Next

The office now has many parts. The last lesson puts them behind one front desk: the developer portal — and draws the whole picture.

git checkout lesson-12-developer-portal

← Previousscorecards Next →developer portal

This page is the lesson's README from the lesson-11-measuring branch, shown here so the whole School stays on one site. Code files open on GitHub at the same branch.