🖼️ See the drawing + lab 🏠 Course home 🌿 Branch on GitHub ✏️ View source

🖼️ आकृती आणि labThe drawing + lab पूर्ण पानावर उघडा ↗Open full page ↗

📈 धडा 08 — Delivery metrics: DORA four keys, flow, आणि Goodhart's law

📍 तुम्ही इथे आहात: 12 पैकी धडा 08 · मागचा: lesson-07-technical-debt · पुढचा: lesson-09-team-topologies

📦 या ब्रँचमध्ये काय आहे

धडा 07, आणि delivery system चे मोजमाप. DORA four keys — deployment frequency, lead time for changes, change failure rate, आणि failed-deployment recovery time — आणि त्यासोबत reliability; cycle time आणि flow efficiency सारखे flow metrics; आणि Goodhart's law, म्हणजे यांपैकी कोणतेही माप target बनले की त्याचे काय होते. metrics() lead/demo.py मध्ये; dora() आणि flow() lead/models.py मध्ये.

🧒 5 वर्षांच्या मुलाला समजावल्यासारखे

मुख्याध्यापिकेला जाणून घ्यायचे आहे की गणित विभाग चांगले काम करतो आहे का. म्हणून ती काही गोष्टी मोजते: नवे धडे किती वेळा तयार होतात, त्यांना किती वेळ लागतो, आणि किती वेळा एखादा धडा चुकतो. 📊

उपयोगी! जोपर्यंत ती असे म्हणत नाही: "आजपासून 10 पैकी 1 पेक्षा कमी धडे चुकले पाहिजेत."

मग काहीतरी विचित्र घडते. विभाग सुधारत नाही. पण आकडा सुधारतो. एखादा धडा चुकला की लोक त्याला "नियोजित बदल" म्हणू लागतात. 🙈 आकडा 17% वरून 6% वर येतो. विद्यार्थ्यांना पूर्वीइतकेच वाईट धडे मिळाले.

मग ती म्हणते: "आणखी धडे हवेत!" आणि विभाग प्रत्येक छोट्या worksheet ला "धडा" म्हणून मोजू लागतो. आकडा दुप्पट होतो. विद्यार्थी काहीच नवीन शिकत नाहीत. 📄📄📄

कतरिना समजावते: "एखादा आकडा target बनला की लोक मोजल्या जाणाऱ्या गोष्टीऐवजी आकड्यावरच काम करू लागतात." म्हणून ती आकड्यांचा वापर विभाग कसा काम करतो याबद्दल प्रश्न विचारण्यासाठी करते — शिक्षिकांची क्रमवारी लावण्यासाठी कधीच नाही.

🗺️ आकृती

flowchart LR
    l["📦 18 deploys in 4 weeks"] --> k["four keys<br/>4.5/week · lead 18 h<br/>CFR 16.7% · recovery 45 min"]
    l --> a["reliability<br/>availability 99.52%"]
    k -->|"target: CFR under 10%"| g1["2 failures relabelled<br/>CFR 5.6% · availability still 99.52%"]
    k -->|"target: deploy more"| g2["18 no-op deploys<br/>9.0/week · users see nothing new"]
    f["🌊 flow: cycle times 5–12 days<br/>flow efficiency 40%"]

🗺️ काढलेली आवृत्ती + एक lab: https://school-edh.pages.dev/engineering-leadership/lesson-diagrams.html#l08

❓ काय

DORA — DevOps Research and Assessment कार्यक्रम, ज्याच्या अनेक वर्षांच्या surveys चा सारांश Accelerate (Forsgren, Humble आणि Kim, 2018) आणि दरवर्षीच्या State of DevOps reports मध्ये आहे. या संशोधनात दिसले की त्याची चार delivery मापे संस्थेच्या कामगिरीशी संबंधित आहेत, आणि चांगली कामगिरी करणाऱ्या संस्था वेग आणि स्थिरता दोन्हींत चांगल्या असतात — त्यांच्या data मध्ये या दोघांत देवाणघेवाण दिसली नाही. हे correlation वरचे survey-आधारित संशोधन आहे; एक आकडा हलवल्याने निकाल मिळतीलच असे ते वचन देत नाही.
The four keys — deployment frequency (production मध्ये किती वेळा release करता) · lead time for changes (commit पासून production मध्ये चालू होईपर्यंत) · change failure rate (fix लागणारी failure निर्माण करणाऱ्या deployments चा वाटा) · failed deployment recovery time (अशा failure मधून सावरायला किती वेळ लागतो; आधीच्या reports मध्ये याला time to restore service म्हणत). पहिली दोन throughput आहेत; शेवटची दोन stability.
Reliability — DORA ने 2021 मध्ये पाचवे माप म्हणून जोडले: service तिची स्वतःची reliability targets पूर्ण करते का? हा lab user-impact minutes वरून availability वापरतो.
Flow metrics — टीममधून काम कसे पुढे सरकते: WIP (work in progress), cycle time (एका कामाची सुरुवात ते शेवट), throughput (धडा 05), आणि flow efficiency (active time ÷ elapsed time — उरलेला वेळ म्हणजे वाट पाहणे). Mik Kersten चे Flow Framework आणि Kanban पद्धत दोन्ही यांचा वापर करतात.
Goodhart's law — अर्थशास्त्रज्ञ Charles Goodhart यांचे निरीक्षण (1975); प्रचलित शब्दरचना मानववंशशास्त्रज्ञ Marilyn Strathern यांची आहे: "When a measure becomes a target, it ceases to be a good measure." (एखादे माप target बनले की ते चांगले माप राहत नाही.)
SPACE — 2021 चा एक framework (Forsgren आणि सहकारी), जो सांगतो की developer productivity ला अनेक बाजू आहेत (satisfaction, performance, activity, communication, efficiency) आणि कोणताही एकच आकडा ती पूर्ण पकडू शकत नाही.

🤔 का

कारण टीमच्या कामाच्या पद्धतीतले बदल मदत करत आहेत का हे lead ला कळायला हवे, आणि भावना विश्वासार्ह नसतात. Four keys व्यक्तींचे नव्हे तर delivery system चे वर्णन करतात — बदल users पर्यंत किती वेगाने आणि किती सुरक्षितपणे पोहोचतात. आणि कारण उपयोगी metric बिघडवण्याचा सर्वात जलद मार्ग म्हणजे त्याला target बनवून लोकांशी जोडणे: टीम कसाही करून आकडा सुधारेल.

🔧 कसे (या repo मध्ये)

lead/models.py मधील dora(log, weeks) deploys (lead time hours, failed?, minutes of user impact) या रूपात घेते आणि परत देते: आठवड्याची frequency, median lead time, change failure rate, failed deploys चा median recovery time, आणि availability (1 − impact minutes ÷ 28 दिवसांतली minutes). Impact minutes deploy ला कोणतेही लेबल लावले तरी मोजली जातात — आणि याच मार्गाने demo relabelling उघडे पाडतो. flow(items) (start day, end day, active days) घेते आणि cycle times व flow efficiency परत देते. lead/demo.py मधील metrics() एक प्रामाणिक log आणि दोन फसवलेले logs चालवते. हे आकडे एका system चे वर्णन करतात: विचार करण्याचे साधन, लोकांसाठीचे सूत्र नव्हे.

🧪 करून पाहा

python3 lead/demo.py metrics
python3 - <<'EOF'
import sys; sys.path.insert(0, "lead"); from models import dora, flow
leads = [20, 6, 30, 52, 8, 4, 26, 72, 10, 14, 5, 28, 48, 9, 22, 3, 16, 40]
fails = {3: 45, 7: 120, 12: 30}
for hidden in (0, 1, 2, 3):
    relabel = list(fails)[:hidden]
    log = [(l, i in fails and i not in relabel, fails.get(i, 0)) for i, l in enumerate(leads)]
    d = dora(log, 4)
    print(f"relabel {hidden} of 3 failures → CFR {d['cfr']:>5.1%} · availability {d['availability']:.2%}")
cyc, eff = flow([(0, 5, 2), (1, 9, 3), (2, 4, 2), (3, 15, 4), (5, 8, 2), (6, 16, 3)])
cyc2, eff2 = flow([(0, 3, 2), (1, 5, 3), (2, 4, 2), (3, 8, 4), (5, 7, 2), (6, 10, 3)])
print(f"board A: cycle {cyc} days · flow efficiency {eff:.0%}")
print(f"board B: cycle {cyc2} days · flow efficiency {eff2:.0%}")
EOF

✅ तपासा — तुम्हाला काय दिसायला हवे

metrics हे छापते:

   deployment frequency 4.5/week · lead time for changes (median) 18 h · change failure rate 16.7% · failed-deploy recovery (median) 45 min · availability 99.52%
── target 'change failure rate under 10%' → two failures relabelled 'planned hotfix' → CFR 5.6% · availability still 99.52%
── target 'deploy more' → 18 config-only no-op deploys added → frequency 9.0/week · median lead time 1.8 h · users see nothing new
   Goodhart's law, in Marilyn Strathern's words: when a measure becomes a target, it ceases to be a good measure

तुमचा snippet हे छापतो:

relabel 0 of 3 failures → CFR 16.7% · availability 99.52%
relabel 1 of 3 failures → CFR 11.1% · availability 99.52%
relabel 2 of 3 failures → CFR  5.6% · availability 99.52%
relabel 3 of 3 failures → CFR  0.0% · availability 99.52%
board A: cycle [5, 8, 2, 12, 3, 10] days · flow efficiency 40%
board B: cycle [3, 4, 2, 5, 2, 4] days · flow efficiency 80%

🏁 तुम्ही आत्ताच काय सिद्ध केले

Failures चे लेबल बदलल्याने change failure rate 16.7% वरून 0.0% झाला, पण availability 99.52% वरच राहिली — users ना प्रत्येक मिनिट जाणवले. Log मध्ये no-op deploys भरल्याने deployment frequency दुप्पट झाली (आठवड्याला 4.5 → 9.0) आणि median lead time 1.8 hours वर आला, पण users साठी काहीच नवीन नव्हते. म्हणूनच एकमेकांविरुद्ध ओढणाऱ्या मापांची जोडी (वेग आणि स्थिरता), आणि users ना जाणवणारा एक परिणाम, यांना फसवणे एका आकड्यापेक्षा कठीण असते. Flow boards आणखी एक बाजू दाखवतात: board A वर कामांनी त्यांचा 60% वेळ वाट पाहण्यात घालवला.

⚠️ नेहमीच्या चुका

या आकड्यांनी व्यक्तींची किंवा टीम्सची तुलना करणे — ते systems चे वर्णन करतात, आणि टीम्स वेगवेगळ्या असतात
एका metric वर त्याच्या जोडीदाराशिवाय target ठेवणे (स्थिरतेशिवाय वेग)
user ला दिसणाऱ्या बदलांऐवजी pipeline runs मोजून deployment frequency मोजणे
कोणीच चर्चा न करणारा dashboard — metrics तेव्हाच मदत करतात जेव्हा ते संवाद सुरू करतात
एखाद्या report साठी वर्षातून एकदा हाताने आकडे गोळा करणे
DORA च्या correlations ना हमी समजणे: आकडा सुधारणे म्हणजे system सुधारणे नव्हे

🏭 प्रत्यक्ष वापरात

Four keys पैकी बहुतेक तुमच्याकडे आधीच असलेल्या data मधून काढता येतात: pipeline मधून deploy events, git मधून commit times, incident tracker मधून incidents. खऱ्या repository वर, PR उघडल्यापासून merge पर्यंतचा lead time हा पहिला ढोबळ तुकडा आहे (पूर्ण lead time production पर्यंत चालतो):

gh pr list --state merged --limit 100 --json number,createdAt,mergedAt \
  --jq '.[] | "\(.number) \(.createdAt) \(.mergedAt)"'

Deploys आणि failures ना मूळ ठिकाणीच tag करा, म्हणजे आकडे कोणाच्या लेबलवर अवलंबून राहणार नाहीत:

# a deploy event your pipeline writes on every production release
deploy:
  service: gradebook
  sha: 3f9c2e1
  commit_time: 2026-09-14T09:12:00Z
  deployed_at: 2026-09-14T15:40:00Z
  caused_incident: INC-231        # linked by the incident process, not self-reported

Four keys चा आढावा टीम म्हणून, दर महिन्याला, एकाच प्रश्नाने घ्या: "आपल्या system मधली कोणती गोष्ट हा आकडा असा बनवते आहे?" लहान, वारंवार deploys सुरक्षित करणाऱ्या pipelines साठी CI/CD school पाहा, आणि users ना काय जाणवते हे मोजण्यासाठी Observability school पाहा.

🏭 प्रत्यक्ष वापरात हे का महत्त्वाचे: delivery system मोजा, एकमेकांना प्रामाणिक ठेवणाऱ्या जोड्यांमध्ये, आणि आकड्यांवर टीमसोबत चर्चा करा. ज्या क्षणी एखादा metric कोणाच्या performance review मध्ये येतो, त्या क्षणी त्यावर विश्वास ठेवणे थांबवा.

⏭️ पुढे

भाग 2 पूर्ण झाला: forecasting, prioritisation, debt आणि metrics. भाग 3 टीम्सच्या रचनेबद्दल आहे — आणि तुमच्या संस्थेचा आकार शेवटी तुमच्या software च्या आकारात का उतरतो.

git checkout lesson-09-team-topologies

📈 Lesson 08 — Delivery metrics: the DORA four keys, flow, and Goodhart's law

📍 You are here: Lesson 08 of 12 · Previous: lesson-07-technical-debt · Next: lesson-09-team-topologies

📦 What's in this branch

Lesson 07, plus measuring the delivery system. The DORA four keys — deployment frequency, lead time for changes, change failure rate, and failed-deployment recovery time — plus reliability; flow metrics such as cycle time and flow efficiency; and Goodhart's law, which is what happens to any of them when it becomes a target. metrics() in lead/demo.py; dora() and flow() in lead/models.py.

🧒 Explain like I'm 5

The head teacher wants to know if the maths department is doing well. So she counts things: how often new lessons are ready, how long they take, and how often a lesson goes wrong. 📊

Useful! Until she says: "From now on, fewer than 1 in 10 lessons may go wrong."

Something strange happens. The department does not get better. But the number does. When a lesson goes wrong, people start calling it a "planned change" instead. 🙈 The number drops from 17% to 6%. The pupils had exactly as many bad lessons as before.

Then she says: "More lessons, please!" And the department starts counting every small worksheet as a "lesson". The number doubles. The pupils learn nothing new. 📄📄📄

Katrina explains: "When a number becomes a target, people start working on the number instead of the thing it measured." So she uses the numbers to ask questions about how the department works — never to rank the teachers.

🗺️ Diagram

flowchart LR
    l["📦 18 deploys in 4 weeks"] --> k["four keys<br/>4.5/week · lead 18 h<br/>CFR 16.7% · recovery 45 min"]
    l --> a["reliability<br/>availability 99.52%"]
    k -->|"target: CFR under 10%"| g1["2 failures relabelled<br/>CFR 5.6% · availability still 99.52%"]
    k -->|"target: deploy more"| g2["18 no-op deploys<br/>9.0/week · users see nothing new"]
    f["🌊 flow: cycle times 5–12 days<br/>flow efficiency 40%"]

🗺️ Drawn version + a lab: https://school-edh.pages.dev/engineering-leadership/lesson-diagrams.html#l08

❓ What

DORA — the DevOps Research and Assessment programme, whose multi-year surveys are summarised in Accelerate (Forsgren, Humble and Kim, 2018) and yearly State of DevOps reports. The research found that its four delivery measures relate to organisational performance, and that the stronger performers tended to do well on speed and stability — in their data, the two did not trade off. It is survey-based research on correlation; it does not promise that moving one number causes results.
The four keys — deployment frequency (how often you release to production) · lead time for changes (commit to running in production) · change failure rate (share of deployments that cause a failure needing a fix) · failed deployment recovery time (how long to recover from such a failure; earlier reports called it time to restore service). The first two are throughput; the last two are stability.
Reliability — added by DORA in 2021 as a fifth measure: does the service meet its own reliability targets? This lab uses availability from user-impact minutes.
Flow metrics — how work moves through the team: WIP (work in progress), cycle time (start to finish of one item), throughput (lesson 05), and flow efficiency (active time ÷ elapsed time — the rest is waiting). Mik Kersten's Flow Framework and Kanban practice both use them.
Goodhart's law — economist Charles Goodhart's observation (1975); the common wording is anthropologist Marilyn Strathern's: "When a measure becomes a target, it ceases to be a good measure."
SPACE — a 2021 framework (Forsgren and colleagues) arguing that developer productivity has several dimensions (satisfaction, performance, activity, communication, efficiency) and no single number captures it.

🤔 Why

Because a lead needs to know whether changes to the way the team works are helping, and feelings are unreliable. The four keys describe the delivery system — how fast and how safely changes reach users — rather than individuals. And because the fastest way to break a useful metric is to set it as a target and tie it to people: the team will improve the number, one way or another.

🔧 How (in this repo)

dora(log, weeks) in lead/models.py takes deploys as (lead time hours, failed?, minutes of user impact) and returns frequency per week, median lead time, change failure rate, median recovery time of failed deploys, and availability (1 − impact minutes ÷ minutes in 28 days). The impact minutes count however the deploy was labelled — which is how the demo exposes relabelling. flow(items) takes (start day, end day, active days) and returns cycle times and flow efficiency. metrics() in lead/demo.py runs an honest log and two gamed ones. The numbers describe a system: a thinking tool, not a formula for people.

🧪 Try it

python3 lead/demo.py metrics
python3 - <<'EOF'
import sys; sys.path.insert(0, "lead"); from models import dora, flow
leads = [20, 6, 30, 52, 8, 4, 26, 72, 10, 14, 5, 28, 48, 9, 22, 3, 16, 40]
fails = {3: 45, 7: 120, 12: 30}
for hidden in (0, 1, 2, 3):
    relabel = list(fails)[:hidden]
    log = [(l, i in fails and i not in relabel, fails.get(i, 0)) for i, l in enumerate(leads)]
    d = dora(log, 4)
    print(f"relabel {hidden} of 3 failures → CFR {d['cfr']:>5.1%} · availability {d['availability']:.2%}")
cyc, eff = flow([(0, 5, 2), (1, 9, 3), (2, 4, 2), (3, 15, 4), (5, 8, 2), (6, 16, 3)])
cyc2, eff2 = flow([(0, 3, 2), (1, 5, 3), (2, 4, 2), (3, 8, 4), (5, 7, 2), (6, 10, 3)])
print(f"board A: cycle {cyc} days · flow efficiency {eff:.0%}")
print(f"board B: cycle {cyc2} days · flow efficiency {eff2:.0%}")
EOF

✅ Verify — what you should see

metrics prints:

   deployment frequency 4.5/week · lead time for changes (median) 18 h · change failure rate 16.7% · failed-deploy recovery (median) 45 min · availability 99.52%
── target 'change failure rate under 10%' → two failures relabelled 'planned hotfix' → CFR 5.6% · availability still 99.52%
── target 'deploy more' → 18 config-only no-op deploys added → frequency 9.0/week · median lead time 1.8 h · users see nothing new
   Goodhart's law, in Marilyn Strathern's words: when a measure becomes a target, it ceases to be a good measure

Your snippet prints:

relabel 0 of 3 failures → CFR 16.7% · availability 99.52%
relabel 1 of 3 failures → CFR 11.1% · availability 99.52%
relabel 2 of 3 failures → CFR  5.6% · availability 99.52%
relabel 3 of 3 failures → CFR  0.0% · availability 99.52%
board A: cycle [5, 8, 2, 12, 3, 10] days · flow efficiency 40%
board B: cycle [3, 4, 2, 5, 2, 4] days · flow efficiency 80%

🏁 What you just proved

Relabelling failures took the change failure rate from 16.7% to 0.0% while availability stayed at 99.52% — users felt every minute. Padding the log with no-op deploys doubled deployment frequency (4.5 → 9.0 a week) and cut the median lead time to 1.8 hours, with nothing new for users. That is why a pair of measures that pull against each other (speed and stability), plus an outcome users feel, is harder to game than one number. The flow boards show another view: on board A, items spent 60% of their time waiting.

⚠️ Common mistakes

comparing individuals or teams by these numbers — they describe systems, and teams differ
setting a target on one metric without its partner (speed without stability)
measuring deployment frequency by counting pipeline runs, not user-facing changes
a dashboard nobody discusses — metrics only help if they start a conversation
collecting numbers by hand once a year for a report
reading DORA's correlations as a guarantee: improving a number is not the same as improving the system

🏭 In production

Most of the four keys can be computed from data you already have: deploy events from the pipeline, commit times from git, incidents from the incident tracker. On a real repository, lead time from a PR opening to merge is a first rough slice (the full lead time runs to production):

gh pr list --state merged --limit 100 --json number,createdAt,mergedAt \
  --jq '.[] | "\(.number) \(.createdAt) \(.mergedAt)"'

Tag deploys and failures at the source, so the numbers do not depend on anyone's label:

# a deploy event your pipeline writes on every production release
deploy:
  service: gradebook
  sha: 3f9c2e1
  commit_time: 2026-09-14T09:12:00Z
  deployed_at: 2026-09-14T15:40:00Z
  caused_incident: INC-231        # linked by the incident process, not self-reported

Review the four keys as a team, monthly, with one question: "what in our system is making this number what it is?" See the CI/CD school for pipelines that make small, frequent deploys safe, and the Observability school for measuring what users feel.

🏭 Why this matters in production: measure the delivery system, in pairs that keep each other honest, and discuss the numbers with the team. The moment a metric appears in someone's performance review, stop trusting it.

⏭️ Next

Part 2 is done: forecasting, prioritisation, debt and metrics. Part 3 is about the shape of teams — and why the shape of your organisation ends up in the shape of your software.

git checkout lesson-09-team-topologies

← Previoustechnical debt Next →team topologies

This page is the lesson's README from the lesson-08-delivery-metrics branch, shown here so the whole School stays on one site. Code files open on GitHub at the same branch.