🖼️ See the drawing + lab 🏠 Course home 🌿 Branch on GitHub ✏️ View source

🖼️ आकृती आणि labThe drawing + lab पूर्ण पानावर उघडा ↗Open full page ↗

🧑‍🏫 धडा 04 — Hiring: structured rubric, स्वतंत्र गुण, calibration

📍 तुम्ही इथे आहात: 12 पैकी धडा 04 · मागे: lesson-03-delegation · पुढे: lesson-05-forecasting

📦 या ब्रँचमध्ये काय आहे

धडा 03, आणि hiring. Rubric सह एक structured interview (प्रत्येक उमेदवारासाठी तेच प्रश्न आणि anchored गुण), प्रत्येक interviewer debrief च्या आधी आपले गुण का लिहून ठेवते, आणि calibration — काही interviewers इतरांपेक्षा जास्त किंवा कमी गुण देतात हे लक्षात घेणे. lead/demo.py मधले hiring(); lead/models.py मधले panel(), herd() आणि calibrate().

🧒 5 वर्षांच्या मुलाला समजावल्यासारखे

गणित विभागाला एका नव्या शिक्षिकेची गरज आहे. कतरिना, दीपिका आणि ऐश्वर्या तिघीही उमेदवाराला एक नमुना धडा शिकवताना पाहतात. 🧑‍🏫

मग त्या निर्णय घेण्यासाठी भेटतात. कतरिना आधी बोलते: "ती अप्रतिम होती!" दीपिकाला शंका होत्या — विद्यार्थ्यांना समजले आहे की नाही हे ती कसे तपासते हे उमेदवाराला समजावता आले नाही. पण कतरिना विभागप्रमुख आहे, आणि ती इतकी खात्रीने बोलली… म्हणून दीपिका म्हणते, "हो, बऱ्यापैकी चांगली." ऐश्वर्याही सहमती देते.

उमेदवाराची निवड होते. दीपिकाची शंका — पुराव्याचा एकमेव खरा तुकडा — कधी समोर आलीच नाही. 🙊

पुढच्या वेळी कतरिना नियम बदलते:

प्रत्येकजण तीच score sheet वापरते, प्रत्येक गुणासाठी स्पष्ट वर्णनासह. 📋
प्रत्येकजण meeting च्या आधी तिचे गुण लिहून ठेवते. ✍️
Meeting मध्ये सर्वात कनिष्ठ व्यक्ती आधी बोलते.

आता त्याच तीन मतांची बेरीज वेगळे उत्तर देते — प्रामाणिक उत्तर.

🗺️ आकृती

flowchart LR
    s["📋 same rubric, 4 competencies<br/>scores 1–4, bar 3.0"] --> i["✍️ written before the debrief<br/>Katrina 3.75 · Dipika 2.25 · Aishwarya 2.50"]
    i --> p1["panel 2.83 → no hire"]
    s --> h["🗣️ Katrina speaks first,<br/>others move halfway to her"]
    h --> p2["panel 3.29 → hire"]
    c["⚖️ calibration, 5 past panels<br/>Katrina +0.37 · Dipika -0.33"] --> x["X raw 3.25 → 3.08<br/>Y raw 3.00 → 3.18"]

🗺️ काढलेली आवृत्ती + एक lab: https://school-edh.pages.dev/engineering-leadership/lesson-diagrams.html#l04

❓ काय

Structured interview — एखाद्या भूमिकेसाठी प्रत्येक उमेदवाराला तेच प्रश्न (किंवा तेच काम) मिळतात, आणि प्रशिक्षित interviewers त्याच rubric नुसार गुण देतात. Unstructured: प्रत्येक interviewer तिला हवे ते विचारते आणि एकूण छाप तयार करते. निवड-पद्धतींवरील संशोधनाच्या मोठ्या आढाव्यांमध्ये — Schmidt आणि Hunter (1998), आणि Sackett व सहकाऱ्यांचे 2022 मधील पुनर्विश्लेषण — असे आढळले की structured interviews unstructured पेक्षा job performance चा अधिक चांगला अंदाज देतात, आणि अभ्यासलेल्या सर्वात प्रभावी पद्धतींमध्ये आहेत. "अधिक चांगला अंदाज" हे अनेक hires च्या सरासरीबद्दलचे विधान आहे, एका hire ची हमी नाही.
Anchors असलेली rubric — प्रत्येक competency साठी प्रत्येक गुण कसा दिसतो याचे लिखित वर्णन, म्हणजे प्रत्येक interviewer साठी "3" चा अर्थ एकच असतो. "testing approach" चे उदाहरण: 1 testing चा उल्लेखच नाही · 2 विचारल्यावर, नंतर tests · 3 न विचारता unit आणि integration tests चे वर्णन करते · 4 धोक्यानुसार tests निवडते आणि trade-offs समजावते.
Independent scoring — प्रत्येक interviewer इतर कोणाचे ऐकण्याआधी तिचे गुण आणि पुरावा लिहून ठेवते. लोक चर्चा करू लागले की नंतरची मते आधीच्या आणि मोठ्या आवाजातील मतांकडे झुकतात (anchoring, सामाजिक दबाव). Kahneman, Sibony आणि Sunstein यांचे Noise (2021) नेमक्या याच कारणासाठी चर्चेआधी स्वतंत्र निर्णयांची शिफारस करते.
Herding (या lab मध्ये) — त्या झुकण्याचे एक शिकवण्यासाठीचे model: पहिल्या वक्तीचे ऐकल्यावर प्रत्येक इतर interviewer तिच्या गुणांकडे ठरलेल्या प्रमाणात (pull) सरकते. खरे झुकणे जास्त गुंतागुंतीचे असते; मुद्दा दिशेचा आहे.
Calibration — interviewers च्या सवयी वेगवेगळ्या असतात: काही जास्त गुण देतात, काही कमी. त्याच उमेदवारांवर प्रत्येकीच्या गुणांची panel मधील इतरांशी तुलना केल्याने तिचा offset अंदाजता येतो. वेगवेगळे उमेदवार वेगवेगळ्या panels ना भेटतात तेव्हा हे सर्वात महत्त्वाचे असते.
The bar — भूमिकेसाठी आवश्यक किमान पातळी, interviews सुरू होण्याआधी ठरवलेली.

🤔 का

कारण hire हा lead च्या सर्वात महाग आणि उलटायला सर्वात कठीण निर्णयांपैकी एक आहे (धडा 10 याला one-way door म्हणतो), आणि unstructured interviews मध्ये सर्वात मोठ्या आवाजातील मत, किंवा interviewers सारखाच सर्वात जास्त दिसणारा उमेदवार, निर्णय घेतो. Rubric स्वतःहून hiring न्याय्य करत नाही — प्रश्न तरीही पक्षपाती असू शकतात — पण ती पुरावा दृश्य आणि तुलना करण्याजोगा करते, आणि स्वतंत्र गुण प्रत्येक interviewer चा पुरावा चर्चेत टिकवून ठेवतात.

🔧 कसे (या repo मध्ये)

lead/models.py मधले panel(scores) {interviewer: [scores 1–4]} घेते आणि प्रत्येक interviewer ची सरासरी आणि panel ची सरासरी परत देते. herd(scores, first, pull=0.5) first वगळता प्रत्येकीला तिच्या गुणांकडे pull इतक्या प्रमाणात सरकवते. calibrate(history) मागील panels ({interviewer: overall score}) घेते आणि प्रत्येक interviewer चा तिच्या panel च्या सरासरीपासूनचा सरासरी फरक परत देते — offsets ची बेरीज शून्य असते. lead/demo.py मधले hiring() एका उमेदवाराला तीन प्रकारे गुण देते आणि वेगवेगळ्या panels ने पाहिलेल्या दोन उमेदवारांची तुलना करते. Herding चा नियम आणि offsets हे विचार करण्याचे साधन आहे, माणसांसाठीचे सूत्र नाही — कोणाचाही hiring चा निर्णय script ने मोजला जाऊ नये.

🧪 करून पाहा

python3 lead/demo.py hiring
python3 - <<'EOF'
import sys; sys.path.insert(0, "lead"); from models import panel, herd
s = {"Katrina": [4, 4, 4, 3], "Dipika": [2, 3, 2, 2], "Aishwarya": [3, 2, 2, 3]}
for first in ("Katrina", "Dipika"):
    for pull in (0.25, 0.5):
        each, m = panel(herd(s, first, pull))
        print(f"{first} speaks first, pull {pull:.2f} → panel {m:.2f} → {'hire' if m >= 3 else 'no hire'}")
each, m = panel(s)
print(f"nobody speaks first (independent) → panel {m:.2f} → {'hire' if m >= 3 else 'no hire'}")
EOF

✅ तपासा — तुम्हाला काय दिसायला हवे

hiring असे print करते:

   independent: Katrina 3.75 · Dipika 2.25 · Aishwarya 2.50 → panel 2.83 → no hire
   after Katrina speaks first (others move halfway to her): Katrina 3.75 · Dipika 3.00 · Aishwarya 3.12 → panel 3.29 → hire
── calibration on 5 past panels: Katrina +0.37 · Dipika -0.33 · Aishwarya -0.03
   candidate X (Katrina + Aishwarya): raw 3.25 → calibrated 3.08
   candidate Y (Dipika + Aishwarya): raw 3.00 → calibrated 3.18

तुमचा snippet असे print करतो:

Katrina speaks first, pull 0.25 → panel 3.06 → hire
Katrina speaks first, pull 0.50 → panel 3.29 → hire
Dipika speaks first, pull 0.25 → panel 2.69 → no hire
Dipika speaks first, pull 0.50 → panel 2.54 → no hire
nobody speaks first (independent) → panel 2.83 → no hire

🏁 तुम्ही आत्ताच काय सिद्ध केले

पुराव्याच्या त्याच तीन संचांनी स्वतंत्रपणे गुण दिल्यावर 2.83 → no hire दिले, आणि कतरिना आधी बोलल्यावर 3.29 → hire — अगदी लहान pull (0.25) नेही bar ओलांडला. दीपिका आधी बोलली तेव्हा panel उलट्या दिशेने सरकले. बोलण्याच्या क्रमाने निर्णय बदलला; उमेदवार बदलला नाही. Calibration ने दुसरा परिणाम दाखवला: उमेदवार X अधिक सक्षम दिसला (3.25 विरुद्ध 3.00) ते फक्त X ला अधिक उदार interviewer भेटली म्हणून. फक्त 5 मागील panels असल्याने ते offsets ढोबळ आहेत — गुण देण्याच्या सवयींबद्दल बोलण्याची सूचना, आंधळेपणाने लागू करायची दुरुस्ती नव्हे.

⚠️ नेहमीच्या चुका

उमेदवारांना भेटल्यानंतर bar ठरवणे
"culture fit" एक गुण म्हणून — त्याचा अर्थ सहसा "आमच्यासारखी" असा होतो; नेमक्या, कामाशी संबंधित behaviours ना गुण द्या
debrief मध्ये सर्वात वरिष्ठ व्यक्तीने आधी बोलणे
गुण लिहिण्याआधी interviewers ने corridor मध्ये एकमेकींशी चर्चा करणे
labels आहेत पण anchors नाहीत अशी rubric ("3 = good") — प्रत्येकजण "good" वेगळ्या अर्थाने वाचते
एकाच भूमिकेसाठी वेगवेगळ्या उमेदवारांना वेगवेगळे प्रश्न

🏭 प्रत्यक्ष वापरात

Job description सोबत ठेवलेले interview scorecard, पहिल्या interview आधी लिहिलेले:

role: maths teacher (software engineer II)
bar: 3.0 average, no competency below 2
competencies:
  - name: testing approach
    question: "Walk me through how you would test the grade-export change."
    anchors:
      1: no testing mentioned
      2: tests after the fact, when prompted
      3: describes unit and integration tests unprompted
      4: chooses tests by risk and explains the trade-offs
  - name: explaining to a colleague
  - name: debugging an unfamiliar system
  - name: working through disagreement
process:
  - each interviewer submits scores and evidence before the debrief
  - the debrief starts with the most junior interviewer
  - the hiring manager decides; the reasons are written down

काही applicant-tracking systems तुम्ही स्वतःचा feedback submit करेपर्यंत इतर interviewers चा feedback लपवू देतात — तुमच्या system मध्ये ही सोय असेल तर ती चालू करा. दर काही महिन्यांनी सामायिक उमेदवारांवरील interviewers च्या गुणांची तुलना करा (calibration), आणि rubric चे गुण निवडलेल्या लोकांच्या प्रत्यक्ष कामगिरीशी जुळतात का ते तपासा.

🏭 प्रत्यक्ष वापरात हे का महत्त्वाचे आहे: पहिल्या उमेदवाराला भेटण्याआधी rubric आणि bar लिहा, debrief आधी गुण गोळा करा, आणि सर्वात शांत पुराव्याला आधी बोलू द्या. याला एक तास तयारी लागते आणि चांगले panels वाईट निर्णय घेतात त्याचा सर्वात नेहमीचा मार्ग बंद होतो.

⏭️ पुढे

भाग 1 पूर्ण झाला: नवे काम, 1:1s, delegation आणि hiring. भाग 2 स्वतः कामाबद्दल आहे. प्रत्येक विभागप्रमुखाला विचारला जाणारा पहिला प्रश्न: "हे कधी पूर्ण होईल?"

git checkout lesson-05-forecasting

🧑‍🏫 Lesson 04 — Hiring: a structured rubric, independent scores, calibration

📍 You are here: Lesson 04 of 12 · Previous: lesson-03-delegation · Next: lesson-05-forecasting

📦 What's in this branch

Lesson 03, plus hiring. A structured interview with a rubric (the same questions and anchored scores for every candidate), why each interviewer writes her scores before the debrief, and calibration — noticing that some interviewers score higher or lower than others. hiring() in lead/demo.py; panel(), herd() and calibrate() in lead/models.py.

🧒 Explain like I'm 5

The maths department needs a new teacher. Katrina, Dipika and Aishwarya each watch the candidate teach a sample lesson. 🧑‍🏫

Then they meet to decide. Katrina speaks first: "She was wonderful!" Dipika had doubts — the candidate could not explain how she checks pupils' understanding. But Katrina is the head, and she sounded so sure… so Dipika says, "Yes, pretty good." Aishwarya agrees too.

The candidate is hired. Dipika's doubt — the one real piece of evidence — never reached the table. 🙊

Next time, Katrina changes the rules:

Everyone uses the same score sheet, with clear descriptions for each score. 📋
Everyone writes her scores down before the meeting. ✍️
In the meeting, the most junior person speaks first.

Now the same three opinions add up to a different answer — the honest one.

🗺️ Diagram

flowchart LR
    s["📋 same rubric, 4 competencies<br/>scores 1–4, bar 3.0"] --> i["✍️ written before the debrief<br/>Katrina 3.75 · Dipika 2.25 · Aishwarya 2.50"]
    i --> p1["panel 2.83 → no hire"]
    s --> h["🗣️ Katrina speaks first,<br/>others move halfway to her"]
    h --> p2["panel 3.29 → hire"]
    c["⚖️ calibration, 5 past panels<br/>Katrina +0.37 · Dipika -0.33"] --> x["X raw 3.25 → 3.08<br/>Y raw 3.00 → 3.18"]

🗺️ Drawn version + a lab: https://school-edh.pages.dev/engineering-leadership/lesson-diagrams.html#l04

❓ What

Structured interview — every candidate for a role gets the same questions (or the same task), scored against the same rubric by trained interviewers. Unstructured: each interviewer asks what she likes and forms an overall impression. Large reviews of selection research — Schmidt and Hunter (1998), and a 2022 re-analysis by Sackett and colleagues — found structured interviews to be more predictive of job performance than unstructured ones, and among the strongest methods studied. "More predictive" is a statement about averages across many hires, not a guarantee about one.
Rubric with anchors — for each competency, a written description of what each score looks like, so a "3" means the same thing from every interviewer. Example for "testing approach": 1 no testing mentioned · 2 tests after the fact, when prompted · 3 describes unit and integration tests unprompted · 4 chooses tests by risk and explains the trade-offs.
Independent scoring — each interviewer writes her scores and evidence before hearing anyone else. Once people discuss, later opinions drift toward earlier and louder ones (anchoring, social pressure). Kahneman, Sibony and Sunstein's Noise (2021) recommends independent judgements before discussion for exactly this reason.
Herding (in this lab) — a teaching model of that drift: after hearing the first speaker, each other interviewer moves a fixed fraction (pull) of the way toward her scores. Real drift is messier; the point is the direction.
Calibration — interviewers differ in habit: some score high, some low. Comparing each person's scores with the rest of the panel on the same candidates estimates her offset. It matters most when different candidates meet different panels.
The bar — the minimum the role needs, agreed before interviews start.

🤔 Why

Because a hire is one of a lead's most expensive and hardest-to-reverse decisions (lesson 10 calls it a one-way door), and unstructured interviews let the loudest opinion, or the candidate who is most like the interviewers, decide. A rubric does not make hiring fair by itself — the questions can still be biased — but it makes the evidence visible and comparable, and independent scores keep each interviewer's evidence in the room.

🔧 How (in this repo)

panel(scores) in lead/models.py takes {interviewer: [scores 1–4]} and returns each interviewer's mean and the panel mean. herd(scores, first, pull=0.5) moves everyone except first a fraction pull of the way toward her scores. calibrate(history) takes past panels ({interviewer: overall score}) and returns each interviewer's average difference from her panel's mean — the offsets sum to zero. hiring() in lead/demo.py scores one candidate three ways and compares two candidates seen by different panels. The herding rule and the offsets are a thinking tool, not a formula for people — nobody's hiring decision should be computed by a script.

🧪 Try it

python3 lead/demo.py hiring
python3 - <<'EOF'
import sys; sys.path.insert(0, "lead"); from models import panel, herd
s = {"Katrina": [4, 4, 4, 3], "Dipika": [2, 3, 2, 2], "Aishwarya": [3, 2, 2, 3]}
for first in ("Katrina", "Dipika"):
    for pull in (0.25, 0.5):
        each, m = panel(herd(s, first, pull))
        print(f"{first} speaks first, pull {pull:.2f} → panel {m:.2f} → {'hire' if m >= 3 else 'no hire'}")
each, m = panel(s)
print(f"nobody speaks first (independent) → panel {m:.2f} → {'hire' if m >= 3 else 'no hire'}")
EOF

✅ Verify — what you should see

hiring prints:

   independent: Katrina 3.75 · Dipika 2.25 · Aishwarya 2.50 → panel 2.83 → no hire
   after Katrina speaks first (others move halfway to her): Katrina 3.75 · Dipika 3.00 · Aishwarya 3.12 → panel 3.29 → hire
── calibration on 5 past panels: Katrina +0.37 · Dipika -0.33 · Aishwarya -0.03
   candidate X (Katrina + Aishwarya): raw 3.25 → calibrated 3.08
   candidate Y (Dipika + Aishwarya): raw 3.00 → calibrated 3.18

Your snippet prints:

Katrina speaks first, pull 0.25 → panel 3.06 → hire
Katrina speaks first, pull 0.50 → panel 3.29 → hire
Dipika speaks first, pull 0.25 → panel 2.69 → no hire
Dipika speaks first, pull 0.50 → panel 2.54 → no hire
nobody speaks first (independent) → panel 2.83 → no hire

🏁 What you just proved

The same three sets of evidence gave 2.83 → no hire when scored independently and 3.29 → hire when Katrina spoke first — even a small pull (0.25) crossed the bar. When Dipika spoke first, the panel moved the other way. The order of speaking changed the decision; the candidate did not change. Calibration showed a second effect: candidate X looked stronger (3.25 vs 3.00) only because X met the more lenient interviewer. With only 5 past panels, those offsets are rough — a prompt to talk about scoring habits, not a correction to apply blindly.

⚠️ Common mistakes

deciding the bar after meeting the candidates
"culture fit" as a score — it usually means "like us"; score specific, job-related behaviours
the most senior person speaking first in the debrief
interviewers comparing notes in the corridor before writing scores
a rubric with labels but no anchors ("3 = good") — everyone reads "good" differently
different questions for different candidates for the same role

🏭 In production

An interview scorecard kept with the job description, written before the first interview:

role: maths teacher (software engineer II)
bar: 3.0 average, no competency below 2
competencies:
  - name: testing approach
    question: "Walk me through how you would test the grade-export change."
    anchors:
      1: no testing mentioned
      2: tests after the fact, when prompted
      3: describes unit and integration tests unprompted
      4: chooses tests by risk and explains the trade-offs
  - name: explaining to a colleague
  - name: debugging an unfamiliar system
  - name: working through disagreement
process:
  - each interviewer submits scores and evidence before the debrief
  - the debrief starts with the most junior interviewer
  - the hiring manager decides; the reasons are written down

Some applicant-tracking systems let you hide other interviewers' feedback until you have submitted your own — turn that on if yours does. Every few months, compare interviewers' scores on shared candidates (calibration), and check whether the rubric's scores relate to how hires actually did.

🏭 Why this matters in production: write the rubric and the bar before you meet the first candidate, collect scores before the debrief, and let the quietest evidence speak first. It costs an hour of preparation and removes the most common way good panels make bad decisions.

⏭️ Next

Part 1 is done: the new job, 1:1s, delegation and hiring. Part 2 is about the work itself. First question every head of department is asked: "when will it be done?"

git checkout lesson-05-forecasting

← Previousdelegation Next →forecasting

This page is the lesson's README from the lesson-04-hiring branch, shown here so the whole School stays on one site. Code files open on GitHub at the same branch.