🚧 धडा 05 — Guardrails: सहलीचे नियम

📍 तुम्ही इथे आहात: 8 पैकी धडा 05 · मागे: lesson-04-memory · पुढे: lesson-06-failures

📦 या ब्रँचमध्ये काय आहे

धडे 01–04, आणि न थकणाऱ्या loop च्या हाती खऱ्या क्षमता देणे सुरक्षित करणारे नियम — प्रत्येक नियम agent/agent.py मध्ये implement केलेला आहे.

🧒 5 वर्षांच्या मुलाला समजावल्यासारखे

शाळा विद्यार्थ्यांना कामांसाठी बाहेर जाऊ देते — पण दारावर सहलीचे नियम चिकटवून. हे आपले नियम, आणि प्रत्येक code मध्ये कुठे आहे:

🕐 जेवणापर्यंत परत या (MAX_STEPS = 8): काम कसेही चालो, N steps नंतर तुम्ही थांबता. कोणताही विद्यार्थी कायम भटकत नाही; कोणताही loop एका रात्रीत महिन्याभराचे tokens जाळत नाही. मर्यादा गाठली की अभिमानाने "🛑 guardrail, not a bug" छापले जाते — कारण तुम्ही design केलेला थांबा हेच यश आहे.
🚧 कायमच्या marker ने करायच्या कृतींसाठी शिक्षिका लागते (write gate): "writes": True असा flag असलेली tools चालण्याआधी माणसाच्या [y/N] साठी थांबतात. वाचन वाहते; लिहिणे थांबते. एक if statement — बहुधा file मधली सर्वात मौल्यवान ओळ.
🔒 फसवता न येणारी tools (_safe_eval): tool चे inputs अविश्वसनीय text वाचणाऱ्या, सहज पटवता येणाऱ्या model कडून येतात — म्हणून calculator eval() ऐवजी AST allowlist parse करतो. caller गोंधळलेला असू शकतो असे गृहीत धरून tools बनवा, कारण कधी कधी तो असतोच (धडा 06).
🧾 प्रत्येक step ची नोंद (छापलेला log): think, act, observe — क्रमांकित, दिसणारे, पुन्हा चालवता येणारे. काही चुकले की log "काय घडले?" याचे उत्तर देतो (CloudTrail ची सवय — AWS L06).

आणि production त्यावर जोडते ते नियम: budgets (tokens/पैसे/वेळ), sandboxes (containers! — Docker course अचानक परत येतो 🍱), scoped credentials (agent ची tools least-privilege च्या टोप्या घालतात, AWS L03–05), आणि माणसांना नेहमी पोहोचता येतील असे kill switches.

🗺️ आकृती

flowchart TB
    agent["🔁 the loop wants to run forever,<br/>touch everything, trust every input"]
    g1["🕐 MAX_STEPS + budgets<br/>bounded loops, bounded bills"]
    g2["🚧 write gates<br/>reads flow · writes wait for a human"]
    g3["🔒 hardened tools<br/>allowlists, not eval() · least-privilege creds"]
    g4["🧾 full logs<br/>every think/act/observe, replayable"]
    g5["📦 sandboxes<br/>blast radius = one container"]
    agent --> g1
    agent --> g2
    agent --> g3
    agent --> g4
    agent --> g5

❓ काय

gate चे वर्गीकरण: आपोआप परवानगी (reads) · खात्री करा (writes) · मनाई (कधीच-नको tools: delete_all, transfer_money — ती पट्ट्यावर ठेवूच नका). पट्टा हाच permission model आहे.
--yes (आपला auto-approve flag) demos आणि CI साठी आहे — आणि खऱ्या hosts मध्ये ज्यावर अविश्वास ठेवायला हवा तो नेमका "always allow" toggle हाच आहे (MCP शाळा L07 मध्येही हेच सांगितले: तो toggle माणसाचे gate काढून टाकतो).
Guardrails MCP सोबत जुळतात: servers जाहीर करतात, hosts gate करतात, agents log करतात — तीन थर, एकच तत्त्वज्ञान.
अस्वस्थ करणारे सत्य: guardrails ची किंमत सोयीत मोजावी लागते. जे teams ते "तात्पुरते" काढतात ते case studies बनतात. या अडथळ्याचा हिशोब आधीच धरा.

🤔 का

धडा 01 ने परिणामांचे वचन दिले; परिणाम म्हणजे जगाला हात लावणे; जगाला हात लावणे म्हणजे failure modes चे परिणाम भोगावे लागतात. "छान demo" आणि "deploy केलेला agent" यांतला फरक चांगले model नाही — तो आहे हे पाच कंटाळवाणे नियम, अस्तित्वात असलेले आणि test केलेले. कंटाळवाणे असणे हेच ध्येय (k8s course चे ब्रीदवाक्य परत येते).

🧪 करून पाहा — प्रत्येक नियम अनुभवा

# rule 2 — the gate, live:
python3 agent/agent.py            # at the GATE prompt, answer N
# → "denied by the human — pick another way": the loop ADAPTS, goal continues

# rule 1 — the curfew:
python3 agent/agent.py --drive    # just keep calling lookup, 8 times
# → "🛑 stopped: hit MAX_STEPS" — the wandering student comes home

# rule 3 — the hardened tool:
# your move> calc {"expression": "__import__('os').system('echo pwned')"}
# → tool error — the allowlist held 🔒

⏭️ पुढे

चुकते तेव्हा प्रत्यक्षात काय चुकते: failure modes — compounding errors, loops, injection — आणि evals त्यांना कसे पकडतात.

git checkout lesson-06-failures

🚧 Lesson 05 — Guardrails: the field-trip rules

📍 You are here: Lesson 05 of 8 · Previous: lesson-04-memory · Next: lesson-06-failures

📦 What's in this branch

Lessons 01–04, plus the rules that make it safe to hand a tireless loop real capabilities — every one implemented in agent/agent.py.

🧒 Explain like I'm 5

The school lets students run errands — WITH field-trip rules taped to the door. Ours, and where each lives in the code:

🕐 Be back by lunch (MAX_STEPS = 8): however the errand goes, you stop after N steps. No student wanders forever; no loop burns a month of tokens overnight. Hitting the limit prints a proud "🛑 guardrail, not a bug" — because a stop you designed IS success.
🚧 Permanent-marker actions need the teacher (the write gate): tools flagged "writes": True pause for a human [y/N] before running. Reads flow; writes wait. One if statement — arguably the most valuable line in the file.
🔒 Tools that can't be tricked (_safe_eval): tool inputs come from a persuadable model reading untrusted text — so the calculator parses an AST allowlist instead of eval(). Build tools as if the caller might be confused, because sometimes it is (lesson 06).
🧾 Every step on the record (the printed log): think, act, observe — numbered, visible, replayable. When something goes wrong, the log answers "what happened?" (CloudTrail instincts — AWS L06).

And the rules production adds on top: budgets (tokens/money/time), sandboxes (containers! — the Docker course suddenly reappears 🍱), scoped credentials (an agent's tools wear least-privilege hats, AWS L03–05), and kill switches humans can always reach.

🗺️ Diagram

flowchart TB
    agent["🔁 the loop wants to run forever,<br/>touch everything, trust every input"]
    g1["🕐 MAX_STEPS + budgets<br/>bounded loops, bounded bills"]
    g2["🚧 write gates<br/>reads flow · writes wait for a human"]
    g3["🔒 hardened tools<br/>allowlists, not eval() · least-privilege creds"]
    g4["🧾 full logs<br/>every think/act/observe, replayable"]
    g5["📦 sandboxes<br/>blast radius = one container"]
    agent --> g1
    agent --> g2
    agent --> g3
    agent --> g4
    agent --> g5

❓ What

The gate taxonomy: auto-allow (reads) · confirm (writes) · forbid (never-tools: delete_all, transfer_money — don't put them on the belt at all). The belt IS the permission model.
--yes (our auto-approve flag) exists for demos and CI — and is exactly the "always allow" toggle you should distrust in real hosts (MCP school L07 said the same: that toggle deletes the human gate).
Guardrails compose with MCP: servers declare, hosts gate, agents log — three layers, same philosophy.
The uncomfortable truth: guardrails cost convenience. Teams that remove them "temporarily" become case studies. Budget the friction.

🤔 Why

Lesson 01 promised outcomes; outcomes mean touching the world; touching the world means the failure modes have consequences. The difference between "cool demo" and "deployed agent" is not a better model — it's these five boring rules, present and tested. Boring is the goal (the k8s course's motto returns).

🧪 Try it — feel each rule

# rule 2 — the gate, live:
python3 agent/agent.py            # at the GATE prompt, answer N
# → "denied by the human — pick another way": the loop ADAPTS, goal continues

# rule 1 — the curfew:
python3 agent/agent.py --drive    # just keep calling lookup, 8 times
# → "🛑 stopped: hit MAX_STEPS" — the wandering student comes home

# rule 3 — the hardened tool:
# your move> calc {"expression": "__import__('os').system('echo pwned')"}
# → tool error — the allowlist held 🔒

⏭️ Next

What actually goes wrong when it goes wrong: failure modes — compounding errors, loops, injection — and how evals catch them.

git checkout lesson-06-failures

← Previousmemory Next →failures

This page is the lesson's README from the lesson-05-guardrails branch, shown here so the whole School stays on one site. Code files open on GitHub at the same branch.