🖼️ See the drawing + lab 🏠 Course home 🌿 Branch on GitHub ✏️ View source

🖼️ आकृती आणि labThe drawing + lab पूर्ण पानावर उघडा ↗Open full page ↗

🧵 धडा 08 — Concurrency आणि GIL: एक बॅटन, अनेक धावपटू

📍 तुम्ही इथे आहात: 12 पैकी धडा 08 · मागे: lesson-07-io-batching · पुढे: lesson-09-queueing

📦 या ब्रँचमध्ये काय आहे

धडे 01–07, आणि concurrency: एकाच वेळी अनेक कामे करणे. पहिला प्रश्न नेहमी हाच असतो: काम कशाची वाट पाहत आहे. I/O-bound काम उत्तरांची वाट पाहते, आणि threads किंवा asyncio ही वाट पाहणे एकमेकांवर चढवू (overlap) शकतात. CPU-bound कामाला processor लागतो, आणि standard CPython मध्ये एका वेळी एकच thread Python code चालवतो — हाच GIL. त्यासाठी तुम्हाला processes (किंवा free-threaded build) लागतात. perf/demo.py मधील concurrency() आणि perf/sim.py मधील schedule — हे tick-by-tick चालणारे शिकवण्यासाठीचे model आहे, खरे threads नव्हेत.

🧒 5 वर्षांच्या मुलाला समजावल्यासारखे

रिले संघात 8 धावपटू आहेत आणि एकच बॅटन. 🥢 या track वर धावपटू फक्त बॅटन हातात असतानाच धावू शकते.

पाण्याच्या टेबलाची कामे (I/O-bound): प्रत्येक धावपटू 5 पावले जॉग करते, मग पाण्याच्या टेबलाजवळ तिच्या पेयासाठी 100 सेकंद थांबते. 🚰 थांबायला बॅटन लागत नाही! म्हणून धावपटू 1 तिची 5 पावले जॉग करते, बॅटन खाली ठेवते, आणि थांबायला जाते; धावपटू 2 ते उचलते, आणि असेच पुढे. सगळ्या 8 जणी एकाच वेळी थांबतात. एकूण: सुमारे 140, 840 नव्हे. 🎉
Sprint ची कामे (CPU-bound): प्रत्येक धावपटूला 100 पावले sprint करावे लागते. Sprint ला बॅटन लागते. एकच बॅटन असेल तर, कितीही धावपटू असोत, sprints एकामागून एक होतात: 800. 😩

Sprints वेगवान कसे करायचे? अधिक tracks वापरा, प्रत्येकावर स्वतःचे बॅटन (अनेक cores वर processes). किंवा असा नवा track जिथे प्रत्येक धावपटू एकाच वेळी धावू शकते (free-threaded build). मात्र दुसऱ्या tracks वर पोहोचायला आधी थोडा वेळ लागतो.

🗺️ आकृती

flowchart LR
    io["🚰 I/O-bound × 8<br/>5 ms Python + 100 ms waiting"] --> seq1["one after another: 840 ms"]
    io --> thr1["8 threads (GIL): 140 ms"]
    io --> as1["asyncio: 140 ms"]
    cpu["🏃 CPU-bound × 8<br/>100 ms Python"] --> seq2["one after another: 800 ms"]
    cpu --> thr2["8 threads (GIL): 800 ms"]
    cpu --> pr["4 processes: 250 ms"]
    cpu --> ft["free-threaded, 4 cores: 200 ms"]

🗺️ रेखाटलेली आवृत्ती + एक lab: https://school-edh.pages.dev/performance/lesson-diagrams.html#l08

❓ काय

Concurrency विरुद्ध parallelism — concurrency म्हणजे एकाच वेळी अनेक tasks हाताळणे (त्यांच्यात आलटून पालटून जाणे); parallelism म्हणजे एकाच क्षणी अनेक tasks करणे (अनेक CPU cores). वाट पाहणे फक्त concurrency नेही overlap होते; गणनेला parallelism लागतो.
I/O-bound — वेळ मुख्यतः वाट पाहण्यात जातो (network, disk, database). CPU-bound — वेळ मुख्यतः गणनेत जातो. Profile (धडा 03) किंवा फक्त CPU time ची wall time शी तुलना केल्यावर कोणते ते कळते.
GIL (global interpreter lock) — standard CPython build मध्ये एक lock जो एका वेळी एकाच thread ला Python bytecode चालवू देतो. Thread I/O ची वाट पाहत असताना (sockets, files, time.sleep) CPython तो सोडतो, आणि अनेक C extensions लांब गणनेदरम्यान तो सोडतात (उदा. मोठ्या data वर hashlib, अनेक operations मध्ये NumPy). तो धरून ठेवलेल्या thread ला switch interval नंतर (sys.getswitchinterval(), default 5 ms) तो सोडायला सांगितले जाते.
Threads (threading, concurrent.futures.ThreadPoolExecutor) — memory share करतात; I/O overlap करण्यासाठी उत्तम; standard build मध्ये शुद्ध-Python CPU काम वेगवान करत नाहीत.
asyncio — एक thread, अनेक tasks जे प्रत्येक await वर नियंत्रण सोडतात. प्रत्येक task खूप स्वस्त, हजारो थांबलेल्या connections साठी चांगले. Coroutine मधील CPU-जड function संपेपर्यंत प्रत्येक task अडवते (ते asyncio.to_thread किंवा process pool ने बाहेर हलवा).
Processes (multiprocessing, ProcessPoolExecutor) — वेगवेगळे interpreters, प्रत्येकाचा स्वतःचा GIL, म्हणून CPU काम अनेक cores वर parallel चालते. खर्च: processes सुरू करणे, आणि arguments व निकाल processes दरम्यान पाठवण्यासाठी त्यांचे pickling.
Free-threaded CPython — GIL शिवायचा build (PEP 703). तो Python 3.13 मध्ये experimental म्हणून आला (बहुधा python3.13t म्हणून install होतो) आणि 3.14 मध्ये अधिकृतपणे supported, पण अजूनही optional झाला (PEP 779). Default download मध्ये अजूनही GIL आहे. sys._is_gil_enabled() (3.13+) तुम्ही कोणता चालवत आहात ते सांगतो; support जाहीर न केलेला C extension import केल्यास GIL पुन्हा चालू होऊ शकतो.

🤔 का

कारण "threads वाढवा" हा performance चा सर्वात सामान्य सल्ला आहे आणि तो अनेकदा चुकीचा असतो. Web scraper साठी threads किंवा asyncio मोठी speed-up देऊ शकतात. शुद्ध Python मधल्या आकडेमोडीच्या loop साठी, standard build मधले threads काहीच speed-up देत नाहीत, आणि processes प्रत्येकी जास्तीत जास्त एका core इतका फायदा देतात — start-up आणि data copying वजा करून. वाट पाहण्याच्या प्रकारानुसार निवडल्याने खूप वाया जाणारे काम वाचते.

🔧 कसे (या repo मध्ये)

perf/sim.py मधील schedule(tasks, workers, cpu_slots, start_ms) एक tick-by-tick model चालवते: प्रत्येक task म्हणजे phases ची यादी, ("cpu", ms) किंवा ("io", ms). एका वेळी जास्तीत जास्त workers tasks चालू असतात; वाट पाहण्याला (io) CPU लागत नाही; एकाच millisecond मध्ये फक्त cpu_slots tasks गणना करू शकतात — GIL सह threads किंवा asyncio साठी 1, processes किंवा free-threaded build साठी cores ची संख्या — round-robin पद्धतीने वाटून. प्रत्येक process एकदाच start_ms मोजते. खरे CPython प्रत्येक millisecond ला नव्हे तर दर काही milliseconds ला switch करते, आणि खऱ्या I/O व start-up च्या वेळा बदलत असतात; पुढे टिकतो तो निकालांचा आकार.

🧪 करून पाहा

python3 perf/demo.py concurrency
python3 - <<'EOF'
import sys; sys.path.insert(0, "perf"); from sim import schedule
mixed = [[("cpu", 20), ("io", 80)]] * 8
print("8 mixed tasks (20 ms Python + 80 ms waiting):")
for label, w, slots, start in (("one after another", 1, 1, 0), ("8 threads, GIL", 8, 1, 0), ("4 processes", 4, 4, 50), ("8 processes", 8, 4, 50)):
    print(f"  {label:<18} {schedule(mixed, w, slots, start):>4} ms")
for n in (2, 4, 8, 16):
    print(f"CPU-bound × 8 on {n:>2} threads with the GIL: {schedule([[('cpu', 100)]] * 8, n, 1)} ms")
EOF
python3 - <<'EOF'
import time, threading
def wait(): time.sleep(0.1)
t0 = time.perf_counter(); [wait() for _ in range(8)]; seq = time.perf_counter() - t0
ts = [threading.Thread(target=wait) for _ in range(8)]
t0 = time.perf_counter(); [t.start() for t in ts]; [t.join() for t in ts]; thr = time.perf_counter() - t0
print(f"real: 8 sleeps of 0.1 s → one after another {seq:.2f} s · 8 threads {thr:.2f} s   (your numbers will differ)")
EOF

शेवटचा snippet खरे threads आणि खरे घड्याळ वापरतो — साधारण 0.82 s विरुद्ध 0.11 s. तुमचे आकडे थोडे वेगळे असतील; 8× चा आकार बदलणार नाही, कारण sleep GIL सोडतो.

✅ तपासा — तुम्हाला काय दिसायला हवे

concurrency हे छापते:

── 8 tasks · I/O-bound: 5 ms of Python + 100 ms waiting for a reply · CPU-bound: 100 ms of Python (a teaching model)
                                             I/O-bound   CPU-bound
   one after another                           840 ms      800 ms
   8 threads, one baton (the GIL)              140 ms      800 ms
   asyncio, 1 thread, 8 tasks                  140 ms      800 ms
   4 processes on 4 cores (+50 ms start)       260 ms      250 ms
   8 threads, free-threaded build, 4 cores     110 ms      200 ms

तुमचा पहिला snippet हे छापतो:

8 mixed tasks (20 ms Python + 80 ms waiting):
  one after another   800 ms
  8 threads, GIL      240 ms
  4 processes         250 ms
  8 processes         170 ms
CPU-bound × 8 on  2 threads with the GIL: 800 ms
CPU-bound × 8 on  4 threads with the GIL: 800 ms
CPU-bound × 8 on  8 threads with the GIL: 800 ms
CPU-bound × 8 on 16 threads with the GIL: 800 ms

🏁 तुम्ही आत्ताच काय सिद्ध केले

GIL सह, CPU कामाला 2 threads असोत की 16, 800 ms लागले — Python काम फक्त आळीपाळीने होते. वाट पाहणे पूर्णपणे overlap झाले: 840 → 140 ms. Processes नी CPU ची शर्यत जिंकली (250 ms) पण I/O ची शर्यत हरली (260 ms विरुद्ध 140 ms), कारण त्यांचा start-up खर्च आणि फक्त 4 workers. मिश्र कामात, GIL मुळे 8 × 20 ms = 160 ms Python एकाच रांगेत राहिले, अधिक एक 80 ms ची प्रतीक्षा: 240 ms. तुमचे काम कशाची वाट पाहते ते ओळखा, मग निवडा.

⚠️ नेहमीच्या चुका

standard build वर शुद्ध-Python CPU कामासाठी threads, speed-up ची अपेक्षा ठेवून
async def मध्ये CPU-जड किंवा blocking call (requests.get, time.sleep) — बाकी प्रत्येक task थांबतो
प्रत्येक छोट्या task साठी एक process — start-up आणि pickling चा खर्च कामापेक्षा जास्त होतो
प्रत्येक call वर worker processes ना मोठे objects पाठवणे
"GIL मुळे माझा code thread-safe होतो" — होत नाही; दोन threads मधून count += 1 केल्यास अजूनही updates हरवू शकतात, म्हणून locks किंवा queues वापरा
प्रत्येक Python ला GIL आहे, किंवा कोणालाच नाही, असे गृहीत धरणे — 3.13+ वर sys._is_gil_enabled() तपासा

🏭 प्रत्यक्ष वापरात

खऱ्या machine वर — standard library तिन्ही प्रकार हाताळते:

from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
import asyncio

with ThreadPoolExecutor(max_workers=16) as pool:          # I/O-bound: overlap the waits
    pages = list(pool.map(fetch_page, urls))

with ProcessPoolExecutor() as pool:                       # CPU-bound: one process per core
    scores = list(pool.map(score_heat, heats, chunksize=64))

async def main():                                         # thousands of waiting connections
    return await asyncio.gather(*(fetch_async(u) for u in urls))
asyncio.run(main())

तुमच्याकडे कोणता interpreter आहे ते तपासा, आणि free-threaded build (3.13+) वापरून पाहा:

python3 -c "import sys; print(sys.version); print('GIL enabled:', getattr(sys, '_is_gil_enabled', lambda: True)())"
python3.13t -X gil=0 my_cpu_job.py        # a free-threaded build, if installed; PYTHON_GIL=0 does the same

Web service साठी server चे worker model हे तुमच्यासाठी ठरवते — उदाहरणार्थ अनेक worker processes (parallel CPU) असलेले Gunicorn, प्रत्येकात threads किंवा async loop (I/O overlap करणारे):

gunicorn app:app --workers 4 --threads 8

🏭 प्रत्यक्ष वापरात हे का महत्त्वाचे: process च्या CPU time ची त्याच्या wall time शी तुलना करा. जर एका core वर CPU ≈ wall असेल, तर ते CPU-bound आहे: processes, चांगले algorithms, किंवा native code चा विचार करा. जर CPU ≪ wall असेल, तर ते वाट पाहत आहे: प्रतीक्षा overlap करा, त्यांचे batching करा (धडा 07), किंवा cache करा (धडा 06).

⏭️ पुढे

एकाच पाण्याच्या टेबलाचा वापर अधिक धावपटू करू लागल्या की एक नवी समस्या दिसते: रांग. पुढे: utilisation विरुद्ध प्रतीक्षेचा वेळ, Little चा नियम, आणि Amdahl चा सर्वात हळू धावपटू.

git checkout lesson-09-queueing

🧵 Lesson 08 — Concurrency & the GIL: one baton, many runners

📍 You are here: Lesson 08 of 12 · Previous: lesson-07-io-batching · Next: lesson-09-queueing

📦 What's in this branch

Lessons 01–07, plus concurrency: doing several things at once. The first question is always what the work is waiting for. I/O-bound work waits for replies, and threads or asyncio can overlap the waiting. CPU-bound work needs the processor, and in standard CPython only one thread runs Python code at a time — the GIL. For that you need processes (or a free-threaded build). concurrency() in perf/demo.py and schedule in perf/sim.py — a tick-by-tick teaching model, not real threads.

🧒 Explain like I'm 5

The relay team has 8 runners and one baton. 🥢 On this track, a runner may only run while holding the baton.

Water-table jobs (I/O-bound): each runner jogs 5 steps, then waits 100 seconds at the water table for her drink. 🚰 Waiting does not need the baton! So runner 1 jogs her 5 steps, drops the baton, and goes to wait; runner 2 picks it up, and so on. All 8 wait at the same time. Total: about 140, not 840. 🎉
Sprint jobs (CPU-bound): each runner must sprint 100 steps. Sprinting needs the baton. With one baton, the sprints happen one after another, however many runners you have: 800. 😩

How do you make sprints faster? Use more tracks, each with its own baton (processes on several cores). Or a new kind of track where every runner can run at once (the free-threaded build). Getting to the other tracks takes a little time first, though.

🗺️ Diagram

flowchart LR
    io["🚰 I/O-bound × 8<br/>5 ms Python + 100 ms waiting"] --> seq1["one after another: 840 ms"]
    io --> thr1["8 threads (GIL): 140 ms"]
    io --> as1["asyncio: 140 ms"]
    cpu["🏃 CPU-bound × 8<br/>100 ms Python"] --> seq2["one after another: 800 ms"]
    cpu --> thr2["8 threads (GIL): 800 ms"]
    cpu --> pr["4 processes: 250 ms"]
    cpu --> ft["free-threaded, 4 cores: 200 ms"]

🗺️ Drawn version + a lab: https://school-edh.pages.dev/performance/lesson-diagrams.html#l08

❓ What

Concurrency vs parallelism — concurrency is dealing with many tasks at once (switching between them); parallelism is doing many at the same instant (several CPU cores). Waiting overlaps with concurrency alone; computing needs parallelism.
I/O-bound — the time is mostly waiting (network, disk, database). CPU-bound — the time is mostly computing. A profile (lesson 03) or simply comparing CPU time to wall time tells you which.
The GIL (global interpreter lock) — in the standard CPython build, a lock that lets only one thread at a time execute Python bytecode. CPython releases it while a thread waits on I/O (sockets, files, time.sleep) and many C extensions release it during long computations (for example hashlib on large data, NumPy in many operations). A thread holding it is asked to let go after a switch interval (sys.getswitchinterval(), 5 ms by default).
Threads (threading, concurrent.futures.ThreadPoolExecutor) — share memory; great for overlapping I/O; do not speed up pure-Python CPU work in the standard build.
asyncio — one thread, many tasks that give up control at every await. Very cheap per task, good for thousands of waiting connections. A CPU-heavy function inside a coroutine blocks every task until it finishes (move it out with asyncio.to_thread or a process pool).
Processes (multiprocessing, ProcessPoolExecutor) — separate interpreters, each with its own GIL, so CPU work runs in parallel on several cores. Costs: starting processes, and pickling arguments and results to send them between processes.
Free-threaded CPython — a build without the GIL (PEP 703). It shipped as experimental in Python 3.13 (often installed as python3.13t) and became officially supported, but still optional in 3.14 (PEP 779). The default download still has the GIL. sys._is_gil_enabled() (3.13+) tells you which one you are running; importing a C extension that does not declare support can turn the GIL back on.

🤔 Why

Because "add threads" is the most common performance advice and it is often wrong. For a web scraper, threads or asyncio can give a big speed-up. For a number-crunching loop in pure Python, threads in the standard build give none, and processes give up to one core's worth each — minus start-up and data copying. Choosing by the kind of waiting saves a lot of wasted work.

🔧 How (in this repo)

schedule(tasks, workers, cpu_slots, start_ms) in perf/sim.py runs a tick-by-tick model: each task is a list of phases, ("cpu", ms) or ("io", ms). Up to workers tasks are in progress at once; waiting (io) needs no CPU; only cpu_slots tasks may compute in the same millisecond — 1 for threads or asyncio with the GIL, the number of cores for processes or a free-threaded build — shared round-robin. Each process pays start_ms once. Real CPython switches every few milliseconds, not every one, and real I/O and start-up times vary; the shape of the results is what carries over.

🧪 Try it

python3 perf/demo.py concurrency
python3 - <<'EOF'
import sys; sys.path.insert(0, "perf"); from sim import schedule
mixed = [[("cpu", 20), ("io", 80)]] * 8
print("8 mixed tasks (20 ms Python + 80 ms waiting):")
for label, w, slots, start in (("one after another", 1, 1, 0), ("8 threads, GIL", 8, 1, 0), ("4 processes", 4, 4, 50), ("8 processes", 8, 4, 50)):
    print(f"  {label:<18} {schedule(mixed, w, slots, start):>4} ms")
for n in (2, 4, 8, 16):
    print(f"CPU-bound × 8 on {n:>2} threads with the GIL: {schedule([[('cpu', 100)]] * 8, n, 1)} ms")
EOF
python3 - <<'EOF'
import time, threading
def wait(): time.sleep(0.1)
t0 = time.perf_counter(); [wait() for _ in range(8)]; seq = time.perf_counter() - t0
ts = [threading.Thread(target=wait) for _ in range(8)]
t0 = time.perf_counter(); [t.start() for t in ts]; [t.join() for t in ts]; thr = time.perf_counter() - t0
print(f"real: 8 sleeps of 0.1 s → one after another {seq:.2f} s · 8 threads {thr:.2f} s   (your numbers will differ)")
EOF

The last snippet uses real threads and a real clock — something like 0.82 s vs 0.11 s. Your numbers will differ a little; the 8× shape will not, because sleep releases the GIL.

✅ Verify — what you should see

concurrency prints:

── 8 tasks · I/O-bound: 5 ms of Python + 100 ms waiting for a reply · CPU-bound: 100 ms of Python (a teaching model)
                                             I/O-bound   CPU-bound
   one after another                           840 ms      800 ms
   8 threads, one baton (the GIL)              140 ms      800 ms
   asyncio, 1 thread, 8 tasks                  140 ms      800 ms
   4 processes on 4 cores (+50 ms start)       260 ms      250 ms
   8 threads, free-threaded build, 4 cores     110 ms      200 ms

Your first snippet prints:

8 mixed tasks (20 ms Python + 80 ms waiting):
  one after another   800 ms
  8 threads, GIL      240 ms
  4 processes         250 ms
  8 processes         170 ms
CPU-bound × 8 on  2 threads with the GIL: 800 ms
CPU-bound × 8 on  4 threads with the GIL: 800 ms
CPU-bound × 8 on  8 threads with the GIL: 800 ms
CPU-bound × 8 on 16 threads with the GIL: 800 ms

🏁 What you just proved

With the GIL, CPU work took 800 ms whether there were 2 threads or 16 — the Python work simply takes turns. Waiting overlapped perfectly: 840 → 140 ms. Processes won the CPU race (250 ms) but lost the I/O race (260 ms vs 140 ms) because of their start-up cost and only 4 workers. For mixed work, the GIL left 8 × 20 ms = 160 ms of Python in a single line, plus one 80 ms wait: 240 ms. Know what your work waits for, then choose.

⚠️ Common mistakes

threads for pure-Python CPU work on the standard build, expecting a speed-up
a CPU-heavy or blocking call (requests.get, time.sleep) inside an async def — every other task stops
a process per tiny task — start-up and pickling cost more than the work
sending huge objects to worker processes on every call
"the GIL makes my code thread-safe" — it does not; count += 1 from two threads can still lose updates, so use locks or queues
assuming every Python has a GIL, or that none does — check sys._is_gil_enabled() on 3.13+

🏭 In production

On a real machine — the standard library covers the three shapes:

from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
import asyncio

with ThreadPoolExecutor(max_workers=16) as pool:          # I/O-bound: overlap the waits
    pages = list(pool.map(fetch_page, urls))

with ProcessPoolExecutor() as pool:                       # CPU-bound: one process per core
    scores = list(pool.map(score_heat, heats, chunksize=64))

async def main():                                         # thousands of waiting connections
    return await asyncio.gather(*(fetch_async(u) for u in urls))
asyncio.run(main())

Check which interpreter you have, and try a free-threaded build (3.13+):

python3 -c "import sys; print(sys.version); print('GIL enabled:', getattr(sys, '_is_gil_enabled', lambda: True)())"
python3.13t -X gil=0 my_cpu_job.py        # a free-threaded build, if installed; PYTHON_GIL=0 does the same

For a web service, the server's worker model decides this for you — for example Gunicorn with several worker processes (CPU in parallel), each with threads or an async loop (overlapping I/O):

gunicorn app:app --workers 4 --threads 8

🏭 Why this matters in production: compare a process's CPU time with its wall time. If CPU ≈ wall on one core, it is CPU-bound: think processes, better algorithms, or native code. If CPU ≪ wall, it is waiting: overlap the waits, batch them (lesson 07), or cache (lesson 06).

⏭️ Next

With more runners sharing one water table, a new problem appears: the queue. Next: utilisation vs waiting time, Little's law, and Amdahl's slowest runner.

git checkout lesson-09-queueing

← Previousio batching Next →queueing

This page is the lesson's README from the lesson-08-concurrency branch, shown here so the whole School stays on one site. Code files open on GitHub at the same branch.