Threads, the GIL & concurrent.futures

Doing more than one thing at once

Every real service waits: on a database, an HTTP API, a file, a message queue. If you fetch 100 URLs one at a time, your program spends nearly all its wall-clock time blocked, doing nothing. Concurrency lets those waits overlap, so 100 slow calls finish in roughly the time of the slowest one instead of the sum of all of them. Choosing the right model (threads, processes, or async) is a classic interview question because the wrong choice makes code either no faster or outright wrong.

The reason overlapping waits pays off so enormously is that the costs are not close to each other. Each rung below is roughly ten times the one before it:

What a Python program is actually waiting on

Step 1 / 5

CPU instruction~1 ns

Main memory read

SSD read

Datacenter round trip

Internet API call

CPU instruction: ~1 ns. One bytecode step. The GIL serialises these across threads.

Read the gap between the bottom rung and the top one and the decision rule below almost writes itself: if your program is sitting on the top rung, giving it more cores changes nothing, but letting the waits overlap changes everything.

The GIL: one bytecode at a time

CPython protects its internal memory with a Global Interpreter Lock. Only one thread executes Python bytecode at any instant, so pure-Python threads never run in parallel across cores. What rescues threading is that the GIL is released during blocking I/O (and inside many C extensions like numpy). While one thread waits on a socket, it drops the lock and another thread runs. That gives you the decision rule:

• I/O-bound work (network, disk, waiting): threads help, because the waits overlap.
• CPU-bound work (hashing, parsing, pure-Python math): threads do not help. Use ProcessPoolExecutor (separate processes, each with its own GIL) or push the work into a native library.

concurrent.futures: one API for both

concurrent.futures gives threads and processes the same interface. The common pattern maps a function over inputs:

from concurrent.futures import ThreadPoolExecutor

def double(n):
    return n * 2

with ThreadPoolExecutor(max_workers=4) as executor:
    print(list(executor.map(double, [1, 2, 3])))   # [2, 4, 6]

executor.map returns results in input order, not completion order, even though the tasks finish out of order. Swap in ProcessPoolExecutor and the code is identical. For finer control, executor.submit(fn, x) returns a Future, and as_completed(futures) yields them as they finish. Two things to remember about map: it returns a lazy iterator, so wrap it in list(...) when you need a real list (the exercise does), and it re-raises a worker's exception when you iterate to that result, not when you call map.

Pitfall: threads sharing state

Independent tasks are safe to parallelize. Shared mutable state is not. count += 1 is read, add, write: three steps, and the interpreter can switch threads between them, so two threads read the same value and one increment is lost. The GIL does not make your code thread-safe. Fix it with a Lock, or better, design the work so tasks never touch shared state (as double does here).

Running where there are no threads

This in-browser sandbox (Pyodide/WASM) has no OS threads, so building a pool raises RuntimeError: can't start new thread. A portable run_all tries the pool and falls back to a sequential map, producing identical ordered results everywhere:

try:
    with ThreadPoolExecutor(max_workers=4) as executor:
        return list(executor.map(double, numbers))
except RuntimeError:
    return [double(n) for n in numbers]

Interview nuance: Interviewers often follow up with "what is the difference between concurrency and parallelism?" Concurrency is structuring work so tasks make progress by interleaving, which is what a thread pool and async give you under the GIL. Parallelism is tasks running at the same instant on different cores, which is what a process pool gives you (or the experimental free-threaded no-GIL build added in Python 3.13). So a ThreadPoolExecutor buys you concurrency and overlaps I/O waits, but only processes buy you CPU parallelism. Naming that distinction, and tying it to the GIL, is exactly the signal they are listening for.