itertools: chain, islice, groupby & product

The loops you keep rewriting already have names

Most loops are one of a dozen shapes. Flatten a list of lists. Take the first ten of something. Walk runs of equal keys. Pair every item with every other item. itertools gives each of those shapes a name, and the name is worth more than the saved lines: a reader sees the intent immediately instead of reconstructing it from an accumulator and an index.

Everything in the module returns a lazy iterator, so nothing intermediate is built and the pipeline works just as well on a stream you could never fit in memory.

Table

Each of these is a loop you could write by hand in four lines. The value is not the line count: a named function says what the loop is FOR, and it hands back a lazy iterator, so nothing intermediate is ever materialised.

What you want	The hand-written loop	The itertools name
Every item from several lists	a nested for loop and an append	chain.from_iterable(chunks)
The first n of something unsliceable	a counter and a break	islice(stream, n)
Runs of equal keys	a previous-key variable and a buffer	groupby(rows, key=...) over SORTED rows
Every ordered pair	two nested for loops	product(items, repeat=2)
Every unordered pair	a nested loop starting at i + 1	combinations(items, 2)

chain: one flat pass over several sources

from itertools import chain

pages = [[1, 2], [3], [], [4, 5]]
list(chain.from_iterable(pages))     # [1, 2, 3, 4, 5]
list(chain([1, 2], [3]))             # same idea, sources passed separately

chain(a, b) takes the iterables as arguments; chain.from_iterable(pages) takes one iterable OF iterables, which is the form you want whenever the sources arrive as a list. Neither builds a combined list, so chaining a hundred files costs one file's worth of memory.

islice: slicing something you cannot subscript

A generator has no [0:3], because it has no index. islice gives it one:

from itertools import count, islice

list(islice(count(1), 5))        # [1, 2, 3, 4, 5] out of an infinite counter
list(islice(rows, 10, 20))       # start and stop, like a slice

groupby: runs, not groups

This is the one that surprises people, because the name is borrowed from SQL and the behaviour is not. groupby walks the input once and starts a new group every time the key CHANGES. It is closer to Unix uniq than to GROUP BY.

from itertools import groupby

rows = [{"team": "core"}, {"team": "web"}, {"team": "core"}]
ordered = sorted(rows, key=lambda row: row["team"])
{team: len(list(group)) for team, group in groupby(ordered, key=lambda row: row["team"])}
# {'core': 2, 'web': 1}

Sort by the same key you group by, every time. The Practice exercise is exactly this pairing, and skipping the sort is how it fails.

product and combinations: nested loops with names

from itertools import combinations, permutations, product

list(product("ab", repeat=2))       # [('a','a'), ('a','b'), ('b','a'), ('b','b')]
list(permutations("abc", 2))        # order matters, no reuse: 6 pairs
list(combinations("abc", 2))        # order does not matter, no reuse: 3 pairs

Pitfalls

• An iterator is single use. Anything from itertools is consumed as you read it, so calling list() twice on the same object gives you the contents and then an empty list. Materialise once if you need it twice.
• chain(pages) is not chain.from_iterable(pages). The first treats the outer list as a single source and yields the inner lists themselves; the second flattens. It is a silent shape bug rather than an error.
• groupby compares keys with == on adjacent items only. It never sorts, hashes, or reorders, so the correctness of your grouping rests entirely on the ordering you handed it.

Interview nuance: knowing groupby groups runs rather than values is a small fact that signals you have actually used the module rather than skimmed the docs. The wider point is worth saying out loud too: itertools pipelines are lazy, so islice(chain.from_iterable(files), 100) reads only as much of the first file as it needs and never materialises anything. That is the same streaming argument that makes generators worth reaching for at all.