Functional patterns & iterators

itertools.chain(*iterables) conceptually just yields every element from the first iterable, then every element from the second, and so on — trivially implementable as a Python generator (for it in iterables: yield from it). The reason itertools’s actual implementations are written in C rather than pure Python matters beyond a generic “C is faster” — two specific, concrete advantages: constant memory overhead per item, since a C-level iterator avoids allocating a full Python generator frame object for the iteration itself, and no per-item Python bytecode dispatch overhead — every step of a pure-Python generator has to go through the bytecode interpreter’s per-instruction overhead, whereas a C implementation executes as compiled machine code with no such per-step interpretation cost. This becomes genuinely relevant, not just theoretically interesting, when chaining/transforming very large iterables where the per-item overhead, multiplied across millions of items, adds up to a real, measurable difference.

An iterable is anything you can call iter() on to get an iterator (i.e., defines __iter__); an iterator is the object actually doing the stepping, defining __next__ (and typically also __iter__ returning itself, so an iterator is trivially also iterable). A common mistake: writing __iter__ to simply return self._some_generator, where self._some_generator is a single generator object created once (e.g., in __init__) and stored — this works for exactly one full pass, but a second for loop over the same object gets that same, now-exhausted generator back, silently producing zero items instead of a fresh pass. The correct pattern: __iter__ should create and return a brand-new generator/iterator every time it’s called (e.g., def __iter__(self): return (x for x in self._data) — a fresh generator expression evaluated fresh on each call), so every independent for loop over the object gets its own independent iteration state.

Yes, it does — Python has a genuine, supported fallback iteration mechanism: if a class defines no __iter__ at all, for falls back to repeatedly calling __getitem__(0), then __getitem__(1), and so on, incrementing the index each time, until an IndexError is eventually raised (at which point the loop naturally stops) — this is precisely how sequences supported iteration in Python before __iter__ existed as a dedicated protocol, and it remains a genuinely supported fallback today. Worth explicitly recognizing this is a fallback and not the primary, recommended mechanism — writing an explicit __iter__ remains genuinely better modern practice — but knowing this fallback exists demystifies why certain older or unusual classes seem to mysteriously support for iteration despite never defining __iter__ at all.

reduce(function, iterable) repeatedly applies a two-argument function cumulatively across an iterable’s items, collapsing it down to a single value (e.g., reducing a list of numbers to their product). Guido van Rossum (Python’s creator) has been on record disliking reduce specifically for readability reasons — for anything beyond the simplest, most obviously-associative operations (sum, product), a reduce call with a nontrivial lambda tends to be genuinely harder to read at a glance than the equivalent explicit loop, since you have to mentally simulate the accumulation to understand what it’s doing, whereas a loop’s intent is usually visible line-by-line. It survived into Python 3 but was demoted from a builtin to living in functools, signaling exactly this “use sparingly, prefer alternatives” stance. In practice, the specific common cases reduce is used for usually have clearer, purpose-built alternatives: sum() for addition, math.prod() for multiplication, and itertools.accumulate() when you actually want the running intermediate values (not just the final result).