Memory management & performance

CPython manages memory primarily through reference counting: every object carries a hidden counter tracking how many references (names, container slots, etc.) currently point to it; each time a new reference is made the count goes up, each time one goes away (a variable is reassigned, goes out of scope, or is explicitly deleted) the count goes down, and the instant the count hits zero, the object is immediately deallocated (its memory freed) — this is why, unlike in many garbage-collected languages, CPython’s memory reclamation is largely deterministic and immediate rather than happening in unpredictable batches. The catch: reference counting alone can’t detect a reference cycle — a group of objects that reference each other (directly or indirectly) forming a loop, such as a parent object holding a reference to a child, and that child holding a reference back to its parent — because even if nothing outside the cycle references any of them anymore, each object inside the cycle still has a non-zero count (from the other cycle members), so pure refcounting would leak that memory forever. To handle this, CPython layers a separate cyclic garbage collector on top, which periodically scans for groups of objects that are only reachable from within themselves (and are therefore genuinely unreachable garbage despite non-zero refcounts) and frees them. This collector organizes tracked objects into three generations based on survival: new objects start in generation 0; anything that survives a collection pass gets promoted to generation 1, then eventually generation 2 — the idea (borrowed from the general “generational hypothesis” in GC design) being that most objects die young, so scanning generation 0 frequently and the older generations progressively less often is more efficient than scanning everything every time. A GC pass is triggered automatically based on allocation thresholds (roughly, once enough new objects have been allocated relative to freed ones since the last pass), but can also be triggered manually via gc.collect(), and disabled entirely via gc.disable() — a real optimization technique when you know you’re about to create a huge burst of objects with no cycles involved and want to avoid GC overhead during that burst, re-enabling afterward.

By default, every instance of an ordinary Python class carries its own __dict__ — a per-instance dictionary holding all its attributes — which gives you the flexibility to add new attributes to any instance at any time, but costs real memory overhead per instance (a dict has non-trivial fixed overhead beyond just the attributes it stores, since it needs to support fast arbitrary-key lookup, resizing, etc.). __slots__ is a class-level declaration (a list/tuple of allowed attribute names) that tells Python “instances of this class will only ever have these specific attributes” — in exchange, Python allocates a fixed, compact structure for instances instead of a full __dict__, which can meaningfully cut per-instance memory usage, especially at scale (the savings become significant once you’re creating millions of instances — a rare optimization in typical business-logic code, but a real and common one in data-processing-heavy or scientific code).

The saving, concretely

class PointDict:
    def __init__(self, x, y):
        self.x = x
        self.y = y

class PointSlots:
    __slots__ = ("x", "y")
    def __init__(self, x, y):
        self.x = x
        self.y = y

PointDict instances each carry a __dict__ ({"x": ..., "y": ...}) — on a typical 64-bit build that dict alone runs well over 100 bytes before counting the two values it holds. PointSlots instances have no __dict__ at all: x and y live in a small fixed-size structure generated from the __slots__ declaration, roughly the size of two pointers. Multiply that per-instance difference by a few million rows of parsed data and the gap becomes real memory, not a micro-optimization.

What you give up

p = PointSlots(1, 2)
p.z = 3
# AttributeError: 'PointSlots' object has no attribute 'z'

import weakref
weakref.ref(p)
# TypeError: cannot create weak reference to 'PointSlots' object

• No dynamic attributes. Assigning to a name not listed in __slots__ raises AttributeError — there’s no __dict__ for it to land in.
• No weak references, by default. weakref needs a hidden __weakref__ slot that isn’t allocated unless you ask for it explicitly: __slots__ = ("x", "y", "__weakref__").
• Multiple inheritance gets restrictive. Two unrelated base classes that each define a non-empty __slots__ can’t both be inherited from — Python raises TypeError: multiple bases have instance lay-out conflict, because each base’s slots claim a fixed memory layout and the two layouts can’t be merged.
• Pickling can need extra work. The default pickle protocol introspects __dict__ to know what to serialize; a slotted class without one needs __getstate__/__setstate__ (or a modern-enough pickle protocol, which handles this automatically in practice) to round-trip correctly.

A subclass that forgets to declare its own __slots__ silently regains a full __dict__ for its own instances, quietly undoing the saving further up the hierarchy — see __slots__ and __dict__ in “Data model & dunders” for that gotcha with a worked example.

A Python list is implemented under the hood as a dynamic array — a contiguous block of memory holding pointers to the actual list elements (the elements themselves live elsewhere in memory; the list just stores references to them), with the array’s allocated capacity typically larger than its current length. When you .append() an element, if there’s spare capacity remaining, it’s simply written into the next free slot — a genuinely O(1) (constant time) operation. When the array is full, CPython over-allocates: rather than growing by exactly one slot (which would force a costly resize on almost every single append), it grows the underlying array by a multiplicative factor (roughly ~1.125x plus a bit extra, by a specific growth formula), allocating noticeably more room than immediately needed. This means resizes (which do require allocating a new, larger block and copying every existing element over — an O(n) operation) happen increasingly rarely as the list grows, and the average cost per append across a long sequence of appends works out to O(1) — this is what “amortized O(1)” means: not every single individual operation is O(1), but the cost averaged over a long sequence of them is. Inserting at index 0 (list.insert(0, x)), by contrast, is always O(n), with no way around it: every existing element has to physically shift one slot to the right to make room at the front, regardless of spare capacity — there’s no equivalent over-allocation trick that helps here, since the “free space” that over-allocation provides is only ever at the end of the array. This is exactly why collections.deque exists as the right tool when you need efficient insertion/removal at both ends.

A real diagnostic workflow, not just naming tools: start with tracemalloc (a built-in module that records, for each memory allocation, the Python-level call stack responsible for it) — take a snapshot early, let the service run and leak for a while, take a second snapshot, and diff them; the diff shows you exactly which lines of code are responsible for the largest net new allocations since the first snapshot, rather than a vague “memory went up” signal. In parallel, objgraph (a third-party but widely used library) helps specifically with reference cycles and answering “what’s actually holding a reference to this object, keeping it alive” — since a cycle involving objects with a custom __del__ finalizer method used to be a real, if now largely historical (Python 3.4+), source of GC-uncollectable garbage. Also worth checking systematically: unbounded caches — functools.lru_cache used without a maxsize (or with one too generous for the actual working set) will happily grow forever, retaining every unique argument combination ever seen; this is one of the single most common real-world Python memory leaks, precisely because caching feels like an optimization rather than a liability. Finally, remember the limits of Python’s own visibility: memory held by C extensions (numpy arrays, database driver buffers, etc.) lives outside what Python’s own GC and refcounting can see or report on directly — a leak there requires tools that understand the specific C library’s own memory management, or lower-level OS/process memory profiling, since Python-level introspection tools are blind to it.

The regular ~10-minute periodicity is itself a strong clue worth chasing first, since it suggests something scheduled rather than purely load-driven — a real diagnostic process: check for a GC pause correlation (Python’s generational garbage collector periodically does more expensive full-collection passes; logging GC stats/timing, via the gc module’s callbacks, alongside request latency can reveal a direct correlation with collection pauses); check connection pool exhaustion (if a database/HTTP connection pool has a fixed size and something is periodically holding connections longer than expected — perhaps a slow background job — requests can queue waiting for a free connection); use py-spy (a sampling profiler that can attach to a live, already-running Python process without needing to restart or add instrumentation code — invaluable specifically because you can’t always reproduce an intermittent production issue on demand) to capture what’s actually executing during a spike; and check for a scheduled job or cron task running on roughly that interval and colliding with request-serving capacity (a background job consuming CPU, holding a lock, or hammering the same database, timed suspiciously close to the observed latency pattern). The regularity of the interval is a specific, actionable clue worth chasing directly, rather than jumping straight to generic “check the logs” advice.

A generator is created by any function containing a yield statement — calling it doesn’t run the function body immediately; instead it returns a generator object, and the function’s body only actually executes, a little bit at a time, as you iterate (each next() call runs the function until the next yield, then pauses). The key mechanical fact: when a generator pauses at a yield, its stack frame (the region of memory holding its local variables, current execution position, and everything a normal function call needs) is not discarded the way an ordinary function’s frame is popped off the call stack the instant it returns — the generator’s frame stays alive, suspended, referenced by the generator object itself, for as long as the generator object exists and hasn’t been exhausted or explicitly closed/garbage collected. The memory-relevant consequence: any large local objects a generator is holding onto between yields (say, a big intermediate buffer it built early on and keeps referencing later) stay alive in memory the entire time the generator itself is alive, even though from the outside it looks like you’re just “streaming” values out lazily one at a time. This is a genuinely underappreciated, easy-to-miss memory retention pattern: people reach for generators specifically to save memory by avoiding materializing a full list upfront, without realizing a poorly-written generator can still quietly pin large objects in memory for its entire (possibly long) lifetime.

A real, sensible progression: start with __slots__ or the modern dataclass(slots=True) shortcut (3.10+) — cheap to apply, and a genuinely large win over plain instances with a full per-instance __dict__. Next, if mutability genuinely isn’t needed, switch to tuple/NamedTuple, which carry even less structural overhead than a slotted class. For genuinely homogeneous numeric data specifically, reach for array.array or numpy arrays instead of a plain list of individually boxed Python objects — this is by far the single biggest lever available, since it eliminates per-element Python object overhead entirely rather than merely reducing it. Only after those simpler, high-leverage wins are exhausted would you reach for more exotic, higher-effort options — memory-mapped files, columnar on-disk formats — genuinely justified only once the data volume has outgrown what fits comfortably in memory at all.

Every Python int — even a small one — is a genuinely full, independently heap-allocated object, carrying a type pointer, a reference count, and its actual numeric payload; a list containing many such ints doesn’t store the numbers directly at all — it stores an array of pointers, each pointing off to one of these separately-allocated boxed integer objects scattered elsewhere in memory. A numpy array, by contrast, stores the raw, fixed-width numeric values themselves, packed contiguously in memory, with essentially zero per-element Python object overhead at all. This “boxing” difference — “array of pointers to separate objects” vs. “array of raw bytes” — is the core reason for the dramatic memory difference, and it also carries a real, separate performance implication beyond just total size: contiguous raw data has far better CPU cache locality than scattered, pointer-chased boxed objects.

sys.getsizeof(obj) reports only the size of that one object itself — for a container, that means its own internal overhead plus the space needed for its pointer slots, but explicitly not the size of whatever objects those pointers actually point to. So sys.getsizeof([1, 2, 3]) reports the list’s own structural size, entirely excluding the actual memory used by the three separate boxed integer objects it references. To get a genuinely accurate total footprint, you need to recursively walk every reference the object holds (either a hand-rolled implementation, or a library like pympler built for exactly this) — and even then, “total” remains genuinely ambiguous once shared/interned objects (the small-int cache, string interning) are involved, since those shared objects would otherwise be double-counted across every place that happens to reference them.

A weak reference is a reference that doesn’t count toward an object’s reference count at all — creating one via weakref.ref(obj) does not increment ob_refcnt, so from the object’s perspective it might as well not exist: CPython remains completely free to collect the object the instant every strong (ordinary) reference to it goes away, regardless of how many weak references are still pointing at it. A weak reference is itself a small callable object — calling it (weak()) dereferences it: if the target is still alive you get the real object back, and if it’s already been collected you get None — so code holding a weak reference always has to check for None before using it, since the target’s continued existence is never guaranteed the way it is with a normal name binding.

import weakref

class Resource:
    def __init__(self, name):
        self.name = name

r = Resource("db-connection")
weak = weakref.ref(r)

weak()            # <Resource object at ...> — still alive, r itself still holds a strong ref
del r             # drop the only strong reference
weak()            # None — the object was genuinely collected, not merely "marked" for it

Not every object supports being weakly referenced — plain built-ins like int/str/tuple don’t (wrapping them, e.g. in a small custom class, is the usual workaround), and a __slots__-using class needs an explicit __weakref__ entry in its __slots__ (see above), since the weak-reference machinery itself needs a hidden slot on the instance to register into.

The classic real use case built on top of this: an observer or cache pattern where some central, long-lived registry holds references to many objects purely for bookkeeping — “notify all currently-registered listeners,” or “look up the live instance for this id if one still exists.” If that registry held ordinary strong references, every object it’s tracking could never be garbage collected, even long after nothing else in the program needs it, purely because the registry itself is still holding on. WeakValueDictionary (values are weak, keys are strong — the natural shape for an identity cache) solves this directly: entries disappear from the dict on their own the moment the last real strong reference to that value elsewhere in the program goes away, with no manual cleanup needed.

_registry: "weakref.WeakValueDictionary[str, Resource]" = weakref.WeakValueDictionary()

def get_resource(name):
    if name in _registry:
        return _registry[name]
    resource = Resource(name)
    _registry[name] = resource
    return resource

a = get_resource("db-connection")
len(_registry)   # 1
del a
len(_registry)   # 0 — the entry vanished by itself once nothing else referenced the Resource

WeakKeyDictionary is the mirror image (weak keys, strong values) — the right shape when you want to attach extra data to an object (“cache a computed value keyed by object identity”) without that attachment being the thing that keeps the object alive.

The mechanism, precisely: @lru_cache builds its internal cache as a plain dict, keyed on a tuple built from the call’s actual argument objects (not a hash-only fingerprint of them, and not a copy) — so every distinct combination of arguments that has ever been passed in is kept as a real, strong-referenced dict key, for as long as that entry stays in the cache. Two consequences follow directly from that: first, caching calls made with a genuinely large argument object (or one with an expensive __hash__/__eq__) retains that entire object for the life of the cache entry, which can dwarf the memory of whatever small result value is actually being cached — the visible “cache size” wildly understates what’s actually being pinned in memory. Second, and much easier to trip over by accident: decorating an instance method directly with @lru_cache puts self into that same key tuple, because self is just the method’s first positional argument like any other.

Because the key is a real dict key, every argument to a cached function must be hashable: positional and keyword arguments are combined into a single hashable key structure, and calling a cached function with an unhashable argument (a plain list or dict) raises TypeError: unhashable type before the function body ever runs. A related subtlety of building the key from the literal call shape: f(1, y=2) and f(1, 2) produce different keys and are cached separately, even though they bind identically.

@lru_cache
def total(items):
    return sum(items)

total((1, 2, 3))   # -> 6, cached
total([1, 2, 3])   # TypeError: unhashable type: 'list'

from functools import lru_cache
import weakref

class Service:
    def __init__(self, id):
        self.id = id

    @lru_cache(maxsize=None)
    def compute(self, x):
        return x * 2

s = Service(1)
s.compute(10)
ref = weakref.ref(s)
del s
ref()   # still a live Service — NOT None

s doesn’t get collected on del, even though that was its only apparent strong reference in the calling code — @lru_cache runs once, at class-definition time, wrapping the function object itself; the cache dict it creates belongs to that one wrapped function, shared by every instance of the class, and lives for as long as that function object does (in practice, for the life of the process, unless .cache_clear() is called explicitly or the entry is evicted by a finite maxsize). So every distinct Service instance that ever calls .compute(...) gets permanently pinned in that shared cache’s keys — a real, easy-to-miss memory leak scoped to the class, not the instance, that silently defeats the assumption that an object becomes collectible once its caller lets go of it.

The fix is to stop putting self in a class-level cache’s key space: decorate a plain module-level function that takes only the identifying primitive data instead of self; or build the cache per-instance instead of per-class, e.g. self._compute = lru_cache(maxsize=None)(self._compute) inside __init__, so the cache dict — and everything it retains — is thrown away along with the instance itself; or reach for a caching library (cachetools, for instance) whose cache variants support weak-keying the cached-on object, so entries can be reclaimed the same way a WeakKeyDictionary entry would be.

The fork start method (available on platforms that support it, notably not natively on Windows) relies on copy-on-write (COW) memory pages at the OS level — a newly-forked child process initially shares the exact same physical memory pages as its parent, without any duplication at fork time at all; a given page is only actually copied, separately, the moment either the parent or that specific child writes to it. This means genuinely read-heavy shared data (a large preloaded ML model, a big shared dataset) doesn’t need to be duplicated per worker process “for free,” at least in principle. Where this assumption breaks down, subtly: Python’s own reference counting causes writes to memory internally — every single refcount increment or decrement touches the memory page holding that count, meaning merely reading an object in Python (which still bumps its refcount as a side effect) often also triggers a write at the OS memory-page level, which can defeat far more of the expected COW memory savings than people naively expect — a genuinely subtle, real gotcha for anyone building multiprocess data pipelines assuming shared read-only data stays cheaply shared.

sys.intern() explicitly forces a given string into CPython’s internal pool of interned strings, ensuring that repeated, separately-created but value-equal strings all end up sharing exactly one single underlying object rather than each allocating its own separate copy. This is genuinely useful when processing huge volumes of text containing a relatively small set of frequently-repeated substrings (e.g., parsing millions of log lines that all reuse the same limited set of field names/values) — where the memory savings from deduplication genuinely matter at that scale — and it also has the side effect of letting you safely use is for equality comparison between those specific interned strings afterward (though relying on that as a general habit remains bad practice outside this narrow, deliberate context).

tracemalloc records, for every allocation, the actual Python-level call stack responsible for it — a real diagnostic workflow: take an initial snapshot, let the suspected leak/bloat accumulate for a while under real conditions, take a second snapshot, and diff them — the diff shows exactly which specific lines of code, by file and line number, are responsible for the largest amount of net new allocation between the two snapshots. This is genuinely more actionable than an OS-level memory profiler simply reporting total process memory usage rising over time, since that generic view tells you that memory is growing but nothing at all about which specific code is actually responsible for the growth.

str’s immutability means every single s += chunk inside a loop creates an entirely new string object each time, copying all the previously-accumulated characters over again — genuinely O(n²) total cost across a loop building up a string of length n piece by piece. "".join(list_of_chunks) avoids this entirely, computing the final required size once and copying each chunk into place exactly one time, for genuinely O(n) total cost.