Error handling & exceptions

BaseException sits at the very top of Python’s exception hierarchy; nearly every exception you’d normally deal with (including all the built-in errors like ValueError, TypeError, KeyError) actually descends from Exception, which is itself just one subclass of BaseException — but a small, deliberate set of exceptions sit outside Exception, as direct siblings of it under BaseException: SystemExit (raised when sys.exit() is called, representing an intentional program shutdown), KeyboardInterrupt (raised when the user presses Ctrl-C), and GeneratorExit (delivered into a generator when it’s being closed). This placement is deliberate: except Exception: — an extremely common, broad-catch pattern people reach for — will not accidentally intercept any of these, meaning a Ctrl-C still actually stops the program, and sys.exit() still actually exits, even inside code with a broad exception handler somewhere in the call stack. If you instead use a truly bare except: (catching literally everything, including BaseException itself), you lose this protection entirely — the program can become nearly impossible to interrupt or shut down cleanly, since even the shutdown/interrupt signals themselves get silently swallowed.

A bare raise (with no exception specified), used inside an except block, re-raises the exception currently being handled, preserving its original traceback exactly as it was. Constructing and raising a genuinely new exception (raise SomeException()) instead starts a fresh traceback from that point — unless you explicitly chain it to the original using raise NewError("...") from original_err, which preserves the link between them (surfacing as __cause__) even though it’s technically a different exception object. from None is a specific, deliberate variant that suppresses the chain display entirely, useful when translating a low-level, internal exception into a domain-specific one at a public API boundary — the original internal traceback would just be confusing noise to an external caller who doesn’t need or want to see your implementation details, so from None presents a clean, focused error instead.

The parentheses are genuinely required in Python 3 to group multiple exception types together into a single tuple that except checks against; without them, except TypeError, ValueError: was actually valid, meaningful Python 2 syntax with an entirely different, and now-removed, meaning — binding the caught exception instance itself to the name ValueError (an old, now-removed alternate way of naming the caught exception, distinct from Python 3’s as keyword) — a real trap specifically for anyone porting code, or muscle-memory habits, over from Python 2.

__context__ is set automatically, with no special syntax needed, any time a new exception happens to be raised while Python is already in the middle of handling a previous one (e.g., your except block’s own cleanup code itself raises) — this is implicit chaining, and produces the “During handling of the above exception, another exception occurred” message. __cause__ is set only via the explicit raise ... from ... syntax — explicit chaining, deliberately declared by you — and produces the “The above exception was the direct cause of the following exception” message instead. Both attributes can genuinely coexist on the same exception object simultaneously even though only one message typically prints in the default traceback formatting; from None specifically suppresses both from being displayed, even though the underlying __context__ may technically still be set internally.

try:
    try:
        {}["missing"]
    except KeyError as e:
        raise ValueError("bad config") from e
except ValueError as err:
    print(repr(err.__cause__))          # -> KeyError('missing')   explicit
    print(repr(err.__context__))        # -> KeyError('missing')   also set implicitly
    print(err.__suppress_context__)     # -> True   so only the "direct cause" message prints

Which message the default traceback prints is decided by these attributes: if __cause__ is set, the “direct cause” message is shown (and raise ... from also sets __suppress_context__ = True, hiding the implicit context); otherwise, if __context__ is set and not suppressed, the “During handling” message is shown. raise ... from None sets __cause__ = None and __suppress_context__ = True, so neither prints.

Good design here balances two competing needs: giving callers a single, common base exception for the whole library (so except PaymentError: reliably catches anything the library might raise, without needing to enumerate every specific subtype), while also giving meaningful subclassing so callers who do care about a specific failure mode can catch it narrowly. The specific hierarchy shape should be driven by how callers will actually want to react — e.g., splitting into something like TransientPaymentError (network blips, temporary service unavailability — safe and sensible to retry) versus PermanentPaymentError (insufficient funds, invalid card — retrying is pointless or actively wrong) is a genuinely useful split because it directly drives real retry-logic decisions, rather than subclassing purely for the sake of taxonomic completeness. A common mistake worth flagging: over-engineering a deep, elaborate 15-level exception hierarchy that no caller actually branches on in practice — restraint matters as much as thoughtful structure.

.args holds the raw positional arguments the exception was actually constructed with, as a genuine tuple; str(exception) calls the exception’s __str__ (inherited by default from Exception, which typically just joins .args into a readable string) — but if a custom exception subclass overrides __init__ to store genuinely richer structured state beyond a single simple message, the default __str__ may no longer produce a genuinely useful, informative message, meaning you’d want to override __str__ explicitly yourself, specifically so that logging tools that call str(exc) actually produce something genuinely useful rather than a cryptic, unhelpful default.

EAFP (“easier to ask forgiveness than permission”) means just attempting an operation and handling the exception if it fails (try: value = d[key] \n except KeyError: ...); LBYL (“look before you leap”) means checking preconditions first before attempting the operation (if key in d: value = d[key]). Python culture leans EAFP for good reasons (it avoids a separate, redundant check that can itself race against concurrent modification, and often reads more naturally). LBYL is genuinely better specifically when the “ask forgiveness” operation would have a real, unwanted side effect if attempted speculatively and failed (e.g., you don’t want to speculatively kick off an expensive or irreversible operation just to catch its failure), or in a genuinely hot, performance-sensitive loop where raising and catching exceptions has real, measurable cost in CPython — worth noting that Python 3.11 made the non-raising path of exception handling essentially free (zero-cost when no exception actually occurs), which shifted this specific performance tradeoff somewhat, though the “unwanted side effect” reason for preferring LBYL remains independent of that.

The else clause on a try statement runs its code only if the try block completed with no exception at all — the key, easy-to-miss distinction from simply placing that same code after the whole try/except: any exception raised inside the else block itself is not caught by the except clauses that precede it (since those clauses are only listening for exceptions from the try body, not from else), whereas code accidentally placed inside the try body instead of else would be vulnerable to being unintentionally caught by an overly broad except clause meant only for the original risky operation. A lot of experienced Python developers have simply never used else on a try statement and don’t know precisely why it exists as a distinct clause from “just put it after.”

In the most common, narrow use case, they’re largely equivalent in effect — both discard the specified exception silently. But there’s a real, meaningful safety difference: contextlib.suppress(SomeException) only swallows exactly the exception type(s) you named, letting anything else propagate normally, whereas a genuinely bare except: (with no type specified at all) swallows everything, including BaseException subclasses like KeyboardInterrupt/SystemExit — a real, if perhaps unintended, safety gap. suppress is the safer default even when your actual intent was already narrow, precisely because it makes that narrowness an explicit, enforced constraint rather than an accidental byproduct of writing except SpecificError: pass correctly (which achieves the same narrow safety, but suppress communicates the intent to suppress more clearly and concisely as a one-liner).

If a finally block itself raises a new exception while an earlier exception was already actively propagating out of the try/except, the new exception from finally takes over and becomes the one that actually propagates further up the call stack — the original exception becomes that new exception’s __context__ (the same implicit-chaining mechanism described above) but is otherwise effectively displaced from the primary propagation path, easy to lose track of unless someone specifically inspects the exception chain. This is a genuine, real production footgun: cleanup code inside finally (closing a connection, releasing a lock) can itself fail — and if it does, the original error (often the actual root cause someone actually needed to see and fix) gets silently overshadowed by a secondary cleanup failure, which is rarely tested for precisely because writing a test that deliberately makes cleanup logic also fail, on top of the original failure, isn’t something most people think to exercise.

A well-designed retry mechanism distinguishes transient failures (network timeouts, temporary 503-style unavailability — genuinely worth retrying, since the same call might well succeed moments later) from non-transient ones (validation errors, 4xx-style client errors — retrying identically won’t help, since the request itself is fundamentally invalid, not just unlucky). It uses exponential backoff with jitter — waiting progressively longer between each retry attempt (backoff), with some randomization added (jitter) specifically to avoid many clients all retrying in perfect lockstep and slamming the recovering service simultaneously (a “thundering herd”) — and enforces a sensible maximum attempt count rather than retrying forever. The other essential piece of this is idempotency: an operation is idempotent if performing it multiple times has the same effect as performing it once (e.g., “set this field to X” is idempotent; “charge this card $10” performed twice is very much not). Blindly retrying a non-idempotent operation (like a payment charge) risks duplicating a real-world side effect — the actual production remedy involves idempotency keys or similar mechanisms, not just “add a retry loop.”

The core principle: internal exception details (raw messages, stack traces, internal class names) should generally never leak directly to an external caller — beyond being poor UX, it’s a real security concern, since internal error text can inadvertently reveal implementation details, file paths, or other information useful to an attacker probing the system. The right design has a deliberate translation layer at the API boundary — mapping specific, known domain exceptions to specific, well-defined API error codes/messages (e.g., a domain InsufficientFundsError maps to a clean 402 Payment Required with a client-safe message) — while anything unexpected (a bug, an unhandled edge case) falls through to a generic 500 Internal Server Error response with a minimal, non-revealing message to the client, while the full internal exception details (message, traceback, context) are still captured and logged internally for on-call engineers to actually diagnose. Letting arbitrary internal exceptions bubble directly to a serializer without this deliberate translation step is a common, real production anti-pattern.

logging.exception(...) (or equivalently, logging.error(..., exc_info=True)) captures and logs the full traceback — the entire call stack showing exactly where the exception originated and propagated through — alongside the message, whereas logging.error(str(e)) only logs whatever text str(e) happens to produce for that specific exception, which can be extremely thin: for something like a bare KeyError, str(e) might just print the missing key itself, with no indication whatsoever of which line of code, in which function, actually raised it. This is an extremely common, genuinely costly real-world mistake — a production incident where the only logged information is a cryptic "'user_id'" (a KeyError’s minimal string representation) with no traceback at all can turn a two-minute diagnosis into a much longer one, purely because of how the exception was logged, not because of anything about the bug itself.

Exceptions-as-control-flow for signaling failure is deeply idiomatic in Python (the same EAFP culture described above) — swallowing a failure and returning None (or some other sentinel value) instead discards the actual reason something failed, making the caller’s job harder (they know something didn’t work, but not why), and creates a genuine ambiguity problem when None is already a perfectly valid, meaningful return value in the successful case (now the caller can’t distinguish “legitimately got None” from “something silently failed”). Result/Either-style patterns (a return type that’s explicitly either “success, here’s the value” or “failure, here’s the error,” commonly seen in Rust, Haskell, and similar languages) are a real, legitimate alternative some Python codebases deliberately adopt — but it’s worth recognizing this as importing a different language culture’s convention into Python rather than something the language itself is built around — both approaches carry genuine tradeoffs, and neither is straightforwardly better in every case.

Beyond the general vagueness of “swallowing errors is bad,” there are concrete, specific failure modes: silent data corruption — if an exception occurs partway through a multi-step operation (e.g., updating several related records) and it’s simply swallowed, the operation can be left in an inconsistent, half-completed state with no error surfaced anywhere to alert anyone; masked resource leaks — if an exception prevented a cleanup step from running, and the exception is just discarded, you lose the only signal that would have told you a file handle, connection, or lock never got released; and a genuinely subtle scoping detail worth knowing precisely: except Exception does not actually catch everything — KeyboardInterrupt and SystemExit specifically inherit from BaseException, not Exception, specifically so that broad-but-common patterns like this one don’t accidentally swallow “the user pressed Ctrl-C” or “the program is trying to shut down cleanly” signals. A genuinely broad except: (bare, catching literally everything including BaseException subclasses) is even more dangerous for exactly that reason. The fix in all cases isn’t “never catch broadly,” it’s catching the narrowest exception type you actually expect and know how to handle, and letting everything else propagate.

An exception raised naturally inside a generator’s own body (during the code executing between two yield statements) propagates out to whatever’s driving the generator — the for loop, or whoever called next() — at exactly the point where that next()/iteration step was requested; from the consumer’s perspective, it looks just like the generator itself raised at that call. Separately, you can also actively throw an exception into an already-paused generator using .throw(), which delivers it at the exact point the generator is currently suspended (at its most recent yield) — this is precisely the same underlying mechanism used to deliver CancelledError into a suspended coroutine, and to deliver exceptions into a contextlib.contextmanager-based context manager’s generator body — understanding this one mechanism explains several seemingly-separate behaviors across the language.

No — assert statements are specifically meant for debugging assumptions and test invariants, genuinely not for handling real, expected error conditions in production code. Running Python with the -O (optimize) flag actually strips out every single assert statement entirely from the compiled bytecode — meaning any code that has come to silently rely on an assert for genuine, real runtime validation or safety-checking will simply have that entire check vanish completely under -O, with no warning whatsoever, a real and genuinely dangerous production footgun for any team that isn’t fully aware of this specific flag’s actual behavior.