Error handling shapes whether your code is reliable. Production Python has consistent patterns; let's cover them.
The basics
try:
risky_operation()
except SpecificError as e:
handle(e)
except (TypeError, ValueError) as e:
handle_multiple(e)
except Exception as e:
log_unexpected(e)
raise
finally:
cleanup()
finally runs whether exception or not. Use for cleanup that with doesn't cover.
EAFP vs LBYL (Python style)
EAFP: Easier to Ask Forgiveness than Permission. Try the operation; catch failure.
try:
value = my_dict[key]
except KeyError:
value = default
LBYL: Look Before You Leap. Check before the operation.
if key in my_dict:
value = my_dict[key]
else:
value = default
Pythonic style favors EAFP for most cases:
- Cleaner code (one path, not two).
- No race condition (LBYL can fail if state changes between check and use).
- Idiomatic.
But: when checking is cheap and the failure is common, LBYL is fine.
Designing exception hierarchies
For your library/app, create a hierarchy:
class AppError(Exception):
# Base for all app errors.
pass
class UserError(AppError):
# User did something wrong.
pass
class ValidationError(UserError):
# Input failed validation.
pass
class AuthError(UserError):
# Authentication failed.
pass
class ServerError(AppError):
# Something went wrong on our end.
pass
class DatabaseError(ServerError): ...
class ExternalServiceError(ServerError): ...
Why:
- Callers can catch by category (
except UserErrorfor all user errors). - Specific errors are still distinguishable.
- Errors form a graph showing what they mean.
Adding context to exceptions
class UserNotFoundError(AppError):
def __init__(self, user_id):
self.user_id = user_id
super().__init__(f"User {user_id} not found")
Include relevant data on the exception. Helps debugging.
For HTTP APIs, exceptions can carry status codes:
class APIError(AppError):
status_code = 500
class NotFoundError(APIError):
status_code = 404
Re-raising vs swallowing
try:
operation()
except SomeError as e:
log(e)
raise # re-raise after logging
# vs
try:
operation()
except SomeError as e:
log(e)
# swallow — operation failed, but we continue
Don't silently swallow. If you catch, either:
- Handle (recover or default).
- Re-raise (after logging or transformation).
- Wrap with a custom exception (
raise MyError(...) from e).
raise from e preserves the cause chain.
Retries with tenacity
For transient errors (network, rate limits), retry with backoff:
from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception_type
@retry(
stop=stop_after_attempt(3),
wait=wait_exponential(multiplier=1, min=1, max=10),
retry=retry_if_exception_type((HTTPError, ConnectionError)),
)
def fetch_data():
return requests.get(URL).json()
What this does:
- Try up to 3 times.
- Wait 1s, 2s, 4s, ... (exponential backoff) between attempts.
- Retry only on HTTPError or ConnectionError; not on ValueError.
Adding jitter
To avoid thundering herd:
from tenacity import wait_exponential_jitter
wait=wait_exponential_jitter(initial=1, max=10)
Adds random jitter so all clients don't retry at exactly the same time.
Manual retry pattern
import time
def fetch_with_retry(url, retries=3):
for attempt in range(retries):
try:
return requests.get(url).json()
except (HTTPError, ConnectionError) as e:
if attempt == retries - 1:
raise
time.sleep(2 ** attempt)
Tenacity is cleaner; the manual version helps you understand what's happening.
Result types (functional style)
Some teams prefer explicit error returns over exceptions:
from typing import Generic, TypeVar
from dataclasses import dataclass
T = TypeVar("T")
E = TypeVar("E")
@dataclass
class Ok(Generic[T]):
value: T
@dataclass
class Err(Generic[E]):
error: E
Result = Ok | Err
def divide(a: int, b: int) -> Result[float, str]:
if b == 0:
return Err("division by zero")
return Ok(a / b)
result = divide(10, 0)
match result:
case Ok(value):
print(value)
case Err(error):
print(error)
Pros: errors are part of the type signature; can't be forgotten. Cons: more verbose than exceptions; not idiomatic Python.
Library: result (returns library) implements this.
When to use:
- Domain-specific operations where caller MUST handle the failure case.
- Functional code style.
For most Python: exceptions are fine. Result types are an option, not the default.
Don't catch Exception (usually)
# Bad: catches EVERYTHING
try:
operation()
except Exception:
pass
This catches:
- The actual error you meant to handle.
- Bugs (TypeError from wrong code).
- KeyboardInterrupt (Ctrl+C) — actually no,
Exceptionexcludes BaseException's subclasses like KeyboardInterrupt and SystemExit. Phew.
Still: catching Exception masks bugs. Catch specifically.
Exception: at top-level error boundaries (web framework's error handler, background task wrapper) you DO catch broadly to prevent crashes:
try:
process_message(msg)
except Exception:
logger.exception("Worker error processing message")
# send to dead letter queue
Application boundary, not random functions.
Graceful degradation
Some failures shouldn't kill the request:
def get_user_with_recommendations(user_id):
user = db.get_user(user_id)
try:
recommendations = recommendation_service.get(user_id)
except RecommendationError:
logger.warning(f"Recs unavailable for {user_id}")
recommendations = [] # degrade gracefully
return user.with_recs(recommendations)
If recommendations fail, return user with empty recs instead of failing the whole request.
Common error-handling mistakes
except Exception: passsilently swallows bugs.- Re-raising without context. Lost the original error.
- No timeout on external calls. Hangs forever on bad services.
- Retrying non-retryable errors. ValueError isn't going to fix itself.
- No backoff on retries. Hammer the failing service.
- Catching too broadly. Hides real bugs.
- Catching too narrowly. Misses sibling errors.
Production patterns
Top-level error handler
def main():
try:
run_app()
except KeyboardInterrupt:
logger.info("shutting down")
except Exception:
logger.exception("fatal error")
sys.exit(1)
Per-request handler in web framework
@app.exception_handler(AppError)
async def app_error_handler(request, exc):
return JSONResponse({"error": str(exc)}, status_code=exc.status_code)
Circuit breaker (advanced)
When an external service is failing, stop calling it for a while:
from pybreaker import CircuitBreaker
api_breaker = CircuitBreaker(fail_max=5, reset_timeout=60)
@api_breaker
def call_external_api():
return requests.get(URL).json()
After 5 failures, calls fail immediately for 60 seconds. Service gets a chance to recover; clients don't pile up.
Takeaway
EAFP > LBYL in Python (cleaner, no race). Custom exception hierarchies (AppError → ServerError → DatabaseError). Retry transient errors with tenacity (exponential backoff + jitter). Result types for explicit functional style; exceptions for most Python. Don't catch Exception except at boundaries. Graceful degradation where possible. Circuit breaker for failing external services.