Recap — Structured Logging and 12-Factor Config

# Structured Logging and 12-Factor Config Two production essentials. Logging makes your app observable; config makes it deployable. ## Why not just `print()` ```python print(f"Processing user {user_id}") ``` Problems: - Can't filter by severity. - Goes to stdout only (no rotation, no file). - No structured data (just strings). - No timestamps unless you add them. - Mixed with normal output. Use the `logging` module. ## Python logging basics ```python import logging logging.basicConfig(level=logging.INFO, format="%(asctime)s %(name)s %(levelname)s %(message)s") logger = logging.getLogger(__name__) logger.debug("very detailed") # usually disabled in prod logger.info("notable event") # important normal events logger.warning("unexpected") # something to investigate later logger.error("problem") # caught error; action needed logger.exception("with stack trace") # in except block; includes traceback logger.critical("disaster") # process dying ``` `getLogger(__name__)` gets a logger named after the module. Industry standard. ## Log levels | Level | When to use | |---|---| | DEBUG | Verbose; disabled in production | | INFO | Important events (request started, job completed) | | WARNING | Unexpected but recoverable | | ERROR | Failures requiring attention | | CRITICAL | About to die | Production: INFO and above. Bumped to DEBUG when investigating. ## Structured logging Plain text logs are hard to query at scale. Structure them: ```python logger.info("user_login", extra={"user_id": 123, "ip": "1.2.3.4", "method": "password"}) ``` Or use JSON output: ```python {"ts": "2026-05-26T14:32:01Z", "level": "INFO", "event": "user_login", "user_id": 123, "ip": "1.2.3.4"} ``` Easy to ingest into log systems (ELK, Loki, Datadog) and query: `level=INFO AND event=user_login AND user_id=123`. ### Setup with python-json-logger ```python from pythonjsonlogger import jsonlogger import logging handler = logging.StreamHandler() handler.setFormatter(jsonlogger.JsonFormatter()) logger = logging.getLogger() logger.addHandler(handler) logger.setLevel(logging.INFO) logger.info("user_login", extra={"user_id": 123}) ``` ### structlog (richer) ```python import structlog log = structlog.get_logger() log.info("user_login", user_id=123, ip="1.2.3.4") ``` structlog binds context that propagates through call chains: ```python log = log.bind(request_id="abc123") # subsequent calls include request_id automatically log.info("processing") # logs with request_id ``` Useful for request tracing. ### loguru (simpler alternative) ```python from loguru import logger logger.info("user_login | user_id={}", 123) ``` Less configurable than `logging` but easier to use. Good for small apps. ## What to log Yes: - Request received / response sent (with status, latency). - Background job started / completed. - Errors with full context (user_id, request_id, operation). - Notable state changes (user signed up, order placed). - External service calls (latency, success/fail). No: - Every function entry/exit (too verbose). - Sensitive data (passwords, full credit cards). - Inside hot loops (perf hit + noise). ### Sensitive data NEVER log: - Passwords, API keys, secrets. - Full credit card numbers (last 4 only). - PII you're not authorized to log (GDPR/HIPAA). - Encryption keys. Use redaction: ```python def safe_log_user(user): return {"id": user.id, "email": redact_email(user.email)} ``` ## Logger configuration For libraries, get a logger but don't configure it: ```python # my_library/mod.py import logging logger = logging.getLogger(__name__) ``` The APPLICATION configures the root logger: ```python # app entry point logging.basicConfig(level=logging.INFO, format="...") ``` This lets users of your library route its logs as they want. ### Per-module log levels ```python logging.getLogger("urllib3").setLevel(logging.WARNING) # quiet noisy library logging.getLogger("my_app.db").setLevel(logging.DEBUG) # verbose for one module ``` ### Production config For containerized apps: - Log to stdout (no files; let the container runtime handle). - JSON format. - INFO level default. - Configurable via env var. ```python import logging, os, sys from pythonjsonlogger import jsonlogger level = os.getenv("LOG_LEVEL", "INFO") handler = logging.StreamHandler(sys.stdout) handler.setFormatter(jsonlogger.JsonFormatter()) root = logging.getLogger() root.setLevel(level) root.addHandler(handler) ``` Container platform (K8s, ECS) ingests stdout into the log system. ## 12-Factor Configuration Don't hardcode config. The 12-factor app principle: **config in environment variables**. ### Why env vars - Same code runs in dev, staging, prod with different config. - No secrets in code. - No need to rebuild image for config change. - Standard across languages and platforms. ### The patterns ```python import os DATABASE_URL = os.environ["DATABASE_URL"] DEBUG = os.getenv("DEBUG", "false").lower() == "