W12 Outline — Pipeline System-Design Videos

# W12 — Pipeline System-Design Videos Outline 1. **Design a nightly e-commerce pipeline (45 min)** — sources, volume math, batch vs micro-batch, warehouse layout, backfill story. 2. **CDC at scale (30 min)** — watermark gaps, log-based capture, ordering, exactly-once vs at-least-once + dedup. 3. **The DAG that pages you (30 min)** — retries, idempotency, SLA/alerting, partial-failure semantics. 4. **Warehouse cost clinic (25 min)** — partitioning, clustering, slot/query pricing, reading a query plan. 5. **Streaming mini-design (25 min)** — Kafka -> Bronze/Silver/Gold, late data, watermarks in the streaming sense. Each video ends with a 5-minute whiteboard drill: draw the architecture from memory, then state two trade-offs and one failure mode.