Closing: Lakehouse Checklist & Where Next

# Closing: Lakehouse Checklist & Where Next ## Ship-it checklist (all 4 deliverables) - [ ] Lab 1: idempotent Parquet ETL, date partitions, README counts + plan. - [ ] Lab 2: MERGE upserts, time-travel + RESTORE evidence, OPTIMIZE proof. - [ ] Lab 3: partition evolution without rewrite, pruning + rollback evidence. - [ ] Redpanda mini: kill/restart with count == 1000, zero duplicates. ## The 5 interview one-liners 1. Lazy transforms + actions; narrow vs wide (shuffle) transforms. 2. Partition on filter columns with modest cardinality; prune, don't scan. 3. Parquet = format; Delta/Iceberg = Parquet + log giving ACID/MERGE/travel. 4. OPTIMIZE compacts, VACUUM deletes — never vacuum inside your travel window. 5. Iceberg hidden partitioning decouples logical values from paths: evolve freely. ## Where next - Orchestrate these jobs (Airflow/Dagster) with partition-aware backfills. - Serve the gold layer (dbt models on top of Delta/Iceberg). - Table health SLIs: file counts, snapshot age, stream lag — alert on drift.