Delta Lake adds an ACID transaction log (_delta_log/) on top of Parquet.
Every write is an atomic version: readers always see a consistent snapshot.
1. Writes that plain Parquet cannot do
from delta import configure_spark_with_delta_pip
# spark configured with Delta extensions, then:
df.write.format("delta").mode("append").save("lake/delta_orders")
2. Upserts with MERGE (the money operation)
MERGE INTO delta_orders t
USING daily_updates s ON t.order_id = s.order_id
WHEN MATCHED THEN UPDATE SET *
WHEN NOT MATCHED THEN INSERT *;
One statement handles inserts + updates atomically — no duplicate-window hacks.
3. Time travel & rollback
spark.read.format("delta").option("versionAsOf", 3).load("lake/delta_orders")
spark.read.format("delta").option("timestampAsOf", "2026-09-20").load("lake/delta_orders")
Bad backfill? RESTORE TABLE delta_orders TO VERSION AS OF 3 rolls back the log pointer — data files are untouched until VACUUM.
4. File health: OPTIMIZE + Z-ORDER + VACUUM
OPTIMIZEcompacts small files (critical after streaming appends).ZORDER BY (customer_id)co-locates related rows so data-skipping prunes files.VACUUM lake/delta_orders RETAIN 168 HOURSdeletes files no longer referenced. Never set retention below your longest time-travel / rollback window.
5. Delta vs plain Parquet (interview one-liner)
Parquet is a file format; Delta is Parquet + a transaction log giving ACID, MERGE, time travel, and schema enforcement. Same files, plus guarantees.