Portfolio Projects/Spark Lakehouse Medallion Build
₹99data engineer · Core Skillintermediate · ~8 Hours

Spark Lakehouse Medallion Build

Build a Bronze -> Silver -> Gold medallion lakehouse with PySpark + Delta Lake on local Docker: deduped Silver, Gold marts, quality gates, time travel, and CI — fully local, no shared infra, the Week-9 Spark theory hardened into a portfolio repo.

DatabaseCloud Data Store
ToolchainVS Code + SQLTools
Milestones5 Guided Stages
Recruiter OutputVerified GitHub Repo
View 5 Milestones
Full project workspace, live database credentials, AI mentor & verified certificate
Project Deliverables

What You Will Ship to GitHub

Hiring managers don't want to see another cookie-cutter Titanic or Iris tutorial. You will build a professional repository with modular scripts and executive business memos that demonstrate commercial analytical maturity.

github.com/your-username/de-spark-lakehouse-medallionPortfolio Structure
├── sql/
01_exploratory_profiling.sql // Schema audits, table volume, date ranges
02_revenue_trends.sql // Monthly GMV, order volume & AOV evolution
03_customer_concentration.sql // Pareto 80/20 distribution, repeat buyer cohorts
├── findings/
00_executive_summary.md // 3 core business takeaways for executive stakeholders
01_business_health.md // Detailed breakdown of margins, cancellations, and seasonality
03_customer_segments.md // Customer lifetime spend & cohort concentration
├── README.md // Professional project front-door with data schema diagram
Outcomes

What You Will Master

Run PySpark + Delta Lake on local Docker (own machine only)
Build Bronze/Silver/Gold layers with schema enforcement
Gate Silver with dedup + null + range quality checks
Use Delta time travel + VACUUM and explain the tradeoff
Curriculum

5 Guided Project Milestones

M1

Local Docker Spark + repo scaffold

Compose stack boots Spark; repo layout runs a smoke job.

~30 mins
M2

Bronze ingest to Delta

Raw fixtures land as append-only versioned Bronze.

~90 mins
M3

Silver + Gold medallion

Core Milestone

Deduped Silver, Gold marts, schema enforcement on.

~90 mins
M4

Quality gates + time travel

Failing gates block Gold; time travel + vacuum demonstrated.

~90 mins
M5

CI + README + demo

Review Pass Required

Green CI on every push, portfolio README, recorded demo.

~60 mins
Career Value

ATS Keywords & Resume Ready Bullets

Tools
PySparkDelta LakeDockerGit/GitHub
Methods
Common Table Expressions (CTEs)Window FunctionsTime-Bucket AggregationsCohort AnalysisData Profiling & Auditing
Business Concepts
AOV (Average Order Value)Customer ConcentrationRepeat Purchase BehaviorCohort RetentionRevenue Health Metrics
Pre-Written Resume Bullet (Copy & Paste)

"Authored an end-to-end data analytics project on Spark Lakehouse Medallion Build utilizing PySpark, Delta Lake, Docker — delivered production-grade queries, cohort distributions, and an executive stakeholder summary published as a portfolio GitHub repository."

Preparation

Reading & References

Lakehouse project brief: local Docker setup
setup guide · Premium Access
Lakehouse starter: jobs skeleton
article · Premium Access
Lakehouse project rubric
markdown · Premium Access

Ready to build and ship Spark Lakehouse Medallion Build?

Instant access to cloud database credentials, milestones, and starter queries.

Project Overview

Free Spark Lakehouse Medallion Build portfolio project — with AI review

Build a real spark lakehouse medallion build project for your data analyst portfolio in your own public GitHub repo — free brief, starter template, and milestone guides. When your code is ready, the ₹99 pass gets every milestone reviewed against a fixed rubric (de spark lakehouse medallion walkthrough included) and issues a verifiable certificate on completion.

Is the Spark Lakehouse Medallion Build project free?

The full project brief, milestone guides, and starter template are free to audit. The AI-powered review pass and verified certificate unlock at ₹99.

Can I add the Spark Lakehouse Medallion Build project to my resume and GitHub?

Yes — that is the point. You build in your own public GitHub repo, every milestone is reviewed against a fixed rubric, and the certificate links to the repo employers can inspect. Only list what is visible in your repo.

How long does the Spark Lakehouse Medallion Build project take?

Most students finish the 5 milestones in 2–4 weekends. Milestones unlock in order, and you can re-submit any milestone that needs work.

Rubric-reviewed · Plagiarism-checked · Verifiable certificate

Build your portfolio & get certified

Want more projects like this? Join our Data Analyst Work Experience Program to complete 5+ guided industry projects, gain real experience, and earn an internship certificate.

Join Work Experience
Access
₹99