Portfolio Projects/A/B Testing & Product Experimentation Analysis
₹99data analyst · Core Skillintermediate · ~10 Hours

A/B Testing & Product Experimentation Analysis

A product team ran a 50,000-user A/B test on a new homepage design and reports a +13% conversion lift (p=0.015). The CPO asks: 'Should we ship this to 100% of users?' You'll audit the experiment end-to-end: check Sample Ratio Mismatch, measure 95% confidence bounds against MDE, uncover device segment anomalies (mobile friction), verify revenue guardrail stability with Welch's t-test, and author a definitive 1-page executive decision memo.

DatabaseCloud Data Store
ToolchainJupyter
Milestones5 Guided Stages
Recruiter OutputVerified GitHub Repo
View 5 Milestones
Full project workspace, live database credentials, AI mentor & verified certificate
Project Deliverables

What You Will Ship to GitHub

Hiring managers don't want to see another cookie-cutter Titanic or Iris tutorial. You will build a professional repository with modular scripts and executive business memos that demonstrate commercial analytical maturity.

github.com/your-username/ab-testing-product-analyticsPortfolio Structure
├── sql/
01_exploratory_profiling.sql // Schema audits, table volume, date ranges
02_revenue_trends.sql // Monthly GMV, order volume & AOV evolution
03_customer_concentration.sql // Pareto 80/20 distribution, repeat buyer cohorts
├── findings/
00_executive_summary.md // 3 core business takeaways for executive stakeholders
01_business_health.md // Detailed breakdown of margins, cancellations, and seasonality
03_customer_segments.md // Customer lifetime spend & cohort concentration
├── README.md // Professional project front-door with data schema diagram
Outcomes

What You Will Master

Deduplicate raw clickstream event logs and isolate step-by-step conversion drop-offs
Construct triangular cohort retention matrices and visualize decay with Seaborn heatmaps
Evaluate binary conversion rates with two-sample proportion Z-tests and 95% Wald CIs
Verify revenue guardrail stability with Welch's two-sample T-test (equal_var=False)
Audit experimental integrity using Chi-square goodness-of-fit for Sample Ratio Mismatch (SRM)
Detect temporal novelty effect decay across multi-week experiment windows
Structure and defend a 1-page executive decision memo using the Minto SCR framework
Curriculum

5 Guided Project Milestones

M1

Set up your project repo

Before any analytical work: create a public GitHub repository, push the standard skeleton, and paste the URL into your workspace.

~30 mins
M2

Multi-Step Funnel & Friction Analysis

Ingest user clickstream logs, deduplicate multi-session events, compute step-over-step conversions, and isolate mobile drop-off friction.

~90 mins
M3

Monthly Cohort Retention & Lifecycle Decay

Core Milestone

Construct triangular monthly cohort retention matrices and visualize user lifecycle stabilization using Seaborn heatmaps.

~90 mins
M4

A/B Test Statistical Evaluation & Diagnostics

Ingest 50k experiment rows, compute two-sample proportion Z-tests, 95% Wald CIs, Welch's ARPU tests, Sample Ratio Mismatch checks, and novelty fade.

~90 mins
M5

Executive Decision Memo & Portfolio Deliverable

Review Pass Required

Synthesize findings into a high-stakes 1-page CPO executive recommendation memo and assemble your polished GitHub portfolio repository.

~60 mins
Career Value

ATS Keywords & Resume Ready Bullets

Tools
Pythonpandasscipy.statsstatsmodelsseabornJupyterGit/GitHub
Methods
Common Table Expressions (CTEs)Window FunctionsTime-Bucket AggregationsCohort AnalysisData Profiling & Auditing
Business Concepts
AOV (Average Order Value)Customer ConcentrationRepeat Purchase BehaviorCohort RetentionRevenue Health Metrics
Pre-Written Resume Bullet (Copy & Paste)

"Authored an end-to-end data analytics project on A/B Testing & Product Experimentation Analysis utilizing Python, pandas, scipy.stats — delivered production-grade queries, cohort distributions, and an executive stakeholder summary published as a portfolio GitHub repository."

Preparation

Reading & References

A/B Testing Dataset (homepage_ab_test.csv)
article
A/B Testing & Product Analytics Master Starter Notebook
article
Ronny Kohavi - Trustworthy Online Controlled Experiments Guide
article

Ready to build and ship A/B Testing & Product Experimentation Analysis?

Instant access to cloud database credentials, milestones, and starter queries.

Project Overview

Free A/B Testing & Product Experimentation Analysis portfolio project — with AI review

Build a real a/b testing & product experimentation analysis project for your data analyst portfolio in your own public GitHub repo — free brief, starter template, and milestone guides. When your code is ready, the ₹99 pass gets every milestone reviewed against a fixed rubric (ab testing product analytics walkthrough included) and issues a verifiable certificate on completion.

Is the A/B Testing & Product Experimentation Analysis project free?

The full project brief, milestone guides, and starter template are free to audit. The AI-powered review pass and verified certificate unlock at ₹99.

Can I add the A/B Testing & Product Experimentation Analysis project to my resume and GitHub?

Yes — that is the point. You build in your own public GitHub repo, every milestone is reviewed against a fixed rubric, and the certificate links to the repo employers can inspect. Only list what is visible in your repo.

How long does the A/B Testing & Product Experimentation Analysis project take?

Most students finish the 5 milestones in 2–4 weekends. Milestones unlock in order, and you can re-submit any milestone that needs work.

Rubric-reviewed · Plagiarism-checked · Verifiable certificate

Build your portfolio & get certified

Want more projects like this? Join our Data Analyst Work Experience Program to complete 5+ guided industry projects, gain real experience, and earn an internship certificate.

Join Work Experience
Access
₹99