ML Foundations
Back to ML Foundations: The Mental Model

SUBMISSION: Milestone Project 1 — Problem Formulation: Framing a Real Business Problem as an ML Task

Turn a vague retention complaint at a D2C subscription business into a leakage-free ML task: label definition, observation windows, success metric, and a costed baseline plan.

131 minutesProject submission
🎯 Free Guest Mode: You are learning for free. Sign in to save your completion progress and quiz answers.

Assignment Instructions

🎯 Hands-on Portfolio Deliverable. Before any model: the highest-leverage ML skill is turning "churn is up, do something" into a solvable, measurable task. This brief is the artifact you will show for "tell me how you scope an ML problem."

🏢 Business Scenario & Executive Prompt

You are the first ML hire at DailyKart, a D2C grocery subscription service (400K subscribers). The CEO says: "Churn jumped from 6% to 9% this quarter. Build a churn model." The retention team can call 5,000 customers/month. Nothing else is defined — no label, no metric, no baseline.

Your job: produce the task definition a team can execute without rework. Ambiguity you leave here becomes leakage and wasted modeling later.

📋 Source Data & Input Schemas

dailykart_events.csv — 2.1M rows (mock shape; generate with the scaffold):

columndtypedescription
customer_idstrunique subscriber (C-000001 …)
event_tsdatetimeevent timestamp (IST)
event_typecategoryorder | skip | pause | support_ticket | payment_failed
basket_value_inrfloatorder value (> 0 for orders)
plancategoryweekly | monthly
acquired_ondatesubscription start date

customers.csv — 400K rows: customer_id, acquired_on, city_tier, acquisition_channel.

Expected shapes: median tenure 7 months; ~9% of customers lapse (no order in 45 days) in the latest quarter.

🎯 Required Deliverables

  1. problem_framing.md — label definition, observation/prediction windows, success metric + why, baseline to beat, risks.
  2. labels.csv — customer_id, observation_end, label (0/1) for one cohort (≥ 50K rows), built by your code.
  3. baseline.ipynb — runnable notebook: cohort build, dummy/majority baseline scores, metric table.
  4. metric_memo.md — one page: why accuracy is rejected, chosen operating metric, cost math for the 5,000-call capacity.

🛠️ Step-by-Step Implementation Guide

  1. Define the label. Pick an observable lapse event (e.g. no order within 45 days after observation end). Justify the horizon from business cost, not convenience.
  2. Fix the windows. Observation window (features allowed) vs prediction window (label). Draw the timeline diagram; state the as-of rule in one sentence.
  3. Leakage-proof the cohort. Every feature timestamp must be ≤ observation_end. Write an assertion that fails the build if violated.
  4. Choose the metric. Reject accuracy with the 91%-majority math; select precision@K / recall@K tied to 5,000 calls/month and cost per call vs save value.
  5. Set the baseline. Majority-class and rule-based (e.g. "payment_failed recently") baselines with the chosen metric — the bar every later model must clear.
  6. Write the memo. One page a PM can sign: what you predict, when, how success is measured, what could invalidate it.

💻 Complete Runnable Starter Scaffold

import pandas as pd

OBS_END = "2026-04-30"   # as-of point: NOTHING after this may enter features
PRED_WINDOW_DAYS = 45

events = pd.read_csv("dailykart_events.csv", parse_dates=["event_ts"])
ev = events[events["event_ts"] <= OBS_END]  # leakage guardrail #1

# Label: no order in the 45 days after OBS_END (uses future ONLY for labels)
future_orders = events[(events["event_ts"] > OBS_END) & (events["event_type"] == "order")]
lapsed_ids = set(ev["customer_id"]) - set(future_orders["customer_id"])
labels = pd.DataFrame({"customer_id": sorted(set(ev["customer_id"]))})
labels["observation_end"] = OBS_END
labels["label"] = labels["customer_id"].isin(lapsed_ids).astype(int)

assert (pd.to_datetime(labels["observation_end"]) >= ev.groupby("customer_id")["event_ts"].max().reindex(labels["customer_id"]).values).all()
labels.to_csv("labels.csv", index=False)
print(labels["label"].mean(), len(labels))  # expect ~0.09, >= 50000 rows

🧪 Acceptance Criteria & Verification Checks

  • labels.csv has ≥ 50K rows with a documented lapse rate near 9%.
  • Build script asserts all feature timestamps ≤ observation_end (leakage guard runs green).
  • metric_memo.md rejects accuracy with majority-class math and ties the metric to the 5,000-call capacity.
  • Baseline notebook runs top-to-bottom and reports dummy + rule-based scores on the chosen metric.

📊 Evaluation Rubric (100 points — passing ≥ 80)

#CriterionPointsWhat earns full marks
1Modeling correctness25Right algorithm for the task; leakage-free splits/pipelines; metrics match the business decision; numbers reproducible
2Code quality & reproducibility25Seeded RNG, re-runnable notebook/scripts, typed schemas, no hardcoded paths or magic numbers
3Business reasoning & memo25Decision-oriented write-up with costed trade-offs; honest about uncertainty, limits, and risks
4Visualization & communication25Labeled ROC/PR, calibration, SHAP or residual plots where relevant; figures referenced in the narrative

📤 Submission Guidelines

  1. Upload your files (.ipynb, .py, .pkl, .csv, .md, .png) via the file dropzone on this lesson.
  2. Paste your public GitHub repo URL in the submission notes (repo must contain the same files + a 5-line README with the run command).
  3. Name files exactly as listed under Required Deliverables — review checks filenames first.
  4. Passing score is 80/100. You may resubmit; the highest score counts.
Checking your access…