🎯 Hands-on Portfolio Deliverable. Before any model: the highest-leverage ML skill is turning "churn is up, do something" into a solvable, measurable task. This brief is the artifact you will show for "tell me how you scope an ML problem."
🏢 Business Scenario & Executive Prompt
You are the first ML hire at DailyKart, a D2C grocery subscription service (400K subscribers). The CEO says: "Churn jumped from 6% to 9% this quarter. Build a churn model." The retention team can call 5,000 customers/month. Nothing else is defined — no label, no metric, no baseline.
Your job: produce the task definition a team can execute without rework. Ambiguity you leave here becomes leakage and wasted modeling later.
📋 Source Data & Input Schemas
dailykart_events.csv — 2.1M rows (mock shape; generate with the scaffold):
| column | dtype | description |
|---|---|---|
customer_id | str | unique subscriber (C-000001 …) |
event_ts | datetime | event timestamp (IST) |
event_type | category | order | skip | pause | support_ticket | payment_failed |
basket_value_inr | float | order value (> 0 for orders) |
plan | category | weekly | monthly |
acquired_on | date | subscription start date |
customers.csv — 400K rows: customer_id, acquired_on, city_tier, acquisition_channel.
Expected shapes: median tenure 7 months; ~9% of customers lapse (no order in 45 days) in the latest quarter.
🎯 Required Deliverables
problem_framing.md— label definition, observation/prediction windows, success metric + why, baseline to beat, risks.labels.csv—customer_id, observation_end, label (0/1)for one cohort (≥ 50K rows), built by your code.baseline.ipynb— runnable notebook: cohort build, dummy/majority baseline scores, metric table.metric_memo.md— one page: why accuracy is rejected, chosen operating metric, cost math for the 5,000-call capacity.
🛠️ Step-by-Step Implementation Guide
- Define the label. Pick an observable lapse event (e.g. no order within 45 days after observation end). Justify the horizon from business cost, not convenience.
- Fix the windows. Observation window (features allowed) vs prediction window (label). Draw the timeline diagram; state the as-of rule in one sentence.
- Leakage-proof the cohort. Every feature timestamp must be ≤ observation_end. Write an assertion that fails the build if violated.
- Choose the metric. Reject accuracy with the 91%-majority math; select precision@K / recall@K tied to 5,000 calls/month and cost per call vs save value.
- Set the baseline. Majority-class and rule-based (e.g. "payment_failed recently") baselines with the chosen metric — the bar every later model must clear.
- Write the memo. One page a PM can sign: what you predict, when, how success is measured, what could invalidate it.
💻 Complete Runnable Starter Scaffold
import pandas as pd
OBS_END = "2026-04-30" # as-of point: NOTHING after this may enter features
PRED_WINDOW_DAYS = 45
events = pd.read_csv("dailykart_events.csv", parse_dates=["event_ts"])
ev = events[events["event_ts"] <= OBS_END] # leakage guardrail #1
# Label: no order in the 45 days after OBS_END (uses future ONLY for labels)
future_orders = events[(events["event_ts"] > OBS_END) & (events["event_type"] == "order")]
lapsed_ids = set(ev["customer_id"]) - set(future_orders["customer_id"])
labels = pd.DataFrame({"customer_id": sorted(set(ev["customer_id"]))})
labels["observation_end"] = OBS_END
labels["label"] = labels["customer_id"].isin(lapsed_ids).astype(int)
assert (pd.to_datetime(labels["observation_end"]) >= ev.groupby("customer_id")["event_ts"].max().reindex(labels["customer_id"]).values).all()
labels.to_csv("labels.csv", index=False)
print(labels["label"].mean(), len(labels)) # expect ~0.09, >= 50000 rows
🧪 Acceptance Criteria & Verification Checks
-
labels.csvhas ≥ 50K rows with a documented lapse rate near 9%. - Build script asserts all feature timestamps ≤ observation_end (leakage guard runs green).
-
metric_memo.mdrejects accuracy with majority-class math and ties the metric to the 5,000-call capacity. - Baseline notebook runs top-to-bottom and reports dummy + rule-based scores on the chosen metric.
📊 Evaluation Rubric (100 points — passing ≥ 80)
| # | Criterion | Points | What earns full marks |
|---|---|---|---|
| 1 | Modeling correctness | 25 | Right algorithm for the task; leakage-free splits/pipelines; metrics match the business decision; numbers reproducible |
| 2 | Code quality & reproducibility | 25 | Seeded RNG, re-runnable notebook/scripts, typed schemas, no hardcoded paths or magic numbers |
| 3 | Business reasoning & memo | 25 | Decision-oriented write-up with costed trade-offs; honest about uncertainty, limits, and risks |
| 4 | Visualization & communication | 25 | Labeled ROC/PR, calibration, SHAP or residual plots where relevant; figures referenced in the narrative |
📤 Submission Guidelines
- Upload your files (
.ipynb,.py,.pkl,.csv,.md,.png) via the file dropzone on this lesson. - Paste your public GitHub repo URL in the submission notes (repo must contain the same files + a 5-line
READMEwith the run command). - Name files exactly as listed under Required Deliverables — review checks filenames first.
- Passing score is 80/100. You may resubmit; the highest score counts.