SUBMISSION: Project 4 — Multivariable Regression with Multicollinearity Diagnostics (VIF) & Interaction Terms

🎯 Hands-on Portfolio Deliverable. Regression is the most-used model in DS — and the most misread. This project proves you check assumptions instead of quoting R².

🏢 Business Scenario & Executive Prompt

CloudNine SaaS (50,000 accounts, 4.8% quarterly churn) wants to know why customers leave. The PM's draft model regresses churn on 12 features and reports R² = 0.31 with "all p-values significant." You spot sessions_per_week, logins_per_week, and active_days_per_week in the same model — plus a suspicion that tenure effects differ by plan tier. The VP Sales asks: "Which three levers actually move churn, and how much do you trust each number?"

📋 Source Data & Input Schemas

cloudnine_churn.csv — 50,000 rows (mock shape; scaffold generates it):

🎯 Required Deliverables

1. churn_regression.ipynb — diagnostics → VIF triage → interactions → validation notebook.
2. churn_model.py — final OLS specification (formula + fit + get_summary() printing VIF table).
3. vif_report.csv — feature, vif_before, vif_after, action (keep \| drop \| combine).
4. regression_memo.md — three levers with per-lever effect sizes + CIs, assumption verdicts, and what you would NOT claim.

🛠️ Step-by-Step Implementation Guide

1. Baseline + residual diagnostics. Fit OLS on all 12 features (statsmodels). Residual-vs-fitted, Q-Q, scale-location plots. Name each violated assumption in plain language.
2. VIF triage. Compute VIF for all features; the engagement trio will print VIF > 10. Decide per feature: drop, combine (e.g., engagement_index = z-mean), or keep-with-HC-SEs. Record in vif_report.csv; refit and show coefficient stability (before/after table).
3. Interactions. Test tenure_months × plan (enterprise tenure behaves differently). F-test for the interaction block; interpret one interaction coefficient as "for enterprise, each extra month changes churn log-odds by…".
4. Partial-effect interpretation. Every reported coefficient stated as "holding others constant" with 95% CI. One paragraph on why the PM's "all p-values significant" claim is wrong under multicollinearity.
5. Memo. Three levers ranked by standardized effect; honest limits (observational, no causation).

💻 Production Starter Scaffold

import numpy as np, pandas as pd
import statsmodels.api as sm
from statsmodels.stats.outliers_influence import variance_inflation_factor

rng = np.random.default_rng(21)
N = 50_000
tenure = rng.integers(1, 60, N)
plan = rng.choice(["basic","pro","enterprise"], N, p=[.55,.3,.15])
sess = rng.gamma(4, 1.2, N) + (plan=="enterprise")*2
df = pd.DataFrame({
    "acct_id": [f"CN-{i:05d}" for i in range(N)],
    "tenure_months": tenure, "plan": plan,
    "sessions_pw": sess,
    "logins_pw": sess*1.4 + rng.normal(0,.5,N),       # collinear by construction
    "active_days_pw": (sess/2).clip(0,7) + rng.normal(0,.3,N),
    "support_tickets": rng.poisson(1.2, N),
    "contract_value_inr": np.round(rng.lognormal(10.5,.8,N),2),
})
logit = -3 + -0.02*tenure + (plan=="pro")*-0.4 + (plan=="enterprise")*-0.9 \
        + 0.15*df.support_tickets - 0.1*sess + (plan=="enterprise")*tenure*-0.01
df["churned"] = (rng.uniform(size=N) < 1/(1+np.exp(-logit))).astype(int)
print("churn rate:", df.churned.mean().round(4))

# --- VIF triage ---
X = pd.get_dummies(df[["tenure_months","sessions_pw","logins_pw","active_days_pw",
                        "support_tickets","contract_value_inr","plan"]], drop_first=True).astype(float)
vif = pd.DataFrame({"feature": X.columns,
    "vif": [variance_inflation_factor(X.to_numpy(), i) for i in range(X.shape[1])]})
print(vif.sort_values("vif", ascending=False).head(8))

🧪 Acceptance Criteria & Verification Checks

• vif_report.csv lists all features with before/after VIF and a justified action per feature.
• Final model has max VIF < 5 and a before/after coefficient-stability table in the notebook.
• tenure × plan interaction tested with an F-test; one interaction coefficient interpreted correctly.
• Residual diagnostic trio (residual-vs-fitted, Q-Q, scale-location) present with verdicts.
• Memo names three levers with CIs and explicitly disclaims causal reading.

📊 Evaluation Rubric (100 points — passing ≥ 80)

📤 Submission Guidelines

1. Upload your files (.ipynb, .csv, .md, .png) via the file dropzone on this lesson.
2. Paste your public GitHub repo URL in the submission notes (repo must contain the same files + a 5-line README with the run command).
3. Name files exactly as listed under Required Deliverables — autograding checks filenames first.
4. Passing score is 80/100. You may resubmit; the highest score counts.