SUBMISSION: Capstone 1 — Model Interpretability & Bias Audit with SHAP Values

🎯 Hands-on Portfolio Deliverable (Capstone). Accuracy gets the demo; explainability gets the signature. This capstone proves you can open a black-box booster, defend individual decisions, and catch proxy discrimination before a regulator does.

🏢 Business Scenario & Executive Prompt

SahayCredit approves 60K micro-loans/month with an XGBoost scorer (AUC 0.83). Internal review flags a pattern: approval rates for applicants from Tier-3 cities run 14 points below Tier-1 at equal credit scores. The risk head asks: "Is the model biased, which features drive it, and what do we change — with proof the fix doesn't crater portfolio quality?" Your audit lands on the risk committee's desk.

📋 Source Data & Input Schemas

sahaycredit_apps.csv — 240,000 applications (mock shape; generate with the scaffold):

Expected shapes: Tier-3 applicants carry lower scores on average (confounding!); purpose=medical defaults more often.

🎯 Required Deliverables

1. shap_audit.ipynb — runnable: tuned XGBoost, global + local SHAP, segment-sliced importance, mitigation A/B.
2. best_model.pkl — final approved scorer (post-mitigation) with model_features.json.
3. fairness_table.csv — per-city_tier: approval rate, TPR, FPR, mean SHAP of top-5 features.
4. model_card.md — intended use, metrics + calibration, explainability summary, bias findings, mitigation, residual risks, monitoring plan.

🛠️ Step-by-Step Implementation Guide

1. Baseline honestly. Stratified split; XGBoost with early stopping on validation; report ROC-AUC, PR-AUC, Brier, and calibration — before any fairness talk.
2. Explain globally. shap.TreeExplainer: summary + dependence plots for the top features. Separate predictive signal from proxy signal.
3. Explain locally. Waterfall plots for 3 approved + 3 denied edge cases (near-threshold). Write the one-paragraph reason code for each, as a regulator would demand.
4. Slice by segment. Recompute importance and error rates per city_tier. Decompose the 14-point gap: how much is score-mix (confounding) vs model behavior? A feature whose SHAP pushes one segment down is a finding.
5. Mitigate and re-measure. Candidate fixes: drop/quarantine the proxy, reweigh, threshold per segment with equalized-odds check. Ship the variant with the best fairness-quality frontier — document what you traded.
6. Write the model card. Decision, numbers, residual risks (what you did NOT fix), and drift monitors (approval-rate gap alert at ±5 points).

💻 Complete Runnable Starter Scaffold

import pandas as pd, shap, joblib
from sklearn.model_selection import train_test_split
from xgboost import XGBClassifier

df = pd.read_csv("sahaycredit_apps.csv")
X = pd.get_dummies(df.drop(columns=["app_id", "approved_repaid"]), dtype=float)
y = df["approved_repaid"]
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.2, stratify=y, random_state=7)

clf = XGBClassifier(n_estimators=500, learning_rate=0.05, max_depth=5,
                    subsample=0.8, colsample_bytree=0.8,
                    scale_pos_weight=(ytr == 0).sum() / (ytr == 1).sum(),
                    early_stopping_rounds=50, eval_metric="logloss", random_state=7)
clf.fit(Xtr, ytr, eval_set=[(Xte, yte)], verbose=False)

ex = shap.TreeExplainer(clf)
sv = ex(Xte)  # per-row, per-feature attributions
# Global: shap.plots.beeswarm(sv, max_display=12)
# Local:  shap.plots.waterfall(sv[row_idx], max_display=10)
# Sliced: sv[df.loc[Xte.index, "city_tier"] == "T3"].abs.mean(0) vs T1
joblib.dump(clf, "best_model.pkl")

🧪 Acceptance Criteria & Verification Checks

• Notebook shows global SHAP (top features + dependence) and ≥ 6 local waterfall reason codes.
• fairness_table.csv quantifies the Tier-3 gap with approval rate, TPR/FPR, and sliced SHAP — confounding separated from model effect.
• A mitigation variant is measured on both quality (PR-AUC/Brier within agreed tolerance) and fairness (gap narrowed); trade-off is explicit.
• model_card.md covers use, metrics, findings, mitigation, residual risks, and monitoring thresholds.

📊 Evaluation Rubric (100 points — passing ≥ 80)

📤 Submission Guidelines

1. Upload your files (.ipynb, .py, .pkl, .csv, .md, .png) via the file dropzone on this lesson.
2. Paste your public GitHub repo URL in the submission notes (repo must contain the same files + a 5-line README with the run command).
3. Name files exactly as listed under Required Deliverables — review checks filenames first.
4. Passing score is 80/100. You may resubmit; the highest score counts.