Brief
Ship a calibrated classifier on marketing-campaign response with a threshold memo tied to the outreach cost matrix. WE W3-W4 eval framing.
Dataset
Campaign train/holdout. Trap: temporal leakage (campaign wave id encodes the outcome window) — split by time, not at random.
Starter guardrails (88)
- Skeleton slots: time-split, model comparison (RF vs boosting), calibration, threshold memo.
- Threshold frozen on validation before any holdout scoring.
Deterministic checks
Time-split respected (no future rows in train); PR-AUC + Brier recompute; threshold memo cites the cost matrix; holdout scored once.
100-pt rubric
- Leakage control (time split + wave-id handling): 25
- Model comparison quality: 15
- Calibration (reliability evidence): 20
- Threshold-from-costs memo: 20
- Holdout PR-AUC/Brier thresholds met: 15
- Reproducibility: 5