Brief
Audit the leaky starter notebook (trap: date + id columns leak the target), fix it, and re-score. Show the before/after delta.
Dataset
Same housing split; leaky starter notebook provided with date + id features.
Starter guardrails (88)
- Audit template: suspect list -> ablation table -> fix -> re-score.
- Do not delete cells; append the fix so the delta is reviewable.
Deterministic checks
Holdout delta direction correct (leaky train-holdout gap collapses after fix); ablation numbers recompute.
100-pt rubric
- Suspect identification (date + id + reasoning): 25
- Ablation evidence per suspect: 25
- Correct fix (drop/group + pipeline): 20
- Before/after scoring honesty: 15
- Writeup clarity: 15