Brief
Defend one experiment readout on homepage_ab_test.csv: power/sample-size
pre-commit, SRM check, no-peeking statement, decision-first memo with effect
size + CI, guardrails, and threat analysis.
Dataset
homepage_ab_test.csv + AB_Testing playbook. Holdout: a second experiment
window used to check your method transfers (not your numbers).
Starter guardrails (88)
- Use the readout template (Decision / Effect / Guardrails / Threats).
- Pre-commit block (metric, MDE, power, alpha, runtime) must precede results.
Deterministic checks
SRM chi-square recomputes; CI arithmetic correct; template sections all present.
100-pt rubric
- Pre-commit completeness: 20
- SRM + peeking discipline: 20
- Effect size + CI correctness: 20
- Guardrails + threats (novelty/Simpson): 20
- Exec clarity (decision in first paragraph): 20