A/B-Test Writeup (AB-Test49 Pattern)

# A/B-Test Writeup (AB-Test49 Pattern) Refresh target: write the SRM + peeking + readout section of an experiment memo that survives a senior's follow-up questions. Pointer: stats-product-analytics; WE W2. ## 1. Before launch: power and sample size State the primary metric, minimum detectable effect, power (0.8), and alpha (0.05) with the resulting required sample size and runtime. No launch without it. ## 2. During the run: SRM and peeking guards Check Sample Ratio Mismatch on day 1 and day 3 (chi-square on the split). Never stop early on a significant p-value: pre-commit to the runtime or use sequential / CUPED variance reduction declared up front. ## 3. Readout structure (exec-style) Decision first ("Ship / Don't ship / Inconclusive + next step"), then effect size with confidence interval, then guardrail metrics, then threats (novelty, weekend effects, Simpson splits by platform). One paragraph per section. ## Recap Pre-commit power/sample size, SRM checks with no peeking, and a decision-first readout with effect size, guardrails, and threat analysis.