Refresh target: write the SRM + peeking + readout section of an experiment memo that survives a senior's follow-up questions. Pointer: stats-product-analytics; WE W2.
1. Before launch: power and sample size
State the primary metric, minimum detectable effect, power (0.8), and alpha (0.05) with the resulting required sample size and runtime. No launch without it.
2. During the run: SRM and peeking guards
Check Sample Ratio Mismatch on day 1 and day 3 (chi-square on the split). Never stop early on a significant p-value: pre-commit to the runtime or use sequential / CUPED variance reduction declared up front.
3. Readout structure (exec-style)
Decision first ("Ship / Don't ship / Inconclusive + next step"), then effect size with confidence interval, then guardrail metrics, then threats (novelty, weekend effects, Simpson splits by platform). One paragraph per section.
Recap
Pre-commit power/sample size, SRM checks with no peeking, and a decision-first readout with effect size, guardrails, and threat analysis.