Reading Spread Like an Analyst

# Reading Spread Like an Analyst Two SaaS apps have the same mean API latency: 200 ms. App A's latency is tightly clustered between 180–220 ms. App B's latency is mostly 100 ms but occasionally spikes to 5 seconds. Same mean. Very different user experience. **Spread** is what separates them. ## The three numbers you'll actually use **Standard deviation (std)** - average distance from the mean. Easy to compute, easy to communicate ("most users are within ±2 std of the mean"). Works well for symmetric data; less useful when distributions are skewed. **Variance** - std squared. You compute variance to get std. You rarely report variance directly; report std instead. **Percentiles** - `p50` is the median. `p90` is the value below which 90% of observations fall. `p99` is the same for 99%. Percentiles describe spread without assuming the data is normal - they always work. ## Computing quantiles & spread in Python ```python import numpy as np import pandas as pd # Simulated API latency distributions for 100,000 requests np.random.seed(42) # App A: tight bell curve (mean 200ms, std 20ms) latency_app_a = np.random.normal(loc=200, scale=20, size=100000) # App B: log-normal with long tail spikes (mean ~200ms, huge tail) latency_app_b = np.concatenate([ np.random.normal(loc=120, scale=15, size=98000), np.random.uniform(low=2000, high=5000, size=2000) ]) df_b = pd.DataFrame({"latency_ms": latency_app_b}) # Percentiles and spread metrics p50, p90, p95, p99 = df_b["latency_ms"].quantile([0.50, 0.90, 0.95, 0.99]) mean_lat = df_b["latency_ms"].mean() std_lat = df_b["latency_ms"].std() q25, q75 = df_b["latency_ms"].quantile([0.25, 0.75]) iqr = q75 - q25 print("--- App B Latency Spread Analysis ---") print(f"Mean ± Std: {mean_lat:.1f}ms ± {std_lat:.1f}ms (Std > Mean signals long tail)") print(f"IQR (p75-p25): {iqr:.1f}ms (Robust spread metric)") print(f"p50 (Median): {p50:.1f}ms (Typical user experience)") print(f"p90: {p90:.1f}ms") print(f"p99: {p99:.1f}ms (1 in 100 users waits {p99/1000:.1f}s+)") ``` ## The engineering convention: p50, p90, p99 Engineering and ops teams almost always report latency, response time, and queue depth as `p50 / p90 / p99` instead of mean. Why? Because for latency: - The *mean* hides the bad cases by averaging them with the fast ones. - The *p99* tells you "1 in 100 users had an experience this bad or worse." If your p99 page load is 8 seconds, 1% of your users wait 8 seconds. At 10K daily users, that's 100 frustrated people every day. Reporting only the mean of 400 ms hides this entirely. ## When to use std vs IQR **Std (standard deviation)** - when the data looks roughly bell-shaped. The ±1 std rule covers ~68% of data; ±2 std covers ~95%. **IQR (interquartile range, p75 − p25)** - when the data is skewed. IQR is robust to outliers the same way median is. Box plots show IQR by design. A quick rule: if mean ≈ median, std is fine. If mean is much greater than median, switch to IQR + percentiles. ## Takeaway For any metric you report: - If the distribution is roughly bell-shaped → mean + std is enough. - If skewed (revenue, latency, durations) → median + percentiles (p50/p90/p99) is more honest. - Always check: does the std make sense relative to the mean? If std is bigger than the mean, the distribution is almost certainly skewed and your mean is misleading.