Two SaaS apps have the same mean API latency: 200 ms. App A's latency is tightly clustered between 180–220 ms. App B's latency is mostly 100 ms but occasionally spikes to 5 seconds. Same mean. Very different user experience. Spread is what separates them.
The three numbers you'll actually use
Standard deviation (std) - average distance from the mean. Easy to compute, easy to communicate ("most users are within ±2 std of the mean"). Works well for symmetric data; less useful when distributions are skewed.
Variance - std squared. You compute variance to get std. You rarely report variance directly; report std instead.
Percentiles - p50 is the median. p90 is the value below which 90% of observations fall. p99 is the same for 99%. Percentiles describe spread without assuming the data is normal - they always work.
Computing quantiles & spread in Python
import numpy as np
import pandas as pd
# Simulated API latency distributions for 100,000 requests
np.random.seed(42)
# App A: tight bell curve (mean 200ms, std 20ms)
latency_app_a = np.random.normal(loc=200, scale=20, size=100000)
# App B: log-normal with long tail spikes (mean ~200ms, huge tail)
latency_app_b = np.concatenate([
np.random.normal(loc=120, scale=15, size=98000),
np.random.uniform(low=2000, high=5000, size=2000)
])
df_b = pd.DataFrame({"latency_ms": latency_app_b})
# Percentiles and spread metrics
p50, p90, p95, p99 = df_b["latency_ms"].quantile([0.50, 0.90, 0.95, 0.99])
mean_lat = df_b["latency_ms"].mean()
std_lat = df_b["latency_ms"].std()
q25, q75 = df_b["latency_ms"].quantile([0.25, 0.75])
iqr = q75 - q25
print("--- App B Latency Spread Analysis ---")
print(f"Mean ± Std: {mean_lat:.1f}ms ± {std_lat:.1f}ms (Std > Mean signals long tail)")
print(f"IQR (p75-p25): {iqr:.1f}ms (Robust spread metric)")
print(f"p50 (Median): {p50:.1f}ms (Typical user experience)")
print(f"p90: {p90:.1f}ms")
print(f"p99: {p99:.1f}ms (1 in 100 users waits {p99/1000:.1f}s+)")
The engineering convention: p50, p90, p99
Engineering and ops teams almost always report latency, response time, and queue depth as p50 / p90 / p99 instead of mean.
Why? Because for latency:
- The mean hides the bad cases by averaging them with the fast ones.
- The p99 tells you "1 in 100 users had an experience this bad or worse."
If your p99 page load is 8 seconds, 1% of your users wait 8 seconds. At 10K daily users, that's 100 frustrated people every day. Reporting only the mean of 400 ms hides this entirely.
When to use std vs IQR
Std (standard deviation) - when the data looks roughly bell-shaped. The ±1 std rule covers ~68% of data; ±2 std covers ~95%.
IQR (interquartile range, p75 − p25) - when the data is skewed. IQR is robust to outliers the same way median is. Box plots show IQR by design.
A quick rule: if mean ≈ median, std is fine. If mean is much greater than median, switch to IQR + percentiles.
Takeaway
For any metric you report:
- If the distribution is roughly bell-shaped → mean + std is enough.
- If skewed (revenue, latency, durations) → median + percentiles (p50/p90/p99) is more honest.
- Always check: does the std make sense relative to the mean? If std is bigger than the mean, the distribution is almost certainly skewed and your mean is misleading.