Distributions That Matter in DA Work

This lesson on Distributions That Matter in DA Work is hands-on and example-driven. You will identify and interpret the core structural properties of the normal distribution on graphical plots. You will locate the mean, evaluate data dispersion via standard deviation, and apply the empirical rule to estimate relative probabilities across analytics datasets.

What You'll Be Able To Do

• Identify the mean of a symmetric, bell-shaped distribution directly from its center peak.
• Evaluate dataset dispersion by comparing curve width and peak height across population groups.
• Interpret vertical axis values as relative probability density rather than absolute event probability.
• Apply the empirical rule to calculate data proportions within two standard deviations of the mean.
• Categorize real-world metrics like commute duration and physical measurements by variance and spread.

Detailed Concept Walkthrough

1. Anatomy of the Normal Distribution

The normal (Gaussian) distribution is a symmetric, bell-shaped probability distribution centered on the population mean. It models continuous variables where values cluster symmetrically around a central tendency.

• Mechanism: The highest point of the curve aligns precisely with the mean (average), indicating the value with the greatest relative likelihood. The left and right tails taper off symmetrically as values diverge further from the center.
• Under the Hood: Because the curve is completely symmetrical, the central peak represents the mean, median, and mode simultaneously. Deviations to the positive side mirror deviations to the negative side with identical probability mass.
• Best Practice: When plotting empirical product metrics, check for symmetry around the central peak before assuming Gaussian properties for downstream statistical modeling.

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import norm

# Generate a standard normal distribution centered at 0
x = np.linspace(-4, 4, 1000)
y = norm.pdf(x, loc=0, scale=1)

# Plot distribution
plt.plot(x, y, label='Mean=0, SD=1')
plt.axvline(0, color='red', linestyle='--', label='Mean (Center)')
plt.legend()
plt.show()

Key Takeaway: The normal distribution is defined by a central peak at the mean with symmetric, asymptotic tails on both sides.

2. Standard Deviation and Curve Morphology

Standard deviation measures the spread or dispersion of data points, directly dictating both the width and peak height of the normal curve.

• Mechanism: A small standard deviation produces a steep, narrow curve because observations are tightly clustered around the mean. A large standard deviation widens the base and flattens the peak because values are spread across a broader range.
• Under the Hood: The total area under any valid probability density curve must equal 1 (100% total probability). Consequently, widening the distribution along the horizontal axis forces the vertical peak downward to maintain a constant total area.
• Best Practice: Contrast distinct cohorts (e.g., newborn infant heights vs. adult heights) by checking standard deviation; narrower curves signal lower variance in the underlying population.

import numpy as np
from scipy.stats import norm

# Compare narrow (low variance) vs wide (high variance) curves
x_axis = np.linspace(30, 90, 500)

# Newborns: mean=50cm, sd=2cm (narrow)
baby_curve = norm.pdf(x_axis, loc=50, scale=2)

# Adults: mean=65cm, sd=8cm (wide)
adult_curve = norm.pdf(x_axis, loc=65, scale=8)

Key Takeaway: Larger standard deviations produce wider, flatter curves, while smaller standard deviations create narrower, taller peaks.

3. Relative Probability and the Y-Axis

The vertical axis on a normal distribution graph represents probability density (relative probability), showing the comparative likelihood of different values occurring.

• Mechanism: Higher points along the curve reflect values that occur more frequently within the dataset. Values under the tails have low vertical values, showing they occur rarely.
• Under the Hood: For continuous distributions, the probability of hitting an exact, infinite-precision decimal value is mathematically zero. The y-axis shows density, meaning probabilities are evaluated over intervals (areas under the curve) rather than single discrete points.
• Best Practice: Never read the y-axis of a continuous distribution as an absolute percentage chance; use it to compare the relative likelihood between two candidate outcomes.

# Evaluate relative likelihoods via probability density function (PDF)
mean_commute = 30
sd_commute = 5

prob_at_mean = norm.pdf(30, loc=mean_commute, scale=sd_commute)
prob_at_tail = norm.pdf(45, loc=mean_commute, scale=sd_commute)

# Relative likelihood ratio
relative_ratio = prob_at_mean / prob_at_tail
print(f"Mean is {relative_ratio:.2f}x more likely than +3 SD tail")

Key Takeaway: The y-axis represents relative probability density, where higher values indicate more common outcomes.

4. The 68-95-99.7 Empirical Rule

The empirical rule defines fixed proportions of data that fall within integer standard deviation intervals from the mean in any normal distribution.

• Mechanism: Approximately 68% of all data points fall within ±1 standard deviation of the mean. Approximately 95% of data falls within ±2 standard deviations, and 99.7% falls within ±3 standard deviations.
• Under the Hood: This rule holds true regardless of the specific numerical values of the mean or standard deviation. It provides an immediate baseline for bounding expected variation in normally distributed processes.
• Best Practice: Use the 2-standard-deviation threshold (95% interval) as a quick heuristic to isolate expected operational variation from potential outliers.

# Calculate 95% empirical interval (mean +/- 2 SD)
mean = 100
sd = 15

lower_bound = mean - 2 * sd  # 70
upper_bound = mean + 2 * sd  # 130

print(f"95% of observations fall between {lower_bound} and {upper_bound}")

Key Takeaway: Exactly 95% of observations in any normal distribution lie within two standard deviations of the mean.

Topics Covered in Distributions That Matter in DA Work

1. Normal Curve Geometry (0:35 - 1:00) — The characteristic bell-shaped curve and its horizontal symmetry are introduced.
2. Relative Probability Density (1:00 - 1:50) — The vertical y-axis is established as relative probability rather than absolute probability.
3. Locating Central Tendency (1:50 - 2:40) — The mean is identified as the exact center and highest point on the curve.
4. Standard Deviation and Width (2:40 - 3:00) — Standard deviation is shown to govern the spread, width, and vertical peak height.
5. 95% Empirical Rule (3:00 - 4:15) — The principle that 95% of data falls within two standard deviations is demonstrated.
6. Real-World Variable Comparisons (4:15 - 5:13) — Height variations across infants and adults alongside commute times illustrate practical use cases.

Stats & Product Analytics for Analysts Cheat Sheet

• scipy.stats.norm.pdf(x, loc, scale) — Computes probability density (relative likelihood) at point x

  density = norm.pdf(50, loc=50, scale=5)
  

• scipy.stats.norm.interval(alpha, loc, scale) — Returns confidence range spanning specified coverage fraction

  low, high = norm.interval(0.95, loc=100, scale=15)
  

• numpy.random.normal(loc, scale, size) — Generates random samples drawn from normal distribution

  samples = np.random.normal(loc=0, scale=1, size=1000)
  

• Mean - 2 * SD, Mean + 2 * SD — Calculates manual 95% empirical coverage interval

  bounds = (mean - 2*sd, mean + 2*sd)
  

Comparison Table

Common Pitfalls

• Mistake: Interpreting y-axis density values as exact percentage probabilities. Avoid: Treat continuous y-axis values solely as relative likelihood indicators.
• Mistake: Assuming a taller peak represents a larger total dataset size. Avoid: Recognize total area remains constant at one across all curves.
• Mistake: Applying the 95% two-standard-deviation rule to skewed metrics. Avoid: Verify bell-shaped symmetry before applying empirical standard deviation rules.

FAQs

• Why does the curve get shorter when the standard deviation increases? Because the total area under any probability density curve must equal 1.0, spreading the data horizontally forces the peak height downward.
• What metric sits directly underneath the highest peak of the curve? The mean of the distribution sits at the exact peak and center of a symmetric normal distribution.
• How much data is expected to fall within two standard deviations of the mean? Approximately 95% of all observations in a normal distribution fall within two standard deviations (±2 SD) of the mean.
• Why do newborn baby heights show a narrower curve than adult heights? Newborn babies exhibit much lower variance in length, producing a smaller standard deviation and a narrower, taller distribution curve.