Back to Foundations: From Intuition to Mathematics

Sampling Distributions and the CLT

Why everything is normal eventually. The most important theorem in applied statistics. FIND_VIDEO: search 'central limit theorem sampling distribution' — recommended channel: StatQuest / 3Blue1Brown. Aim for 11 min or under.

8 minutesVideo LessonPDF notes

Watch Sampling Distributions and the CLT on YouTube

🎯 Free Guest Mode: You are learning for free. Sign in to save your completion progress and quiz answers.

Ready to continue?

Mark this lesson as complete when you're ready to proceed.

Key moments

CLT Introduction — The Central Limit Theorem (CLT) is the basis for much of statistics and is a simple concept.
Uniform Distribution Example — Sampling means repeatedly from a uniform distribution shows that the distribution of those means becomes normal.
Exponential Distribution Example — Sampling means repeatedly from an exponential distribution also results in a normally distributed set of means.
Generalizing the Principle — The CLT states that the means will be normally distributed regardless of the distribution you start with.
Practical Implications — Because sample means are normal, we can use them for confidence intervals, T-tests, and ANOVA without worrying about the original data distribution.
Rule of Thumb N>=30 — The N>=30 sample size rule is a safe rule of thumb, but it is not a strict requirement for the CLT to apply.
The Fine Print — The only requirement for the CLT is that the distribution must have a calculable mean, excluding rare cases like the Cauchy distribution.

PDF notes

Bottom line: You will be able to explain the Central Limit Theorem (CLT) and why sample means are normally distributed, regardless of the original data distribution. This understanding allows you to justify the use of standard statistical tests, such as T-tests and ANOVA, in data analysis.

This lesson on Sampling Distributions and the CLT is hands-on and example-driven. You will be able to explain the Central Limit Theorem (CLT) and why sample means are normally distributed, regardless of the original data distribution. This understanding allows you to justify the use of standard statistical tests, such as T-tests and ANOVA, in data analysis.

What You'll Be Able To Do

Identify the core principle of the Central Limit Theorem.
Explain why the distribution of sample means tends toward normality.
Illustrate the effect of repeated sampling on uniform and exponential distributions.
List the statistical tests whose validity relies on the Central Limit Theorem.
Define the concept of a sampling distribution of the mean.

Topics Covered in Sampling Distributions and the CLT

CLT Introduction (0:00 - 0:45) — The Central Limit Theorem (CLT) is the basis for much of statistics and is a simple concept.
Uniform Distribution Example (0:45 - 2:30) — Sampling means repeatedly from a uniform distribution shows that the distribution of those means becomes normal.
Exponential Distribution Example (2:30 - 4:30) — Sampling means repeatedly from an exponential distribution also results in a normally distributed set of means.
Generalizing the Principle (4:30 - 5:00) — The CLT states that the means will be normally distributed regardless of the distribution you start with.
Practical Implications (5:00 - 6:00) — Because sample means are normal, we can use them for confidence intervals, T-tests, and ANOVA without worrying about the original data distribution.
Rule of Thumb N>=30 (6:00 - 6:45) — The N>=30 sample size rule is a safe rule of thumb, but it is not a strict requirement for the CLT to apply.
The Fine Print (6:45 - 7:30) — The only requirement for the CLT is that the distribution must have a calculable mean, excluding rare cases like the Cauchy distribution.

SQL Cheat Sheet

Central Limit Theorem — Sample means become normally distributed regardless of parent distribution shape
```
SELECT AVG(value) FROM samples GROUP BY batch;
```
Sampling Distribution — Distribution formed by repeatedly calculating a statistic from multiple samples
```
SELECT AVG(value) FROM samples GROUP BY batch;
```
Normal Distribution — Symmetric, bell-shaped curve; basis for many parametric statistical tests
```
SELECT COUNT(*) FROM data WHERE value BETWEEN 0 AND 100;
```
T-test — Statistical test to determine if two sample means are significantly different
```
SELECT AVG(value) FROM experiment WHERE treatment = 'A';
```
ANOVA — Statistical test to compare means from three or more different samples
```
SELECT AVG(value) FROM experiment GROUP BY treatment_group;
```

Comparison Table

Starting Distribution	Item Distributed	Resulting Distribution Shape
Uniform	Individual Samples	Uniform
Uniform	Sample Means	Normal
Exponential	Sample Means	Normal
Any Distribution	Sample Means	Normal

Common Pitfalls

Mistake: Believing the original data must be normally distributed for the CLT to work. Avoid: The CLT applies regardless of the starting distribution (uniform, exponential, etc.).
Mistake: Confusing the shape of the raw data distribution with the mean's distribution. Avoid: The raw data keeps its shape; only the distribution of the means becomes normal.
Mistake: Strictly adhering to the N>=30 sample size rule for the CLT to apply. Avoid: Use N>=30 as a safe rule of thumb, but smaller samples can work well too.

FAQs

Why is the Central Limit Theorem so important? It allows us to assume the sampling distribution of the mean is normal. This justifies using parametric statistical methods even when the original data distribution is unknown.
What statistical tests rely on the CLT? Common tests include T-tests (comparing two means) and ANOVA (comparing three or more means). These tests assume the means are normally distributed.
Does the CLT work for every single distribution? No, there is fine print. The CLT requires that you can calculate a mean from the sample. Distributions like the Cauchy distribution do not have a defined mean.
Is a sample size of 30 always required? N>=30 is a common rule of thumb considered safe. However, the examples show that the CLT can take effect with smaller sample sizes, such as N=20.

Common pitfalls

Mistake: Believing the original data must be normally distributed for the CLT to work. Avoid: The CLT applies regardless of the starting distribution (uniform, exponential, etc.).
Mistake: Confusing the shape of the raw data distribution with the mean's distribution. Avoid: The raw data keeps its shape; only the distribution of the means becomes normal.
Mistake: Strictly adhering to the N>=30 sample size rule for the CLT to apply. Avoid: Use N>=30 as a safe rule of thumb, but smaller samples can work well too.

Even if you're not normal the average is normal. Hello, I'm Josh Starmer and welcome to StatQuest. Today we're going to talk about the Central Limit Theorem, and it's going to be clearly explained. Note: For this StatQuest to make any sense at all, you should be familiar with the normal distribution. If not, check out The Normal Distribution Clearly Explained. It would also be helpful if you were familiar with the concept of sampling from a statistical distribution. If not, check out Sampling from a Statistical Distribution Clearly Explained. The Central Limit Theorem is the basis for a lot of statistics, and the good news is that it's a pretty simple concept. In this StatQuest, I'll explain what the Central Limit Theorem is and why it's important. Like most things in statistics, I think the Central Limit Theorem is easiest to understand if we look at some examples. So let's start with a uniform distribution. This one goes from 0 to 1. It's called the uniform distribution because there is an equal probability of selecting values between 0 and 1. The probabilities are all equal and thus are uniform. We can collect 20 random samples from this uniform distribution. And then calculate the mean of the samples. And on the right, we can draw a histogram of the mean value. Since we only have one mean value, the histogram isn't very interesting. But after we collect 10 more samples and collect 10 more means, the histogram starts to look a little more interesting. Here's the histogram after collecting 20 samples and calculating 20 means. 30 means, 40 means, 50 means, 60 means, 70 means, 80 means, 90 means, and 100 means. After adding 100 means to the histogram, it's pretty easy to see that these means are normally distributed. However, to make it easy to see that the means are normally distributed, we can overlay a normal distribution. You might have noticed that in the last two slides, I put means are normally distributed in bold. I did this because this is what the Central Limit Theorem is all about. Even though these means were calculated using data from a uniform distribution, the means themselves are not uniformly distributed. Instead, the means are normally distributed. Bam! Here's another example. This time we'll start with an exponential distribution. Just like before, we can collect 20 random samples from this exponential distribution. And just like before, we can calculate the mean of the 20 samples. And lastly, we can draw a histogram of that mean over here on the right. After we collect 10 samples and calculate 10 means, the histogram starts to look a little more interesting. Here's the histogram after 20 means, 30 means, 40 means, 50 means, 60 means, 70 means, 80 means, 90 means, and 100 means. After adding 100 means to the histogram, we can see that they are normally distributed. Even though these means were calculated using data from an exponential distribution, the means themselves are not exponentially distributed. Instead, the means are normally distributed. Bam! So far, we have seen that the means calculated from samples taken from a uniform distribution are normally distributed. And means calculated from samples taken from an exponential distribution are also normally distributed. Well, it turns out that it doesn't matter what distribution you start with. If you collect samples from those distributions, the means will be normally distributed. Yes, there's a little asterisk here that means there's some fine print that will come later. For now, just know it's really fine print and not worth spending too much time worrying about. Double bam! Cool, but what are the practical implications of knowing that the means are normally distributed? When we do an experiment, we don't always know what distribution our data comes from. To this, the Central Limit Theorem says, "Who cares?" The sample means will be normally distributed. Because we know that the sample means are normally distributed, we don't need to worry too much about the distribution that the samples came from. We can use the mean's normal distribution to make confidence intervals, do T-tests, where we ask if there's a difference between the means from two samples, and ANOVA, where we ask if there's a difference among the means from three or more samples, and pretty much any statistical test that uses the sample mean. Triple bam! Note: Out there in the wild, some folks say that in order for the Central Limit Theorem to be true, the sample size must be at least 30. This is just a rule of thumb and generally considered safe. However, as you can see in the examples here where I use a sample size of 20, the rule was meant to be broken. Here's the fine print. In order for the Central Limit Theorem to work at all, you have to be able to calculate a mean from your sample. Off the top of my head, I can think of only one distribution, the Cauchy distribution, that doesn't have a sample mean. And after doing biostatistics for 20 years, I've never come across it in practice. That said, if you know of distributions that don't have means, put them in the comments below and tell us what they're used for. I'm curious about how common this occurs. Hooray! We've made it to the end of another exciting StatQuest. If you like this StatQuest and want to see more of them, please subscribe. And if you want to support StatQuest, well, consider buying one or two of my original songs. All right. Until next time, quest on!

Frequently asked questions

Why is the Central Limit Theorem so important?

It allows us to assume the sampling distribution of the mean is normal. This justifies using parametric statistical methods even when the original data distribution is unknown.

What statistical tests rely on the CLT?

Common tests include T-tests (comparing two means) and ANOVA (comparing three or more means). These tests assume the means are normally distributed.

Does the CLT work for every single distribution?

No, there is fine print. The CLT requires that you can calculate a mean from the sample. Distributions like the Cauchy distribution do not have a defined mean.

Is a sample size of 30 always required?

N>=30 is a common rule of thumb considered safe. However, the examples show that the CLT can take effect with smaller sample sizes, such as N=20.