Back to Foundations: From Intuition to Mathematics

Standard Errors and Estimators

Where the precision of any statistic comes from. The vocabulary every DS interview probes. FIND_VIDEO: search 'standard error estimator unbiased' — recommended channel: StatQuest. Aim for 10 min or under.

12 minutesVideo LessonPDF notes

🎯 Free Guest Mode: You are learning for free. Sign in to save your completion progress and quiz answers.

Ready to continue?

Mark this lesson as complete when you're ready to proceed.

Key moments

Error Bar Types — Error bars commonly represent standard deviation, standard error, or confidence intervals.
Standard Deviation Role — Standard deviation shows how individual data points are distributed around the mean of the collected data.
Standard Error Introduction — Standard error measures how the sample mean itself is distributed if many samples were taken.
Means Cluster Tightly — Taking multiple samples demonstrates that sample means are much less dispersed than the raw data points themselves.
SE Definition — The standard error is formally defined as the standard deviation calculated from the collection of sample means.
SE for Any Statistic — The standard error can be calculated for any statistic, such as the median or mode, by finding the standard deviation of those calculated values across samples.
SE Mean Formula — The standard error of the mean can be estimated simply by dividing the sample standard deviation by the square root of the sample size (n).
Bootstrapping Necessity — Bootstrapping is introduced as a method to estimate standard errors when a simple mathematical formula does not exist.
Bootstrapping Steps — The procedure involves repeatedly resampling the original data with replacement to generate many new bootstrap samples.
Bootstrapping Result — The final standard error is the standard deviation of the statistic calculated across all the generated bootstrapped samples.

PDF notes

Bottom line: You will be able to distinguish between standard deviation and standard error, understanding what each measures regarding data dispersion. You will learn how to calculate the standard error of a statistic, either using a simple formula or through the powerful resampling technique called bootstrapping. This knowledge is crucial for interpreting statistical results and assessing the reliability of sample estimates.

This lesson on Standard Errors and Estimators is hands-on and example-driven. You will be able to distinguish between standard deviation and standard error, understanding what each measures regarding data dispersion. You will learn how to calculate the standard error of a statistic, either using a simple formula or through the powerful resampling technique called bootstrapping. This knowledge is crucial for interpreting statistical results and assessing the reliability of sample estimates.

What You'll Be Able To Do

Define the purpose of standard deviation in describing a single dataset.
Explain why the standard error is typically smaller than the standard deviation.
Calculate the standard error of the mean using the formula based on sample size (n).
Describe the steps involved in performing a bootstrapping procedure.
Identify statistics (like median or mode) for which standard errors can be estimated.

Topics Covered in Standard Errors and Estimators

Error Bar Types (0:00 - 1:00) — Error bars commonly represent standard deviation, standard error, or confidence intervals.
Standard Deviation Role (1:00 - 2:05) — Standard deviation shows how individual data points are distributed around the mean of the collected data.
Standard Error Introduction (2:05 - 3:00) — Standard error measures how the sample mean itself is distributed if many samples were taken.
Means Cluster Tightly (3:00 - 4:30) — Taking multiple samples demonstrates that sample means are much less dispersed than the raw data points themselves.
SE Definition (4:30 - 5:30) — The standard error is formally defined as the standard deviation calculated from the collection of sample means.
SE for Any Statistic (5:30 - 6:30) — The standard error can be calculated for any statistic, such as the median or mode, by finding the standard deviation of those calculated values across samples.
SE Mean Formula (6:30 - 7:00) — The standard error of the mean can be estimated simply by dividing the sample standard deviation by the square root of the sample size (n).
Bootstrapping Necessity (7:00 - 8:00) — Bootstrapping is introduced as a method to estimate standard errors when a simple mathematical formula does not exist.
Bootstrapping Steps (8:00 - 9:30) — The procedure involves repeatedly resampling the original data with replacement to generate many new bootstrap samples.
Bootstrapping Result (9:30 - 10:15) — The final standard error is the standard deviation of the statistic calculated across all the generated bootstrapped samples.

SQL Cheat Sheet

Standard Deviation — How data points are distributed around the mean of one sample
```
SELECT STDDEV(weight) FROM mouse_sample;
```
Standard Error (SE) — How the sample mean is distributed across multiple hypothetical samples
```
SELECT STDDEV(sample_mean) FROM collection_of_means;
```
Bootstrapping — Resampling existing data with replacement to estimate standard error
```
SELECT AVG(bootstrapped_sample) FROM resampled_data;
```
SE of the Mean Formula — Estimate SE using sample SD divided by the square root of n
Estimator — Any statistic (mean, median, mode) calculated from a sample
```
SELECT MEDIAN(score) FROM student_data;
```
Sample Size (n) — The number of measurements taken in a single experiment
```
SELECT COUNT(*) FROM sample_a;
```

Comparison Table

Statistic	Measures Dispersion of	Relative Size
Standard Deviation (SD)	Raw data points in one sample	Larger
Standard Error (SE)	Sample means across multiple samples	Smaller
Confidence Interval (CI)	Range likely containing true population mean	Related to SE

Common Pitfalls

Mistake: Confusing Standard Deviation (SD) with Standard Error (SE). Avoid: Remember SD describes data spread; SE describes mean variability.
Mistake: Assuming bootstrapping requires collecting new data points. Avoid: Bootstrapping resamples existing data with replacement to create new samples.
Mistake: Using the SE mean formula for the SE of the median. Avoid: Use bootstrapping unless a specific formula is known for that statistic.

FAQs

Why is the standard error usually smaller than the standard deviation? Means are less dispersed than raw data points because extreme values in a sample tend to be balanced out by values closer to the center. This central tendency reduces the variability of the means.
What is a 'dynamite plot'? A plot showing only the mean and error bars (SD or SE) without displaying the original raw data points. This visualization is often discouraged as it hides data distribution.
Can I calculate the standard error for statistics other than the mean? Yes, you can calculate the standard error for any statistic (median, mode, percentile) by finding the standard deviation of that statistic across multiple samples.
When should I use the bootstrapping method? Use bootstrapping when there is no simple mathematical formula available to estimate the standard error of a specific statistic, such as the median or a complex percentile.

Common pitfalls

Mistake: Confusing Standard Deviation (SD) with Standard Error (SE). Avoid: Remember SD describes data spread; SE describes mean variability.
Mistake: Assuming bootstrapping requires collecting new data points. Avoid: Bootstrapping resamples existing data with replacement to create new samples.
Mistake: Using the SE mean formula for the SE of the median. Avoid: Use bootstrapping unless a specific formula is known for that statistic.

StatQuest. StatQuest. StatQuest. Hello and welcome to StatQuest. This time we're going to talk about standard errors and we're also going to have a bootstrapping bonus. We'll start by talking about error bars, which are very closely related to standard errors. For example, you might collect measurements from three samples labeled A, B, and C and plot them on a scatter plot just like we see here. You could then calculate the means for the three data sets. And we've illustrated those here with three green horizontal bars approximately halfway up the clusters of data points. After that, we could calculate the standard deviations and add those to the graph. And we've shown those here with red error bars. In manuscripts and presentations, people often don't display the original data, but instead just show the mean and the standard deviation in what's called a dynamite plot because each column in the plot looks like it's the igniter for a stick of dynamite. There are three common types of error bars. The first type are standard deviations, which we just saw and I'm sure you're all familiar with. These tell you how the data are distributed around the mean. Big standard deviations tell you that some of the data points were pretty far from the mean. In most cases, you want to use standard deviations in your graphs since it tells us about your data, the data points that you collected yourself. The second type of error bar comes from standard errors. These tell you how the mean is distributed, not just the data, but the means, which sounds crazy, but it'll become clear once I draw some pictures. The third common type of error bar are confidence intervals, and these are related to standard errors. Confidence intervals will be explained more in a future StatQuest. Since this StatQuest is all about standard errors, that's what we're going to talk about. Let's start by considering a normal distribution. In this case, we can imagine that we weighed a lot of mice and plotted the distribution of differences from the mean. The y-axis is the proportion of the mice that we weighed, and the x-axis is the difference from the mean. Most of the mice had weights close to the average. A few of the mice weighed much less than the average mouse. And a few other mice weighed much more than the average mouse. Usually, you can't afford to measure the weight of all the mice, so you just take a sample. In this example, we'll just assume we took five measurements from the population, rather than measuring all the mice. Since most of the mice have weights close to the average, most of our samples are going to be close to zero. Now, just like we always do, we can calculate the mean and standard deviation from our sample. In this case, the mean of our sample is -0.2, and the standard deviation is 1.923. And we can plot the mean and standard deviation on our graph as the mean plus or minus the standard deviation around the mean. And for all you StatQuesters out there, here's a rule of thumb. Remember that one standard deviation on each side of the mean is supposed to cover about 68% of the data. Two standard deviations on each side of the mean is supposed to cover about 95% of the data. This will come in handy later. The mean is now a lighter color because we're going to take additional samples and overlay additional means and standard deviations on this same graph. Here, we've taken another five measurements. And from those five measurements, we've calculated the mean and the standard deviation. And here we've plotted that mean plus or minus one standard deviation on each side. And now we take another five measurements. This is the first sample where one of the measurements is relatively extreme. However, that one measurement doesn't sway the mean that far from zero. That is to say, the means are relatively close to each other compared to the raw data. This is because for a mean to be far from the middle, most, if not all of the raw data points would have to be in a single cluster that is far away from the middle. For example, the sample of purple points all form a cluster that are far from the middle. This could happen, but very rarely. What's much more likely is to have a sample where most of the points are close to zero, and only one or two are far away. So far, we've shown that you can calculate the standard deviations for each sample. But now that we have three means, we can also calculate the standard deviation of those means. Because one standard deviation will cover 68% of the values and two will cover 95% of the values, the standard deviation of the means won't be as wide as the standard deviations of the data. Here, we've plotted the mean of the means plus or minus one standard deviation of the means. Notice that this standard deviation is much smaller than the standard deviations we got from the individual samples. The standard deviation of the mean is called the standard error of the mean, or more simply, the standard error. The standard error gives us a sense of how much variation we can expect in our means if we took a bunch of independent five measurement samples. So to review, this is how we calculate the standard error of the mean. First, you take a bunch of samples, each with the same number of measurements, or n. In this case, n equals five. The second step is to calculate the mean for each sample. Here we've calculated the mean and standard deviation for each sample. But for the standard error, all we need to do is calculate the mean. Once we've calculated the means for each sample, we can calculate the standard deviation of the means. In this case, the standard error equals 0.86. Here, we notice that the standard error is much less than the standard deviations because the means aren't as widely dispersed as the raw data. We've shown how to calculate the standard error of the mean, but there are other standard errors. For example, we can also take the standard deviation of the standard deviations. This is called the standard error of the standard deviations, which I guess is to avoid a tongue twister. It tells us how the standard deviations of multiple samples are dispersed. You can calculate the standard deviation of any statistic. For example, the median, the mode, percentiles or anything, anything that you can calculate for multiple samples. You just calculate the standard deviation, and then you have the standard error of that. So if we calculated many medians, we could calculate the standard deviation of those medians and we'd have the standard error of those medians. To summarize everything we've talked about so far, know that the standard error is just the standard deviation of multiple means taken from the same population. So if there's a population and we can take a bunch of different samples from it, all we have to do to get the standard error is to calculate the standard deviation of the means of each sample. Well, at this point, you might be wondering if we can calculate standard errors without spending a lot of time and money on doing the same experiment a bunch of times. The good news is the answer is yes. In rare cases, there's a formula you can use to estimate it. The standard error of the mean is one. The formula for that is very simple. It's just the standard deviation divided by the square root of the sample size. However, there aren't many other cases. The good news again is that we can use something called bootstrapping for everything else. Every time we don't have a simple formula, we can bootstrap it. The nice thing about bootstrapping is it's very simple conceptually, and it's easy to make a computer do this work. Here's a bootstrapping example. Just like before, we have an experiment where we took five measurements. As an aside, usually for bootstrapping, it's good to have 10 or more measurements in a single experiment. Now, we bootstrap our data with the following steps. First, we pick a random measurement from the sample that we just took. This random measurement isn't a new measurement that we haven't taken before. It's not a new number that we haven't seen. It's part of the sample that we already have. Now we just write that value down. In this case, it's 1.43. In step three, we just go back to step one and pick a new random measurement and write that value down, and we do that five times. Our second measurement is -1.38. The third measurement is -3.11. Our fourth measurement is 1.43. We've already picked that measurement before, but that's okay. When you're bootstrapping, you just pick five measurements from your sample. And it doesn't matter if you've picked the same one before. Our last measurement is -0.10. Step four in bootstrapping is to calculate the mean, median, mode, or whatever the statistic it is we're interested in understanding the standard error of, and we calculate that with our sample. In this case, we're interested in the standard error of the mean. So all we do is calculate the mean from our new bootstrap sample. The fifth step is to go all the way back to the beginning, step one, and repeat that until you have a lot of means or medians or whatever you're interested in calculating the standard error of. The sixth and final step in the bootstrapping procedure is to simply calculate the standard deviation of all the means that we generated in steps one through five. That's all there is to it. In this case, we calculated the standard error of the mean, and we've plotted it as a black line in the graph. So if there's no fancy formula to help us calculate the standard error, we can do it ourselves from scratch. We can just use bootstrapping and get the job done. And that's it. In this StatQuest, we learned that the standard error is a measure of how we might expect the means from many different samples to vary from one sample to another. We also learned that if we don't have a fancy formula for calculating the standard error, we can do it ourselves using bootstrapping. Okay, so tune in next time and we'll talk about how to use bootstrapping to calculate confidence intervals, and that's when things get really cool.

Frequently asked questions

Why is the standard error usually smaller than the standard deviation?

Means are less dispersed than raw data points because extreme values in a sample tend to be balanced out by values closer to the center. This central tendency reduces the variability of the means.

What is a 'dynamite plot'?

A plot showing only the mean and error bars (SD or SE) without displaying the original raw data points. This visualization is often discouraged as it hides data distribution.

Can I calculate the standard error for statistics other than the mean?

Yes, you can calculate the standard error for any statistic (median, mode, percentile) by finding the standard deviation of that statistic across multiple samples.

When should I use the bootstrapping method?

Use bootstrapping when there is no simple mathematical formula available to estimate the standard error of a specific statistic, such as the median or a complex percentile.