Back to ML Foundations: The Mental Model

What ML Is (and Isn't) — A Practical Definition

ML is statistical pattern recognition. Knowing what it isn't (magic, AGI, causality) is half the value. FIND_VIDEO: search 'machine learning vs statistics vs AI' — recommended channel: StatQuest / Andrew Ng. Aim for 10 min or under.

13 minutesVideo LessonPDF notes

Watch What ML Is (and Isn't) — A Practical Definition on YouTube

🎯 Free Guest Mode: You are learning for free. Sign in to save your completion progress and quiz answers.

Ready to continue?

Mark this lesson as complete when you're ready to proceed.

Key moments

ML as Classification — The decision tree example illustrates machine learning used for classification, predicting if someone will love StatQuest.
ML as Prediction — The yam and running speed example illustrates machine learning used for prediction, estimating a continuous value.
Training Data Defined — The original data used to fit the model (like the black line or green squiggle) is called training data.
Model Evaluation Process — Testing data is used to calculate the sum of distances between real and predicted values to compare model accuracy.
Generalization is Key — The black line was chosen over the green squiggle because it performed better on the testing data, despite fitting the training data worse.
Fancy vs. Performance — The most important factor for any machine learning method is how well it performs with testing data, not how complex it is.
Classification Evaluation — The decision tree is evaluated by running testing data through it and comparing the predicted outcome to the actual outcome.
Data Splitting — The data used for training and testing must be separated, although the exact method for splitting is covered in other lessons.

PDF notes

Bottom line: You will be able to define machine learning based on its core functions: prediction and classification. You will also learn how to evaluate ML models using testing data to ensure they generalize beyond the initial training set.

This lesson on What ML Is (and Isn't) — A Practical Definition is hands-on and example-driven. You will be able to define machine learning based on its core functions: prediction and classification. You will also learn how to evaluate ML models using testing data to ensure they generalize beyond the initial training set.

What You'll Be Able To Do

Define Machine Learning based on its primary goals.
Distinguish between prediction and classification tasks.
Explain the roles of training data and testing data in model development.
Evaluate a model's performance using testing data metrics (e.g., sum of distances).
Recognize the importance of generalization over fitting training data perfectly.
Identify the concept of the bias-variance trade-off.

Topics Covered in What ML Is (and Isn't) — A Practical Definition

ML as Classification (0:45 - 1:45) — The decision tree example illustrates machine learning used for classification, predicting if someone will love StatQuest.
ML as Prediction (1:45 - 2:40) — The yam and running speed example illustrates machine learning used for prediction, estimating a continuous value.
Training Data Defined (2:40 - 3:10) — The original data used to fit the model (like the black line or green squiggle) is called training data.
Model Evaluation Process (3:10 - 5:40) — Testing data is used to calculate the sum of distances between real and predicted values to compare model accuracy.
Generalization is Key (5:40 - 6:30) — The black line was chosen over the green squiggle because it performed better on the testing data, despite fitting the training data worse.
Fancy vs. Performance (6:30 - 7:00) — The most important factor for any machine learning method is how well it performs with testing data, not how complex it is.
Classification Evaluation (7:00 - 8:30) — The decision tree is evaluated by running testing data through it and comparing the predicted outcome to the actual outcome.
Data Splitting (8:30 - 9:00) — The data used for training and testing must be separated, although the exact method for splitting is covered in other lessons.

SQL Cheat Sheet

Machine Learning — Making predictions and classifications from data
```
SELECT yam_eaten, predicted_speed FROM runner_data;
```
Training Data — Original data used to build the machine learning model
```
SELECT * FROM runner_data WHERE data_set = 'Training';
```
Testing Data — New, unseen data used to evaluate model performance
```
SELECT * FROM runner_data WHERE data_set = 'Testing';
```
Prediction — Estimating a continuous value, like speed or price
```
SELECT predict_speed(yam_eaten) FROM new_runners;
```
Classification — Assigning an item to a discrete category or class
```
SELECT classify_love(silly_songs, ml_interest) FROM audience;
```
Bias-Variance Trade-off — Fitting training data well but making poor predictions on new data

Comparison Table

Model Type	Fit to Training Data	Performance on Testing Data
Black Line (Simple)	Fits reasonably well	Better predictions (Lower error sum)
Green Squiggle (Complex)	Fits training data better	Worse predictions (Higher error sum)
Training Data	Used to build the model	Known outcomes
Testing Data	Used to evaluate the model	Unseen outcomes

Common Pitfalls

Mistake: Choosing a model based only on how well it fits the training data. Avoid: Prioritize generalization and prediction accuracy on unseen data.
Mistake: Assuming a complex or 'fancy' model is inherently better than a simple one. Avoid: Always evaluate model performance using testing data.
Mistake: Using the same data set for both training and testing the model. Avoid: Separate data into distinct training and testing sets before modeling.

FAQs

What is the core purpose of machine learning? Machine learning is fundamentally about making accurate predictions or classifications based on input data.
What is the bias-variance trade-off? It describes when a model fits the training data too well (overfitting), leading to poor generalization and bad predictions on new data.
Why is testing data necessary? Testing data measures how well the model generalizes to new, unseen examples, ensuring it isn't just memorizing the training set.
How do we choose between different ML methods? The method that performs best when evaluated against the testing data is the one you should choose, regardless of complexity.

Common pitfalls

Mistake: Choosing a model based only on how well it fits the training data. Avoid: Prioritize generalization and prediction accuracy on unseen data.
Mistake: Assuming a complex or 'fancy' model is inherently better than a simple one. Avoid: Always evaluate model performance using testing data.
Mistake: Using the same data set for both training and testing the model. Avoid: Separate data into distinct training and testing sets before modeling.

Gonna start this StatQuest with a silly song. But if you don't like silly songs, that's okay. StatQuest. Hello. I'm Josh Starmer and welcome to StatQuest. Today we're going to do a gentle introduction to machine learning. Note: This StatQuest was originally prepared for and presented at the Society for Scientific Advancement's annual conference. One of the things that SoSa does is promote science and technology in Jamaica. Let's start with a silly example. Do you like silly songs? If you like silly songs, are you interested in machine learning? If you like silly songs and machine learning, then you'll love StatQuest. If you like silly songs, but not machine learning, are you interested in statistics? If you like silly songs and statistics, but not machine learning, then you'll still love StatQuest. Otherwise, you might not like StatQuest. Wa wa. If you don't like silly songs, are you interested in machine learning? If you don't like silly songs, but you like machine learning, then you'll love StatQuest. If you don't like silly songs or machine learning, are you interested in statistics? If you don't like silly songs or machine learning, but you're interested in statistics, then you will love StatQuest. Otherwise, you might not like StatQuest. Wa wa. This is a silly example, but it illustrates a decision tree, a simple machine learning method. The purpose of this particular decision tree is to predict whether or not someone will love StatQuest. Alternatively, we could say that this decision tree classifies a person as either someone who loves StatQuest or someone who doesn't. Since decision trees are a type of machine learning, then if you understand how we used this tree to predict or classify if someone would love StatQuest, you are well on your way to understanding machine learning. Bam! Here's another silly example of machine learning. Imagine we measured how quickly someone could run 100 meters and how much yam they ate. This is me. I'm not very fast and I don't eat much yam. These are some other people. And this is Usain Bolt. Usain Bolt is very fast and he eats a lot of yam. Given this pretend data, we see that the more yam someone eats, the faster they run the 100 meter dash. We can fit a black line to the data to show the trend. But we can also use the black line to make predictions. For example, if someone told us they ate this much yam, then we could use the black line to predict how fast that person might run. This is the predicted speed. The black line is a type of machine learning because we can use it to make predictions. In general, machine learning is all about making predictions and classifications. Bam! Now that we can make predictions and classifications, let's talk about some of the main ideas in machine learning. First of all, in machine learning lingo, the original data is called training data. So, the black line is fit to training data. Alternatively, we could have fit a green squiggle to the training data. The green squiggle fits the training data better than the black line. But remember, the goal of machine learning is to make predictions. So, we need a way to decide if the green squiggle is better or worse than the black line at making predictions. So, we find a new person and measure how fast they run and how much yam they eat. And then we find another, and another, and another. Altogether, the blue dots represent testing data. We use the testing data to compare the predictions made by the black line to the predictions made by the green squiggle. Let's start by seeing how well the black line predicts the speed of each person in the testing data. Here's the first person in the testing data. They ate this much yam. And they ran this fast. However, the black line predicts that someone who ate this much yam should run a little slower. So, let's measure the distance between the actual speed and the predicted speed, and save the distance on the right while we focus on the other people in the testing data. Here's the second person in the testing data. They ate this much yam, and they ran this fast. But the black line predicts that they will run a little faster. So, we measure the distance between the actual speed and the predicted speed, and add it to the one we measured for the first person in the testing data. Then we measure the distance between the real and the predicted speed for the third person in the testing data, and add it to our running total of distances between the real and predicted speeds for the black line. Then we do the same thing for the fourth person in the testing data, and add that distance to our running total for the black line. This is the sum of all the distances between the real and predicted speeds for the black line. Now let's calculate the distances between the real and predicted speeds using the green squiggle. Remember, the green squiggle did a great job fitting the training data. But when we are doing machine learning, we are more interested in how well the green squiggle can make predictions with new data. So, just like before, we determined this person's real speed, and their predicted speed, and measure the distance between them. And just like we did for the black line, we'll keep track of the distances for the green squiggle over here. Then we do the same thing for the second person in the testing data, and the third person, and the fourth person. This is the sum of the distances between the real and predicted speeds for the green squiggle. The sum of the distances is larger for the green squiggle than the black line. In other words, even though the green squiggle fit the training data way better than the black line, the black line did a better job predicting speeds with the testing data. So, if we had to choose between using the black line or the green squiggle to make predictions, we would choose the black line. Bam! This example teaches two main ideas about machine learning. First, we use testing data to evaluate machine learning methods. Second, don't be fooled by how well a machine learning method fits the training data. Note: Fitting the training data well but making poor predictions is called the bias-variance trade-off. Oh no, a shameless self-promotion. If you want to learn more about the bias-variance trade-off, there's a StatQuest that will walk you through it one step at a time. Before we move on, you may be wondering why we used a simple black line and a silly green squiggle instead of a deep learning convolutional neural network. Or "Insert newest, bestest, most fancy machine learning method here." There are tons of fancy-sounding machine learning methods, and each year, something new and exciting comes on the scene. But regardless of what you use, the most important thing isn't how fancy it is, but how it performs with testing data. Double bam! Now let's go back to the decision tree that we started with. Remember, we wanted to classify if someone loves StatQuest based on a few questions. To create the decision tree, we collected data from people who love StatQuest, and from people who did not love StatQuest. Altogether, this was the training data. And we used it to build the decision tree. Then we got data from a few more people who love StatQuest, and a few more people who did not love StatQuest. Altogether, this forms the testing data. We can use the testing data to see how well our decision tree predicts if someone will love StatQuest. The first person in the testing data did not like silly songs. So, we go to the right side of the decision tree. They didn't like machine learning either. So, we just keep on going down the right side of the decision tree. They didn't like statistics either. So, the decision tree predicts that this person will not love StatQuest. However, this person loved StatQuest, so the decision tree made a mistake. Wa wa. The second person in the testing data liked silly songs. And that takes us down the left side of the decision tree. They were also interested in machine learning. So, we predict that that person loves StatQuest. And since this person actually loved StatQuest, the decision tree did a good job. Hooray! Now we just run all of the other people in the testing data down the decision tree and compare the predictions to reality. Then we can compare this decision tree to the latest, greatest machine learning method. Ultimately, we pick the method that does the best job predicting if someone will love StatQuest or not. Triple bam! In summary, machine learning is all about making predictions and classifications. There are tons of fancy machine learning methods, but the most important thing to know about them isn't what makes them so fancy. It's that we decide which method fits our needs the best by using testing data. One last thing before we go. You may be wondering how we decide which data go into the training set and which data go into the testing set. Earlier, we just arbitrarily decided that these red dots were the training data. But the blue dots could have, just as easily, been the training data. The good news is that there are ways to determine which samples should be used for training data and which samples should be used for testing data. And if you're interested in learning more about this, check out the StatQuest. And there are lots more StatQuests that walk you through machine learning concepts step-by-step, so check them out. Hooray! We've made it to the end of another exciting StatQuest. If you like this StatQuest and want to see more, please subscribe. And if you want to support StatQuest, well, consider buying one or two of my original songs, or getting a T-shirt or a hoodie, or some other slick merchandise. There's links on the screen, and there's links in the description below. All right, until next time, quest on!

Frequently asked questions

What is the core purpose of machine learning?

Machine learning is fundamentally about making accurate predictions or classifications based on input data.

What is the bias-variance trade-off?

It describes when a model fits the training data too well (overfitting), leading to poor generalization and bad predictions on new data.

Why is testing data necessary?

Testing data measures how well the model generalizes to new, unseen examples, ensuring it isn't just memorizing the training set.

How do we choose between different ML methods?

The method that performs best when evaluated against the testing data is the one you should choose, regardless of complexity.