Micro Free Course — 100% Free Learning
All lessons in this module are free to learn. Sign in with Google to save your progress.
Evaluation, Leakage, and Unsupervised
Picking the right metric (AUC/F1/precision-recall/Brier), spotting and preventing data leakage, hyperparameter tuning strategies, and core unsupervised techniques (clustering, PCA, UMAP).
Module Content
Evaluation Metrics — AUC, F1, Precision/Recall, Brier
No metric is universally right. Pick by what the model will be used for. FIND_VIDEO: search 'ROC AUC F1 precision recall explained' — recommended channel: StatQuest. Aim for 11 min or under.
Quiz: Choosing Metrics for the Decision
Practice quiz: Six metrics. Each answers a different question. Match the metric to the business question.
Data Leakage — The #1 ML Bug
The reason your model is too good to be true is almost always leakage. Spot it before deployment. FIND_VIDEO: search 'data leakage machine learning examples' — recommended channel: Kaggle / StatQuest. Aim for 11 min or under.
Quiz: Spotting and Preventing Leakage
Practice quiz: Leakage is information from the future or from the target sneaking into training. The most common ML bug, the hardest to detect, the easiest to make.
Hyperparameter Tuning — Grid, Random, Bayesian
Three strategies: brute force, random, smart. When each is right and where Optuna fits. FIND_VIDEO: search 'hyperparameter tuning bayesian optimization' — recommended channel: StatQuest / sklearn docs. Aim for 10 min or under.
Quiz: Tuning Strategy That Works
Practice quiz: Grid search dies in high dimensions. Random search and Bayesian optimization are the modern alternatives.
Clustering and Dimensionality Reduction
Unsupervised: find structure without labels. The four most-used techniques. FIND_VIDEO: search 'k-means hierarchical PCA UMAP' — recommended channel: StatQuest. Aim for 11 min or under.
Quiz: k-Means, Hierarchical, PCA, UMAP
Practice quiz: The unsupervised toolkit. Pick by what kind of structure you're looking for.
SUBMISSION: Project 4 — Data Leakage Forensic Audit on a Temporal Credit Default Dataset
A 0.97-AUC credit model is too good to be true. Find the planted leaks in a temporal default dataset, rebuild honestly with group- and time-aware splits, and report the number finance can trust.