Target: read three overfit-curve cases and prescribe the correct fix for each.
1. The loop, stripped down
Forward pass, loss, backward pass, optimizer step, repeat. Loss choice follows the task (cross-entropy for classification, MSE for regression); optimizer follows the budget (Adam default, SGD+momentum when you can afford tuning); learning rate dominates everything else.
2. Three curve cases
Case A (both losses high and flat): underfit — bigger model, higher LR, fewer constraints. Case B (train falls, val rises): overfit — regularization, LR cut, augmentation, early stopping. Case C (both noisy/unstable): LR too high or broken batching — cut LR 10x first, then inspect the data pipeline.
3. Regularization menu, ordered by evidence
More real data > augmentation > weight decay/dropout > smaller model. Reach for structural fixes before shrinking capacity.
Recap
Name the case (underfit / overfit / unstable), apply the ordered fix, and re-measure on validation — never tune against the test curve.