Micro Free Course — 100% Free Learning
All lessons in this module are free to learn. Sign in with Google to save your progress.
Serving and Deployment
Batch vs online inference, building a FastAPI prediction service, Docker / Kubernetes packaging, scaling and latency budgets, A/B testing and champion-challenger model rollouts.
Module Content
Batch vs Online Inference
Two serving patterns. The wrong choice causes 90% of avoidable infrastructure pain. FIND_VIDEO: search 'batch vs online inference machine learning' — recommended channel: Made with ML / Chip Huyen. Aim for 10 min or under.
Recap — Choosing Your Serving Pattern
Batch is cheap and easy. Online is hard and necessary only sometimes. Pick deliberately.
Building a Prediction Service — The FastAPI Pattern
FastAPI + a Pydantic schema + a loaded model = production-grade prediction service. The standard pattern. FIND_VIDEO: search 'fastapi machine learning model serving' — recommended channel: FastAPI / Made with ML. Aim for 11 min or under.
Recap — Anatomy of a Production ML Service
The seven elements of a real ML service. Without each, you have a toy.
Containerization and Deployment (Docker, Kubernetes)
Docker locks the environment; Kubernetes (or simpler alternatives) runs the container. The lightweight version of cloud-native ML. FIND_VIDEO: search 'docker machine learning deployment' — recommended channel: Made with ML / Docker docs. Aim for 11 min or under.
Recap — Packaging Models for Deployment
A model in a pickle file is unshippable. A Docker container is shippable. The packaging layer that connects training to serving.
Scaling — Autoscaling, Latency Budgets, Caching
How to keep predictions fast and the bill manageable as traffic grows. FIND_VIDEO: search 'ML service latency caching autoscaling' — recommended channel: Made with ML / Chip Huyen. Aim for 10 min or under.
Recap — Latency Budgets and Optimization
Latency is a contract. Setting one explicitly and optimizing for it produces a system you can actually run.
A/B Testing and Champion-Challenger
The discipline of deploying new models. Never swap; always test. FIND_VIDEO: search 'ml model ab test champion challenger' — recommended channel: Statsig / Made with ML. Aim for 10 min or under.
Recap — Safely Rolling Out New Models
The three-step rollout: shadow → canary → full. Each step catches a different class of bug before it hits all users.