Containerization and Deployment (Docker, Kubernetes)

This lesson on Containerization and Deployment (Docker, Kubernetes) is hands-on and example-driven. You will containerize a FastAPI machine learning inference service using Docker, publish the image to Docker Hub, and deploy it across load-balanced pod replicas on a Kubernetes cluster. You will also configure declarative Deployment and Service manifests to manage compute, routing, and high availability.

What You'll Be Able To Do

• Package a serialized Python ML model and FastAPI serving app into an OCI-compliant Dockerfile.
• Build, tag, and publish container images to Docker Hub using the Docker CLI.
• Draft declarative Kubernetes Deployment manifests with replica counts, label selectors, and resource bounds.
• Configure Kubernetes NodePort Services to decouple transient pod IP addresses from stable network endpoints.
• Deploy, inspect, and validate running cluster workloads using kubectl control plane commands.

Detailed Concept Walkthrough

1. FastAPI Containerization and Image Layering

Containerization packages your ML runtime, dependencies, and model binaries into an isolated, reproducible execution environment. Docker builds these environments using layer caching to avoid re-installing heavy Python packages on every code change.

• Mechanism: A Dockerfile declares an explicit base image (such as python:3.11-slim), sets an isolated working directory /app, and copies dependencies before source code to optimize build caching.
• Under the Hood: Docker caches intermediate filesystem layers; placing COPY requirements.txt . and RUN pip install ahead of application code prevents re-downloading large ML packages whenever endpoint logic changes.
• Best Practice: Define an explicit CMD running an ASGI server like Uvicorn bound to 0.0.0.0 rather than 127.0.0.1 so the container accepts ingress network traffic from outside its local namespace.

# Dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY iris_model.pkl main.py ./
EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Key Takeaway: Order Dockerfile instructions from least-frequently changed to most-frequently changed to maximize build cache reuse for dependencies.

2. Registry Workflows: Build, Tag, and Push

Container images built locally must be uniquely tagged and uploaded to an accessible registry like Docker Hub so that remote Kubernetes nodes can pull and instantiate them.

• Mechanism: docker build packages the local context into an image, while docker tag assigns a remote namespace, repository name, and version semantic identifier.
• Execution Flow: Authenticate via docker login, tag the local image identifier with the target repository pattern <username>/<repo>:<tag>, and execute docker push to transfer layers.
• Best Practice: Always use explicit semantic version tags (e.g., 0.1v or a commit SHA) rather than the mutable latest tag to guarantee deterministic rollouts across cluster nodes.

# Build local image from current context
docker build -t fastapi-ml .

# Authenticate and tag for remote Docker Hub repository
docker login
docker tag fastapi-ml <user>/fastapi-ml:0.1v

# Publish image layers to remote registry
docker push <user>/fastapi-ml:0.1v

Key Takeaway: Kubernetes worker nodes pull images from remote registries, requiring proper namespace tagging and registry authentication.

3. Kubernetes Deployment Workloads and Pods

Deployments provide declarative management of stateless application pods, continuously reconciling the observed cluster state to match the desired replica count and image version.

• Mechanism: The Deployment controller tracks pods using label selectors defined in spec.selector.matchLabels, automatically provisioning replacements if individual pods crash or nodes fail.
• Under the Hood: Pods are the smallest deployable compute units in Kubernetes; they wrap container instances and receive ephemeral private IP addresses from the cluster software-defined network.
• Syntax Rule: The labels configured under spec.template.metadata.labels must exactly match spec.selector.matchLabels for the deployment controller to track its managed pods.

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: fastapi-ml-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: fastapi-ml-api-predictor
  template:
    metadata:
      labels:
        app: fastapi-ml-api-predictor
    spec:
      containers:
      - name: ml-container
        image: <user>/fastapi-ml:0.1v
        ports:
        - containerPort: 8000

Key Takeaway: Deployments maintain desired replica counts and pod lifecycles through declarative label-based controller reconciliation.

4. Service Abstractions and Traffic Routing

Services provide stable virtual IPs, DNS names, and layer-4 load balancing across transient pods, abstracting away individual pod IP churn.

• Mechanism: A Service matches pods via spec.selector key-value pairs, dynamically populating internal endpoints to distribute incoming requests across all healthy replica pods.
• Under the Hood: Kubernetes pod IPs are ephemeral and change whenever a pod restarts; the Service maintains a persistent virtual IP and load balances requests across backend pod targets.
• Best Practice: Use NodePort or LoadBalancer service types to expose internal port mappings (port: 80 to targetPort: 8000) for traffic originating outside the local cluster.

# service.yaml
apiVersion: v1
kind: Service
metadata:
  name: fastapi-ml-service
spec:
  type: NodePort
  selector:
    app: fastapi-ml-api-predictor
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8000

Key Takeaway: Services decouple ingress networking from ephemeral pod IP addresses by load balancing across matching selector labels.

Topics Covered in Containerization and Deployment (Docker, Kubernetes)

1. FastAPI ML Architecture (0:00 - 2:58) — Review the serialized logistic regression model, Pydantic input schemas, and the prediction endpoint.
2. Dockerfile Construction (2:58 - 4:17) — Configure image layers, install dependencies, expose port 8000, and define the Uvicorn entry point.
3. Docker Hub Publishing (4:17 - 7:14) — Build the container image locally, authenticate with Docker Hub, tag the version, and push it remotely.
4. Kubernetes Architecture (7:14 - 9:18) — Start a local Minikube cluster and analyze request routing across Deployments, Pods, and Services.
5. Deployment Configuration (9:18 - 11:13) — Draft a declarative YAML manifest specifying 3 replicas, matching labels, and remote image references.
6. Service Configuration (11:13 - 13:25) — Create a NodePort Service manifest to route external port 80 traffic to internal container port 8000.
7. Cluster Deployment & Validation (13:25 - 15:23) — Apply the manifests to the cluster and inspect pod, deployment, and service states using kubectl CLI.

ML in Practice Cheat Sheet

• docker build -t <image_name> . — Builds a Docker image from local Dockerfile context

  • In practice: docker build -t fastapi-ml .

• docker tag <source> <target> — Aliases an image to target repository and tag

  docker tag fastapi-ml username/fastapi-ml:0.1v
  

• docker push <image_path> — Publishes a local container image to remote registry

  docker push username/fastapi-ml:0.1v
  

• minikube start — Starts a local single-node development Kubernetes cluster

  minikube start
  

• kubectl apply -f <manifest> — Declaratively applies configuration manifests to the cluster

  kubectl apply -f deployment.yaml
  

• kubectl get <resource> — Displays status and metadata of active cluster resources

  kubectl get pods -l app=fastapi-ml-api-predictor
  

Comparison Table

Common Pitfalls

• Mistake: Binding Uvicorn to 127.0.0.1 inside the Dockerfile. Avoid: Bind Uvicorn to 0.0.0.0 so external container traffic can route to the application.
• Mistake: Mismatching Service selector labels and Deployment template labels. Avoid: Ensure spec.selector in the Service exactly matches spec.template.metadata.labels in the Deployment.
• Mistake: Relying on static pod IP addresses for downstream client calls. Avoid: Always route incoming traffic through a stable Kubernetes Service virtual IP.

FAQs

• Why does my Kubernetes Service report no endpoints despite pods running successfully? Check your YAML labels; the spec.selector in your Service must exactly match the key-value pairs defined in spec.template.metadata.labels of your Deployment.
• Why should requirements.txt be copied separately before copying the rest of the application code? Docker caches filesystem layers sequentially; copying dependency manifests separately prevents running slow package installations every time source code changes.
• What happens when a pod crashes in a Deployment with replicas set to 3? The Deployment controller detects the observed count is 2 and immediately schedules a replacement pod to restore the desired state.