Recap — When and How to Scale

# When and How to Scale Scaling is the first lever in system design. The question is rarely "do we need to scale?" — it's "how, and what does it cost?" ## Vertical scaling (scale up) Make the machine bigger. More CPU, more RAM, more disk. ``` 1 server, 4 cores → 1 server, 32 cores ``` Pros: - Simple. No code or architecture change. - Works for any workload (single-process apps, traditional DBs). - Low operational overhead. Cons: - Hard ceiling. The biggest available server is the limit. - Expensive at the high end (price grows non-linearly). - Single point of failure (one big machine). - Often requires downtime to upgrade. When to use: - Early-stage systems where you don't know if growth justifies complexity. - Databases that don't shard easily (some RDBMS). - Apps with tight latency requirements (no network hop between processes). Vertical scaling is a fine first answer in real systems and interviews. Don't dismiss it. ## Horizontal scaling (scale out) More machines doing the same work in parallel. ``` 1 server, 4 cores → 8 servers, 4 cores each ``` Pros: - No hard ceiling (in theory). - Cheaper at scale (commodity hardware). - Fault-tolerant (one machine fails, others continue). - Often elastic (add/remove machines based on demand). Cons: - Code must support it (stateless services, no in-memory session). - Coordination overhead (load balancer, service discovery). - Distributed systems complexity (consistency, network failures). - Some workloads don't horizontally scale well (joins, transactions). When to use: - Web apps and APIs (mostly stateless). - High-traffic systems where vertical scaling is too expensive. - When fault tolerance is required (one machine down can't take the system down). ## Stateless vs stateful **Stateless services** scale horizontally easily — any instance can serve any request. Add more instances; load balancer distributes. **Stateful services** (databases, caches with session data) require careful design: - Replicate state across instances. - Shard state (each instance owns a slice). - Externalize state (move it to a stateful store; service becomes stateless). The "12-factor app" principle: keep services stateless; push state to dedicated stores. ## The scaling cube Three independent dimensions (Art of Scalability): - **X-axis**: clone (more identical copies behind load balancer). - **Y-axis**: split by function (microservices). - **Z-axis**: split by data (sharding). You can combine: 10 user-service instances (X), separate user/order services (Y), each user-service shard handles 1/N of users (Z). ## When you can avoid scaling Premature scaling is expensive. Before adding machines: 1. **Profile and optimize.** Slow code is often the bottleneck, not capacity. 2. **Add caching.** A 100x cache hit reduces load by 99%. 3. **Database indexing.** A missing index can make a query 1000x slower. 4. **Async / queueing.** Offload non-urgent work to background. 5. **CDN for static assets.** Take read load off your origin. These are cheaper than scaling and often work for a long time. ## Capacity estimation Interview classic. For a given service: - **DAU / MAU**: daily / monthly active users. - **QPS**: queries per second (peak, not average). - **Storage**: total + per user. - **Bandwidth**: in/out per second. - **Memory needs**: working set size. Example: "1M DAU, each user makes 100 requests/day, even distribution": - 1M × 100 / 86400 = ~1150 QPS average. - Peak ~3-5x average → ~5000 QPS peak. For 5000 QPS, you need: web tier (5-10 instances), DB capable of 5000 reads/s (cache helps). These envelope numbers shape your design. ## Read vs write scaling **Reads scale easily**: - Cache layer - Read replicas - CDN for static **Writes scale harder**: - Sharding (partition data) - Async writes (queue, batch) - Reduce write volume (aggregate, dedupe) Most systems are read-heavy (10:1 or 100:1). Optimize reads first. ## Common scaling mistakes 1. **Scaling before measuring.** Optimize first; scale when truly needed. 2. **Horizontal scaling a stateful service.** Breaks consistency. 3. **Ignoring the database bottleneck.** App tier scales easily; DB is usually the wall. 4. **No capacity planning.** Adding instances reactively when traffic spikes. 5. **Premature microservices.** Splitting too early adds complexity without benefit. ## Takeaway Vertical = bigger box (simple, capped). Horizontal = more boxes (complex, unbounded). Stateless services scale horizontally easily; stateful needs careful design (replicate, shard, externalize). Read scaling is easier than write scaling — most systems are read-heavy. Profile and cache before adding machines.