Load Balancing — Beginner Friendly System Design Guide

Why Load Balancing Exists

Imagine Swiggy has only:

1 server

Now during dinner time:

• lakhs of users open app
• place orders
• refresh restaurant pages

All traffic hits one server.

Result:

Server overload ❌

App becomes:

• slow
• unstable
• unavailable

This problem is solved using:

Load Balancing

---

What Is Load Balancing?

Load balancing means:

Distributing traffic across multiple servers to prevent overload and improve reliability.

Instead of:

• one server handling everything

traffic is shared among:

• many servers

---

Real-Life Analogy

Restaurant Example

Suppose one cashier handles:

• 500 customers

Huge queue forms.

Restaurant adds:

• multiple cash counters

Customers distribute automatically.

This is load balancing.

---

Real Example — IPL Streaming On Hotstar

During IPL:

• crores of users watch simultaneously

One server cannot stream to everyone.

Hotstar distributes users across:

• hundreds/thousands of servers

using load balancers.

---

Basic Load Balancing Architecture

Users
  │
  ▼
Load Balancer
 ├── Server 1
 ├── Server 2
 └── Server 3

---

Without Load Balancing

All Users
    │
    ▼
Single Server
    │
    ▼
Server Overload ❌

Problems:

• crashes
• slow APIs
• downtime

---

With Load Balancing

Users
  │
  ▼
Load Balancer
  │
 ┌─┼──────────┐
 ▼ ▼          ▼
S1 S2         S3

Traffic distributes evenly.

---

What Does A Load Balancer Actually Do?

Load balancer:

• receives incoming traffic
• decides which server should handle request
• forwards request safely

---

Request Flow

User Request
      │
      ▼
Load Balancer
      │
 ┌────┼────┐
 ▼    ▼    ▼
App1 App2 App3

---

Real Example — Swiggy

Suppose:

• 10 lakh users online

Load balancer distributes traffic:

• some users → Server 1
• some users → Server 2
• some users → Server 3

This prevents:

• overload
• downtime

---

Why Load Balancers Are Critical

Without them:

• scaling becomes difficult
• single server becomes bottleneck

Load balancers improve:

• reliability
• scalability
• availability

---

Main Benefits Of Load Balancing

Benefit	Why Important
Prevent overload	No single server overwhelmed
Better scalability	Add more servers easily
High availability	Traffic rerouted if server fails
Better performance	Faster response times
Fault tolerance	System survives failures

---

Example — One Server Failure

Suppose Server 2 crashes.

Without load balancer:

Users
  │
  ▼
Server 2 ❌

Users affected immediately.

---

With Load Balancer

Users
  │
  ▼
Load Balancer
  │
 ┌─┼─────────┐
 ▼ ▼         ▼
S1 S2❌      S3

Traffic automatically moves to:

• S1
• S3

Application continues working.

---

This Is Called High Availability

System remains operational even during failures.

Very important for:

• banking
• ecommerce
• streaming
• payments

---

Load Balancing Algorithms

Load balancers decide:

• where traffic should go

using algorithms.

---

1. Round Robin

Requests distributed one-by-one.

---

Example

Request 1 → Server 1
Request 2 → Server 2
Request 3 → Server 3
Request 4 → Server 1

Simple and common.

---

2. Least Connections

Traffic goes to server with:

• least active users

---

Example

Server 1 → 500 users
Server 2 → 120 users

New request → Server 2

---

3. Geographic Routing

Users routed based on location.

---

Example

Indian users:

• Mumbai servers

US users:

• US servers

---

Geo Routing Flow

User Location
      │
      ▼
Load Balancer
 ┌────┼────┐
 ▼         ▼
India     USA
Servers   Servers

---

Load Balancers And Health Checks

Load balancer continuously checks:

"Is server healthy?"

---

Health Check Flow

Load Balancer
      │
      ▼
Ping Server
  │          │
Healthy    Failed
  │           │
  ▼           ▼
Use Server Remove Server

---

Real Example — PhonePe

Suppose payment server crashes during UPI traffic.

Load balancer detects failure and:

• redirects requests
• prevents downtime

Critical for payment reliability.

---

Types Of Load Balancers

Two major categories:

---

Hardware Load Balancer

Physical machine handling traffic.

Expensive.

Used in:

• traditional data centers

---

Software Load Balancer

Software-based solutions.

Examples:

• Nginx
• HAProxy
• Envoy

Very common in cloud systems.

---

Popular Load Balancer Technologies

Technology	Usage
Nginx	Web traffic
HAProxy	High performance
AWS ELB	Cloud balancing
Envoy	Microservices
Traefik	Containers/Kubernetes

---

Load Balancing In Microservices

Modern systems use:

• many services

Load balancing becomes even more important.

---

Example Architecture

Users
  │
  ▼
API Gateway
  │
 ┌─┼────────────┐
 ▼ ▼            ▼
Order Payment Inventory
Service Service Service

Each service may have:

• multiple replicas

behind load balancers.

---

Internal vs External Load Balancing

---

External Load Balancer

Handles:

• internet traffic

Example:

• users opening Swiggy

---

Internal Load Balancer

Handles:

• service-to-service traffic

Example:

• Order Service → Payment Service

---

Example Internal Flow

Order Service
      │
      ▼
Internal Load Balancer
      │
 ┌────┼────┐
 ▼    ▼    ▼
P1   P2   P3

---

Load Balancing + Autoscaling

Modern cloud systems combine:

• load balancing
• autoscaling

---

Example

Traffic spike during IPL:

Traffic Spike
      │
      ▼
Autoscaler Adds Servers
      │
      ▼
Load Balancer Distributes Traffic

---

Real Example — Blinkit During Rain

Heavy rain causes:

• sudden grocery demand spike

Blinkit may:

• add more servers automatically
• distribute traffic dynamically

---

Sticky Sessions

Sometimes user must stay connected to same server.

Called:

Sticky Sessions

---

Example

Shopping cart session stored on one server.

User requests routed consistently to same machine.

---

Sticky Session Flow

User A
  │
  ▼
Load Balancer
  │
  ▼
Always → Server 2

---

Problems With Sticky Sessions

Can create:

• uneven load
• scaling complexity

Modern systems prefer:

• Redis/shared sessions

instead.

---

Common Beginner Misconceptions

---

"Load Balancer Makes System Faster"

Not directly.

It mainly:

• distributes traffic
• prevents overload

---

"One Load Balancer Is Enough Forever"

Large systems may use:

• multiple load balancers
• global traffic managers

---

"Only Big Companies Need Load Balancing"

Even medium apps benefit from:

• reliability
• failover
• scalability

---

Real Internet-Scale Architecture

Users Worldwide
        │
        ▼
Global DNS
        │
        ▼
CDN
        │
        ▼
Global Load Balancer
        │
 ┌──────┼─────────┐
 ▼                ▼
India Region    US Region
        │
        ▼
Regional Load Balancer
        │
 ┌──────┼───────┐
 ▼      ▼       ▼
App1   App2    App3
        │
        ▼
Database Cluster

---

Final Mental Model

Load Balancer =
Traffic Manager
for servers

It ensures:

• no single server overloads
• failed servers removed
• system scales safely

---

One-Line Interview Definition

Load balancing is the process of distributing incoming traffic across multiple servers to improve scalability, reliability, fault tolerance, and system availability.