Building a Load Balancer with Four Routing Algorithms

Round-robin, weighted, least-connections, and random — all backed by Redis with health-aware routing

By SysAdmin · Published 2026-05-27

Building a Load Balancer with Four Routing Algorithms

Every cloud platform — AWS ALB, Nginx, HAProxy — implements these exact four algorithms. Building your own reveals the tradeoffs each one makes.

1. The Problem

Design a load balancer that distributes incoming requests across a pool of backend servers. Requirements:

• Four algorithms: round-robin, weighted, least-connections, random
• Health-aware routing: unhealthy servers are skipped
• Connection tracking: record/release active connections per server
• Statistics: per-server metrics (active connections, total requests, weight, health)
• Persistent state: everything in Redis, shared across instances

2. The Naïve Approach (and Why It Fails)

List<String> servers = List.of("server-1", "server-2", "server-3");
int index = 0;

public String getNextServer() {
    return servers.get(index++ % servers.size());
}

This breaks because:

1. No health awareness — routes to dead servers, users see 502 errors
2. Not distributed — each app instance has its own index, so the round-robin is per-instance, not global
3. No weight support — all servers treated equally, even if some have 4x the CPU

3. The Right Model

Redis structures:

4. The Implementation, Walked Through

The Interface

public interface LoadBalancerContract {
    void addServer(String serverId, int weight);
    void removeServer(String serverId);
    String getNextServer(String algorithm);
    void recordConnection(String serverId);
    void releaseConnection(String serverId);
    Map<String, Object> getServerStats(String serverId);
    List<String> getActiveServers();
    void markUnhealthy(String serverId);
    void markHealthy(String serverId);
}

Round-Robin

The simplest: cycle through healthy servers in order.

private String roundRobin(List<String> healthyServers) {
    long idx = redis.incr("lb:rr:index");
    return healthyServers.get((int) ((idx - 1) % healthyServers.size()));
}

💡 Tip: The global counter in Redis means all app instances share the same position. Thread A on Instance 1 and Thread B on Instance 2 won't route to the same server.

Weighted Random

Servers with higher weight get proportionally more traffic:

private String weightedRandom(List<String> healthyServers) {
    int totalWeight = healthyServers.stream()
        .mapToInt(s -> getWeight(s)).sum();
    int rand = RANDOM.nextInt(totalWeight);
    int cumulative = 0;
    for (String server : healthyServers) {
        cumulative += getWeight(server);
        if (rand < cumulative) return server;
    }
    return healthyServers.get(healthyServers.size() - 1);
}

With weights {A:5, B:3, C:2}, A gets 50% of traffic, B gets 30%, C gets 20%.

Least Connections

Route to the server with the fewest active connections:

private String leastConnections(List<String> healthyServers) {
    return healthyServers.stream()
        .min(Comparator.comparingInt(s -> getActiveConnections(s)))
        .orElse(null);
}

This naturally handles uneven processing times — slow requests accumulate connections, so the server gets fewer new ones.

⚠️ Trap: You must pair recordConnection with releaseConnection. If you forget to release, the connection count grows forever and the server becomes permanently deprioritized.

Health-Aware Filtering

All four algorithms share the same pre-filter:

private List<String> getHealthyServersSorted() {
    return getActiveServers().stream()
        .filter(s -> "true".equals(redis.hget(SERVER_PREFIX + s, "healthy")))
        .sorted()
        .collect(Collectors.toList());
}

Unhealthy servers are invisible to routing — they exist in the pool but never receive traffic.

5. Performance + Concurrency

Each getNextServer call does 1-3 Redis reads (health check + algorithm-specific). For round-robin, INCR is atomic — no locking needed. For weighted and least-connections, the selection is deterministic given the same inputs.

The grader expects 5,000+ ops/sec with p99 < 10ms — trivially achievable with Redis hash reads.

6. What the Grader Checks

7. Takeaways

1. Round-robin is not always fair. If requests take different times, round-robin can overload slow servers. Least-connections adapts automatically.

1. Health checks are the most important feature. A load balancer without health-aware routing is worse than no load balancer — it confidently routes traffic to dead servers.

1. Weighted routing requires understanding your fleet. Weight should reflect actual server capacity (CPU, memory, network). Equal weights are a reasonable default, but 2x the weight should mean 2x the capacity.

👉 Try it yourself: Load Balancer on Cruscible