ha-load-balancing

name	ha-load-balancing
description	Configure high-availability load balancing with HAProxy and NGINX reverse proxies. Covers health checks, automatic failover, sticky sessions, and stats monitoring for containerized backends.
license	MIT
metadata	{"author":"Stakpak <[e-mail]>","version":"1.0.19"}

High-Availability Load Balancing with HAProxy and NGINX

Quick Start

Deploy Stack

docker compose up -d
curl http://localhost:8404/\;csv

Prerequisites

• Docker and Docker Compose installed
• Basic understanding of reverse proxies and load balancing concepts
• Backend applications with health check endpoints (returning HTTP 200 for healthy)

Setup

1. Design the Architecture

Plan a multi-tier load balancing architecture:

Client → HAProxy (L7 LB) → NGINX (Reverse Proxy) → Backend App

Reasoning: Separating HAProxy and NGINX provides flexibility - HAProxy handles load balancing decisions while NGINX can handle SSL termination, caching, and request manipulation per-backend.

2. Configure Backend Health Endpoints

Ensure each backend exposes a /health endpoint that:

• Returns HTTP 200 when healthy
• Returns HTTP 503 when unhealthy or degraded
• Responds quickly (< 1 second)

Example FastAPI health endpoint:

@app.get("/health")
async def health():
    # Add actual health checks here (DB, cache, dependencies)
    return {"status": "healthy", "instance": INSTANCE_ID}

3. Configure NGINX Reverse Proxies

Create one NGINX config per backend with appropriate timeouts:

events {
    worker_connections 1024;
}

http {
    upstream backend {
        server backend_container:8000;
    }

    server {
        listen 80;
        
        location / {
            proxy_pass http://backend;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_connect_timeout 5s;
            proxy_read_timeout 10s;
        }

        location /health {
            proxy_pass http://backend/health;
            proxy_connect_timeout 2s;
            proxy_read_timeout 3s;
        }
    }
}

Key settings:

• proxy_connect_timeout: How long to wait for backend connection
• proxy_read_timeout: How long to wait for backend response
• Health endpoint has shorter timeouts for faster failure detection

4. Configure HAProxy Load Balancer

Create HAProxy configuration with health checks and sticky sessions:

global
    log stdout format raw local0
    maxconn 4096

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5s
    timeout client  30s
    timeout server  30s
    retries 3
    option redispatch

# Stats dashboard
frontend stats
    bind *:8404
    mode http
    stats enable
    stats uri /
    stats refresh 5s
    stats show-legends
    stats show-node

# Main frontend
frontend http_front
    bind *:80
    default_backend nginx_backends

# Backend pool with health checks and sticky sessions
backend nginx_backends
    balance roundrobin
    option httpchk GET /health
    http-check expect status 200
    
    # Sticky sessions using cookie
    cookie SERVERID insert indirect nocache
    
    # Health check tuning: interval, failures to mark down, successes to mark up
    default-server inter 3s fall 2 rise 3
    
    server nginx1 nginx1:80 check cookie nginx1
    server nginx2 nginx2:80 check cookie nginx2
    server nginx3 nginx3:80 check cookie nginx3

Critical parameters:

• inter 3s: Health check interval (every 3 seconds)
• fall 2: Mark server DOWN after 2 consecutive failures
• rise 3: Mark server UP after 3 consecutive successes
• option redispatch: Retry failed requests on another server
• cookie SERVERID insert: Enable sticky sessions

5. Create Docker Compose Stack

Orchestrate all components:

services:
  # Backend applications
  backend1:
    build: ./backends
    environment:
      - INSTANCE_ID=backend-1
    networks:
      - ha-network

  backend2:
    build: ./backends
    environment:
      - INSTANCE_ID=backend-2
    networks:
      - ha-network

  # NGINX reverse proxies (one per backend)
  nginx1:
    image: nginx:alpine
    volumes:
      - ./nginx/nginx-backend1.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "8001:80"
    depends_on:
      - backend1
    networks:
      - ha-network

  nginx2:
    image: nginx:alpine
    volumes:
      - ./nginx/nginx-backend2.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "8002:80"
    depends_on:
      - backend2
    networks:
      - ha-network

  # HAProxy load balancer
  haproxy:
    image: haproxy:2.9-alpine
    volumes:
      - ./haproxy/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
    ports:
      - "80:80"      # Main traffic
      - "8404:8404"  # Stats dashboard
    depends_on:
      - nginx1
      - nginx2
    networks:
      - ha-network

networks:
  ha-network:
    driver: bridge

6. Validate the Setup

Run verification tests to confirm:

1. Backend health: Each NGINX proxy returns healthy status
2. Load distribution: Requests are distributed across healthy backends
3. Sticky sessions: Same cookie routes to same backend
4. Failover: Unhealthy backends receive no traffic
5. Stats dashboard: HAProxy stats accessible at :8404

Test commands:

# Test individual backends
curl http://localhost:8001/health
curl http://localhost:8002/health

# Test load balancer distribution
for i in {1..10}; do curl -s http://localhost/ | jq .instance; done

# Test sticky sessions
curl -c cookies.txt http://localhost/
for i in {1..5}; do curl -b cookies.txt -s http://localhost/ | jq .instance; done

# Check HAProxy stats (CSV format)
curl http://localhost:8404/\;csv

7. Monitor and Tune

Access HAProxy stats dashboard at http://localhost:8404 to monitor:

• Backend status (UP/DOWN)
• Request distribution
• Response times
• Failed health checks
• Session counts

Tuning recommendations:

• Adjust inter based on acceptable detection latency
• Increase fall for flaky backends to avoid flapping
• Use slowstart for gradual traffic ramp-up after recovery

Health Check Tuning Guide

Scenario	inter	fall	rise	Notes
Fast failover	1s	2	2	Aggressive, may cause flapping
Balanced	3s	2	3	Good default for most cases
Stable/Flaky backends	5s	3	3	Tolerates occasional failures
Critical services	2s	2	2	Fast detection, fast recovery

Sticky Session Options

Method	Use Case	Configuration
Cookie-based	Stateful apps, user sessions	cookie SERVERID insert indirect nocache
Source IP	Simple persistence	balance source
URL parameter	API routing	balance url_param sessionid

Troubleshooting

Issue	Solution
Backend marked DOWN but appears healthy	Check health endpoint returns exactly HTTP 200, verify network connectivity between HAProxy and backend, check HAProxy logs: docker logs <haproxy_container>
Sticky sessions not working	Verify cookie is being set: curl -c - http://localhost/, check browser isn't blocking cookies, ensure cookie directive matches server names
Uneven load distribution	Check if sticky sessions are causing imbalance, verify all backends have same weight, review HAProxy stats for connection counts

References

• HAProxy Configuration Manual
• HAProxy Health Checks
• NGINX Reverse Proxy Guide
• Docker Compose Networking