docker-compose --scale X nginx.conf configuration

Question

My nginx.conf file currently has the routes defined directly:

worker_processes auto;

events { worker_connections 1024; }

http {
    upstream wordSearcherApi {
          least_conn;

          server api1:61370 max_fails=3 fail_timeout=30s;
          server api2:61370 max_fails=3 fail_timeout=30s;
          server api3:61370 max_fails=3 fail_timeout=30s;
    }

    server {
          listen 80; 
          server_name server_name 0.0.0.0;

          location / {
              proxy_pass http://wordSearcherApi;
          }
    }
}

Is there any way to create just one service in docker-compose.yml and when docker-compose up --scale api=3, does nginx do automatic load balance?

Answer 1

Nginx

Dynamic upstreams are possible in Nginx (normal, sans Plus) but with tricks and limitations.

1. You give up on upstream directive and use plain proxy_pass.

   It gives round robin load balancing and failover, but no extra feature of the directive like weights, failure modes, timeout, etc.

2. Your upstream hostname must be passed to proxy_pass by a variable and you must provide a resolver.

   It forces Nginx to re-resolve the hostname (against Docker networks' DNS).

3. You lose location/proxy_pass behaviour related to trailing slash.

   In the case of reverse-proxying to bare / like in the question, it does not matter. Otherwise you have to manually rewrite the path (see the references below).

Let's see how it works.

docker-compose.yml

version: '2.2'
services:
  reverse-proxy:
    image: nginx:1.15-alpine
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
    ports:
      - 8080:8080
  app:
    # A container that exposes an API to show its IP address
    image: containous/whoami
    scale: 4

nginx.conf

worker_processes  1;

events {
  worker_connections  1024;
}

http {
  access_log /dev/stdout;
  error_log /dev/stderr;

  server {
    listen 8080;
    server_name localhost;

    resolver 127.0.0.11 valid=5s;
    set $upstream app;

    location / {
      proxy_pass http://$upstream:80;
    }
  }
}

Then...

docker-compose up -d
seq 10 | xargs -I -- curl -s localhost:8080 | grep "IP: 172"

...produces something like the following which indicates the requests are distributed across 4 app containers:

IP: 172.30.0.2
IP: 172.30.0.2
IP: 172.30.0.3
IP: 172.30.0.3
IP: 172.30.0.6
IP: 172.30.0.5
IP: 172.30.0.3
IP: 172.30.0.6
IP: 172.30.0.5
IP: 172.30.0.5

References:

1. Nginx with dynamic upstreams
2. Using Containers to Learn Nginx Reverse Proxy
3. Dynamic Nginx configuration for Docker with Python

Traefik

Traefik relies on Docker API directly and may be a simpler and more configurable option. Let's see it in action.

docker-compose.yml

version: '2.2'
services:
  reverse-proxy:
    image: traefik  
    # Enables the web UI and tells Traefik to listen to docker
    command: --api --docker  
    ports:
      - 8080:80      
      - 8081:8080  # Traefik's web UI, enabled by --api
    volumes:
      # So that Traefik can listen to the Docker events
      - /var/run/docker.sock:/var/run/docker.sock  
  app:
    image: containous/whoami
    scale: 4
    labels:
      - "traefik.frontend.rule=Host:localhost"

Then...

docker-compose up -d
seq 10 | xargs -I -- curl -s localhost:8080 | grep "IP: 172"

...also produces something the output that indicates the requests are distributed across 4 app containers:

IP: 172.31.0.2
IP: 172.31.0.5
IP: 172.31.0.6
IP: 172.31.0.4
IP: 172.31.0.2
IP: 172.31.0.5
IP: 172.31.0.6
IP: 172.31.0.4
IP: 172.31.0.2
IP: 172.31.0.5

In the Traefik UI (http://localhost:8081/dashboard/ in the example) you can see it recognised the 4 app containers:

References:

1. The Traefik Quickstart (Using Docker)

Answer 2

It's not possible with your current config since it's static. You have two options -

1. Use docker engine swarm mode - You can define replicas & swarm internal DNS will automatically balance the load across those replicas.
Ref - https://docs.docker.com/engine/swarm/

2. Use famous Jwilder nginx proxy - This image listens to the docker sockets, uses templates in GO to dynamically change your nginx configs when you scale your containers up or down.
Ref - https://github.com/jwilder/nginx-proxy