Docker Compose wait for container X before startin

问题:

I am using rabbitmq and a simple python sample from here together with docker-compose. My problem is that I need to wait for rabbitmq to fully started. From what I searched so far, I don't know how to wait with container x ( in my case worker ) until y (rabbitmq) is started.

I found this blogpost where he checks if the other host is online. I also found this docker command:

wait

Usage: docker wait CONTAINER [CONTAINER...]

Block until a container stops, then print its exit code.

Waiting for a container to stop is maybe not what I am looking for but if it is, is it possible to use that command inside the docker-compose.yml ? My solution so far is to wait some seconds and check the port, but is this the way to achieve this?. If I don't wait I get an error.

docker-compose.yml

worker:
    build: myapp/.
    volumes:
    - myapp/.:/usr/src/app:ro

    links:
    - rabbitmq
rabbitmq:
    image: rabbitmq:3-management

python hello sample (rabbit.py):

import pika
import time

import socket

pingcounter = 0
isreachable = False
while isreachable is False and pingcounter < 5:
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    try:
        s.connect(('rabbitmq', 5672))
        isreachable = True
    except socket.error as e:
        time.sleep(2)
        pingcounter += 1
    s.close()

if isreachable:
    connection = pika.BlockingConnection(pika.ConnectionParameters(
            host="rabbitmq"))
    channel = connection.channel()

    channel.queue_declare(queue='hello')

    channel.basic_publish(exchange='',
                          routing_key='hello',
                          body='Hello World!')
    print (" [x] Sent 'Hello World!'")
    connection.close()

Dockerfile for worker:

FROM python:2-onbuild
RUN ["pip", "install", "pika"]

CMD ["python","rabbit.py"]

Update Nov 2015:

A shell script or waiting inside your program is maybe a possible solution. But after seeing this Issue I am looking for a command or feature of docker/docker-compose itself.

They mention a solution for implementing a health check, which may be the best option. A open tcp connection does not mean your service is ready or may remain ready. In addition to that I need to change my entrypoint in my dockerfile.

So I am hoping for an answer with docker-compose on board commands, which will hopefully the case if they finish this issue.

Update March 2016

There is a proposal for providing a built-in way to determine if a container is "alive". So docker-compose can maybe make use of it in near future.

Update June 2016

It seems that the healthcheck will be integrated into docker in Version 1.12.0

Update January 2017

I found a docker-compose solution see: Docker Compose wait for container X before starting Y

回答1:

Finally found a solution with a docker-compose method. Since docker-compose file format 2.1 you can define healthchecks.

I did it in a example project you need to install at least docker 1.12.0+. I also needed to extend the rabbitmq-management Dockerfile, because curl isn't installed on the official image.

Now I test if the management page of the rabbitmq-container is available. If curl finishes with exitcode 0 the container app (python pika) will be started and publish a message to hello queue. Its now working (output).

docker-compose (version 2.1):

version: '2.1'

services:
  app:
    build: app/.
    depends_on:
      rabbit:
        condition: service_healthy
    links: 
        - rabbit

  rabbit:
    build: rabbitmq/.
    ports: 
        - "15672:15672"
        - "5672:5672"
    healthcheck:
        test: ["CMD", "curl", "-f", "http://localhost:15672"]
        interval: 30s
        timeout: 10s
        retries: 5

output:

rabbit_1  | =INFO REPORT==== 25-Jan-2017::14:44:21 ===
rabbit_1  | closing AMQP connection <0.718.0> (172.18.0.3:36590 -> 172.18.0.2:5672)
app_1     |  [x] Sent 'Hello World!'
healthcheckcompose_app_1 exited with code 0

Dockerfile (rabbitmq + curl):

FROM rabbitmq:3-management
RUN apt-get update
RUN apt-get install -y curl 
EXPOSE 4369 5671 5672 25672 15671 15672

Version 3 no longer supports the condition form of depends_on. So i moved from depends_on to restart on-failure. Now my app container will restart 2-3 times until it is working, but it is still a docker-compose feature without overwriting the entrypoint.

docker-compose (version 3):

version: "3"

services:

  rabbitmq: # login guest:guest
    image: rabbitmq:management
    ports:
    - "4369:4369"
    - "5671:5671"
    - "5672:5672"
    - "25672:25672"
    - "15671:15671"
    - "15672:15672"
    healthcheck:
        test: ["CMD", "curl", "-f", "http://localhost:15672"]
        interval: 30s
        timeout: 10s
        retries: 5

  app:
    build: ./app/
    environment:
      - HOSTNAMERABBIT=rabbitmq
    restart: on-failure
    depends_on:
      - rabbitmq
    links: 
        - rabbitmq

回答2:

Natively that is not possible, yet. See also this feature request.

So far you need to do that in your containers CMD to wait until all required services are there.

In the Dockerfiles CMD you could refer to your own start script that wraps starting up your container service. Before you start it, you wait for a depending one like:

Dockerfile

FROM python:2-onbuild
RUN ["pip", "install", "pika"]
ADD start.sh /start.sh
CMD ["/start.sh"]

start.sh

#!/bin/bash
while ! nc -z rabbitmq 5672; do sleep 3; done
python rabbit.py

Probably you need to install netcat in your Dockerfile as well. I do not know what is pre-installed on the python image.

There are a few tools out there that provide easy to use waiting logic, for simple tcp port checks:

• wait-for-it
• dockerize

For more complex waits:

• goss - Explanation blog

回答3:

Using restart: unless-stopped or restart: always may solve this problem.

If worker container stops when rabbitMQ is not ready, it will be restarted until it is.

回答4:

Quite recently they've added the depends_on feature.

Edit:

As of compose version 2.1+ you can use depends_on in conjunction with healthcheck to achieve this:

From the docs:

version: '2.1'
services:
  web:
    build: .
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_started
  redis:
    image: redis
  db:
    image: redis
    healthcheck:
      test: "exit 0"

Before version 2.1

You can still use depends_on, but it only effects the order in which services are started - not if they are ready before the dependant service is started.

It seems to require at least version 1.6.0.

Usage would look something like this:

version: '2'
services:
  web:
    build: .
    depends_on:
      - db
      - redis
  redis:
    image: redis
  db:
    image: postgres 

From the docs:

Express dependency between services, which has two effects:

• docker-compose up will start services in dependency order. In the following example, db and redis will be started before web.
• docker-compose up SERVICE will automatically include SERVICE’s dependencies. In the following example, docker-compose up web will also create and start db and redis.

Note: As I understand it, although this does set the order in which containers are loaded. It does not guarantee that the service inside the container has actually loaded.

For example, you postgres container might be up. But the postgres service itself might still be initializing within the container.

回答5:

you can also just add it to the command option eg.

command: bash -c "sleep 5; start.sh"

https://github.com/docker/compose/issues/374#issuecomment-156546513

to wait on a port you can also use something like this

command: bash -c "while ! curl -s rabbitmq:5672 > /dev/null; do echo waiting for xxx; sleep 3; done; start.sh"

to increment the waiting time you can hack a bit more:

command: bash -c "for i in {1..100} ; do if ! curl -s rabbitmq:5672 > /dev/null ; then echo waiting on rabbitmq for $i seconds; sleep $i; fi; done; start.sh"

回答6:

restart: on-failure did the trick for me..see below

---
version: '2.1'
services:
  consumer:
    image: golang:alpine
    volumes:
      - ./:/go/src/srv-consumer
    working_dir: /go/src/srv-consumer
    environment:
      AMQP_DSN: "amqp://guest:guest@rabbitmq:5672"
    command: go run cmd/main.go
    links:
          - rabbitmq
    restart: on-failure

  rabbitmq:
    image: rabbitmq:3.7-management-alpine
    ports:
      - "15672:15672"
      - "5672:5672"

回答7:

You can also solve this by setting an endpoint which waits for the service to be up by using netcat (using the docker-wait script). I like this approach as you still have a clean command section in your docker-compose.yml and you don't need to add docker specific code to your application:

version: '2'
services:
  db:
    image: postgres
  django:
    build: .
    command: python manage.py runserver 0.0.0.0:8000
    entrypoint: ./docker-entrypoint.sh db 5432
    volumes:
      - .:/code
    ports:
      - "8000:8000"
    depends_on:
      - db

Then your docker-entrypoint.sh:

#!/bin/sh

postgres_host=$1
postgres_port=$2
shift 2
cmd="$@"

# wait for the postgres docker to be running
while ! nc $postgres_host $postgres_port; do
  >&2 echo "Postgres is unavailable - sleeping"
  sleep 1
done

>&2 echo "Postgres is up - executing command"

# run the command
exec $cmd

This is nowadays documented in the official docker documentation.

PS: You should install netcat in your docker instance if this is not available. To do so add this to your Docker file :

RUN apt-get update && apt-get install netcat-openbsd -y 

回答8:

For container start ordering use

depends_on:

For waiting previous container start use script

entrypoint: ./wait-for-it.sh db:5432

This article will help you https://docs.docker.com/compose/startup-order/

回答9:

There is a ready to use utility called "docker-wait" that can be used for waiting.

回答10:

basing on this blog post https://8thlight.com/blog/dariusz-pasciak/2016/10/17/docker-compose-wait-for-dependencies.html

I configured my docker-compose.yml as shown below:

version: "3.1"

services:
  rabbitmq:
    image: rabbitmq:3.7.2-management-alpine
    restart: always
    environment:
      RABBITMQ_HIPE_COMPILE: 1
      RABBITMQ_MANAGEMENT: 1
      RABBITMQ_VM_MEMORY_HIGH_WATERMARK: 0.2
      RABBITMQ_DEFAULT_USER: "rabbitmq"
      RABBITMQ_DEFAULT_PASS: "rabbitmq"
    ports:
      - "15672:15672"
      - "5672:5672"
    volumes:
      - data:/var/lib/rabbitmq:rw

  start_dependencies:
    image: alpine:latest
    links:
      - rabbitmq
    command: >
      /bin/sh -c "
        echo Waiting for rabbitmq service start...;
        while ! nc -z rabbitmq 5672;
        do
          sleep 1;
        done;
        echo Connected!;
      "

volumes:
  data: {}

Then I do for run =>:

docker-compose up start_dependencies

rabbitmq service will start in daemon mode, start_dependencies will finish the work.

回答11:

One of the alternative solution is to use a container orchestration solution like Kubernetes. Kubernetes has support for init containers which run to completion before other containers can start. You can find an example here with SQL Server 2017 Linux container where API container uses init container to initialise a database

https://www.handsonarchitect.com/2018/08/understand-kubernetes-object-init.html

回答12:

I couldn't use this approach, I actually can't use volume in docker-compose, in both cases I didn't work:

s-flyway_1_d7a78571d25c | env: can't execute 'bash': No such file or directory docker_s-flyway_1_d7a78571d25c exited with code 127

What am I missing here ? I'm a little lost about it right now.....

Project link: https://github.com/jbrasileiro/introducing-example/blob/master/introducing-example-docker-flyway/src/docker/docker-compose.yml

Dockerfile

FROM dhoer/flyway:alpine
COPY ./sql/*.sql /flyway/sql
VOLUME ./wait-for-it.sh:/usr/local/bin/wait-for-it.sh
ADD ./wait-for-it.sh /
RUN chmod 777 /wait-for-it.sh 

docker version : '3'

version: '3'
services:
  hello-world:
    image: ubuntu
    volumes:
      - ./wait-for-it.sh:/usr/local/bin/wait-for-it.sh
    command:
      - wait-for-it.sh
      - github.com:80
      - --
      - echo
      - "hello world"

docker version : '2'

version: '2'
services:
  s-postgres:
    image: postgres:9.4-alpine
    hostname: host-postgres
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
      - POSTGRES_DB=postgres
    networks:
       - network-docker
  s-flyway:
    image: dhoer/flyway:alpine
    build: flyway/
    entrypoint: /wait-for-it.sh s-postgres 5432 --timeout=0 --strict
    command: -url=jdbc:postgresql://s-postgres:5432/postgres -baselineOnMigrate=true -schemas=public -user=postgres -password=postgres migrate
    networks:
       - network-docker
    depends_on:
      - s-postgres
networks:
  network-docker:
    driver: bridge