Run docker-compose build in .gitlab-ci.yml

Question

I have a .gitlab-ci.yml file which contains following:

image: docker:latest

services:
  - docker:dind

before_script:
  - docker info
  - docker-compose --version

buildJob:
  stage: build
  tags:
    - docker
  script:
    - docker-compose build

But in ci-log I receive message:

$ docker-compose --version
/bin/sh: eval: line 46: docker-compose: not found

What am I doing wrong?

Answer 1

If you don't want to provide a custom docker image with docker-compose preinstalled, you can get it working by installing Python during build time. With Python installed you can finally install docker-compose ready for spinning up your containers.

image: docker:latest

services:
- docker:dind

before_script:
- apk add --update python py-pip python-dev && pip install docker-compose # install docker-compose
- docker version
- docker-compose version

test:
  cache:
    paths:
    - vendor/
  script:
  - docker-compose up -d
  - docker-compose exec -T php-fpm composer install --prefer-dist
  - docker-compose exec -T php-fpm vendor/bin/phpunit --coverage-text --colors=never --whitelist src/ tests/

Use docker-compose exec with -T if you receive this or a similar error:

$ docker-compose exec php-fpm composer install --prefer-dist
Traceback (most recent call last):
  File "/usr/bin/docker-compose", line 9, in <module>
    load_entry_point('docker-compose==1.8.1', 'console_scripts', 'docker-compose')()
  File "/usr/lib/python2.7/site-packages/compose/cli/main.py", line 62, in main
    command()
  File "/usr/lib/python2.7/site-packages/compose/cli/main.py", line 114, in perform_command
    handler(command, command_options)
  File "/usr/lib/python2.7/site-packages/compose/cli/main.py", line 442, in exec_command
    pty.start()
  File "/usr/lib/python2.7/site-packages/dockerpty/pty.py", line 338, in start
    io.set_blocking(pump, flag)
  File "/usr/lib/python2.7/site-packages/dockerpty/io.py", line 32, in set_blocking
    old_flag = fcntl.fcntl(fd, fcntl.F_GETFL)
ValueError: file descriptor cannot be a negative integer (-1)
ERROR: Build failed: exit code 1

Answer 2

EDIT I added another answer providing a minimal example for a .gitlab-ci.yml configuration supporting docker-compose.

---

docker-compose can be installed as a Python package, which is not shipped with your image. The image you chose does not even provide an installation of Python:

$ docker run --rm -it docker sh
/ # find / -iname "python"
/ # 

Looking for Python gives an empty result. So you have to choose a different image, which fits to your needs and ideally has docker-compose installed or you maually create one.

The docker image you chose uses Alpine Linux. You can use it as a base for your own image or try a different one first if you are not familiar with Alpine Linux.

I had the same issue and created a Dockerfile in a public GitHub repository and connected it with my Docker Hub account and chose an automated build to build my image on each push to the GitHub repository. Then you can easily access your own images with the GitLab CI.

Answer 3

Alpine linux now has a docker-compose package in their "edge" branch, so you can install it this way in .gitlab-ci.yml

a-job-with-docker-compose:
  image: docker
  services:
    - docker:dind
  script:
    - apk add docker-compose --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/testing/ --allow-untrusted
    - docker-compose -v

Answer 4

I created a simple docker container which has docker-compose installed on top of docker:latest. See https://hub.docker.com/r/tmaier/docker-compose/

Your .gitlab-ci.yml file would look like this:

image: tmaier/docker-compose:latest

services:
  - docker:dind

before_script:
  - docker info
  - docker-compose --version

buildJob:
  stage: build
  tags:
    - docker
  script:
    - docker-compose build

Answer 5

docker-compose now needs more dev-packages, which is described in their official documentation. This is the advised solution, but does not really satisfy me... Too many packages to get it up and running:

# .gitlab-ci.yml
image: docker
services:
  - docker:dind    
build:
  script:
    - apk add --no-cache py-pip python-dev libffi-dev openssl-dev gcc libc-dev make
    - pip install docker-compose
    - docker-compose up -d

I think now it's time to rely on a 3rd-party image or I'd personally configure a nightly build to do the steps above and automatically build a compose-docker-image in our CI for our registry.

We personally do not follow this flow anymore, because you loose control about the running containers and they might end up running endless. This is because of the docker-in-docker executor. We developed a python-script as a workaround to kill all old containers in our CI, which can be found here. But I do not suggest to start containers like this anymore.

Answer 6

Docker also provides an official image: docker/compose

This is the ideal solution if you don't want to install it every pipeline.

Since the image uses docker-compose-entrypoint.sh as entrypoint you'll need to override it back to /bin/sh -c in your .gitlab-ci.yml. Otherwise your pipeline will fail with No such command: sh

variables:
  DOCKER_HOST: tcp://docker:2375/
  DOCKER_DRIVER: overlay2

# Official docker compose image.
image:
  name: docker/compose:latest
  entrypoint: ["/bin/sh", "-c"]

services:
  - docker:dind

before_script:
  - docker version
  - docker-compose version

build:
  stage: build
  script:
    - docker-compose down
    - docker-compose build
    - docker-compose up tester-image