Using docker-compose with CI - how to deal with ex

Question

Right now our Jenkins agents generate a docker-compose.yml for each of our Rails projects and then run docker-compose up. The docker-compose.yml has a main "web" container that has rbenv and all of our other Rails dependencies inside. It is linked to a DB container that contains the test Postgres DB.

The problem comes when we need to actually run the tests and generate exit codes. Our CI server will only deploy if the test script returns exit 0, but docker-compose always returns 0, even if one of the container commands fail.

The other issue is that the DB container runs indefinitely, even after the web container is done running the tests, so docker-compose up never returns.

Is there a way we can use docker-compose for this process? We would need to be able to run the containers, but exit after the web container is complete and return it's exit code. Right now we are stuck manually using docker to spin up the DB container and run the web container with the --link option.

Answer 1

docker-compose run is the simple way to get the exit statuses you desire. For example:

$ cat docker-compose.yml 
roit:
    image: busybox
    command: 'true'
naw:
    image: busybox
    command: 'false'
$ docker-compose run --rm roit; echo $?
Removing test_roit_run_1...
0
$ docker-compose run --rm naw; echo $?
Removing test_naw_run_1...
1

Alternatively, you do have the option to inspect the dead containers. You can use the -f flag to get just the exit status.

$ docker-compose up
Creating test_naw_1...
Creating test_roit_1...
Attaching to test_roit_1
test_roit_1 exited with code 0
Gracefully stopping... (press Ctrl+C again to force)
$ docker-compose ps -q | xargs docker inspect -f '{{ .Name }} exited with status {{ .State.ExitCode }}'
/test_naw_1 exited with status 1
/test_roit_1 exited with status 0

As for the db container that never returns, if you use docker-compose up then you will need to sigkill that container; that's probably not what you want. Instead, you can use docker-compose up -d to run your containers daemonized, and manually kill the containers when your test is complete. docker-compose run should run linked containers for you, but I have heard chatter on SO about a bug preventing that from working as intended right now.

Answer 2

Since version 1.12.0, you can use the --exit-code-from option.

From documentation:

--exit-code-from SERVICE

Return the exit code of the selected service container. Implies --abort-on-container-exit.

Answer 3

Building on kojiro's answer:

docker-compose ps -q | xargs docker inspect -f '{{ .State.ExitCode }}' | grep -v '^0' | wc -l | tr -d ' '

1. get container IDs
2. get last runs exit code for each container ID
3. only status codes that does not start with '0'
4. count number of non-0 status codes
5. trim out white space

Returns how many non-0 exit codes were returned. Would be 0 if everything exited with code 0.

Answer 4

docker-rails allows you to specify which container's error code is returned to the main process, so you CI server can determine the result. It is a great solution for CI and development for rails with docker.

For example

exit_code: web

in your docker-rails.yml will yield the web containers exit code as a result of the command docker-rails ci test. docker-rails.yml is just a meta wrapper around the standard docker-compose.yml that gives you the potential to inherit/reuse the same base config for different environments i.e. development vs test vs parallel_tests.

Answer 5

You can see exist status with:

echo $(docker-compose ps | grep "servicename" | awk '{print $4}')

Answer 6

If you're willing to use docker-compose run to manually kick off your tests, adding the --rm flag, oddly enough, causes Compose to accurately reflect your command's exit status.

Here's my example:

$ docker-compose -v
docker-compose version 1.7.0, build 0d7bf73

$ (docker-compose run kpi false) || echo 'Test failed!'  # False negative.

$ (docker-compose run --rm kpi false) || echo 'Test failed!'  # True positive.
Test failed!

$ (docker-compose run --rm kpi true) || echo 'Test failed!'  # True negative.