Running tests with Docker and Jenkins - Test Resul

I have an application written in Rails. In order to test it I've built a docker image. Everything works fine.

The problem, however, appears when I configure Jenkins to run those tests. Here's what I do, at the building step:

docker-compose up rspec

Where rspec is a service defined at docker-compose.yml and contains the following command:

command: "rspec spec/"

When rspec returns an error the build still succeeds. Here an example of the output:

    ...
21:42:24 [36mrspec_1  |[0m       should save second profile
21:42:24 [36mrspec_1  |[0m 
21:42:24 [36mrspec_1  |[0m Failures:
21:42:24 [36mrspec_1  |[0m 
21:42:24 [36mrspec_1  |[0m   1) New profile Should persist new_profile_pricture
21:42:24 [36mrspec_1  |[0m      Failure/Error: jump_to_four_phase_with(new_profile_picture)
21:42:24 [36mrspec_1  |[0m      RuntimeError:
21:42:24 [36mrspec_1  |[0m        Timeout for '#new_profile' (1) appearance reached!
...
21:42:25 [36mcomposes_rspec_1 exited with code 1
21:42:25 [0m[Profiler] $ /bin/sh -xe /tmp/hudson4606189750126491465.sh
21:42:25 Finished: SUCCESS

36mcomposes_rspec_1 returned 1 and the build still succeeded.

If I check the container by its id with docker ps -a I get "Exited (1) 2 minutes ago"

Do you guys know what is going on?

Is there an easy way to fail the build when the container fails?

Jenkins uses the exit status of the process to judge success or failure.

docker-compose up is designed to orchestrate many containers. When you are dealing with multiple services/containers there is a bit of a grey area as to what constitutes success and failure. All that docker-compose reports on exit is that the docker-compose command completed successfully, not that all containers it ran are ok.

docker-compose run <service> <command> will run a single command for a service and return that commands exit status.

If you rely on multiple services/containers for the tests, then have docker-compose up only bring up the required services. Then run docker-compose run rubyservice rspec afterwards for your tests.

Compose seperation

If you want to keep the tests seperate from the app containers, create a second docker-compose-test.yml file that contains a service definition just for the tests.

version: "2.1"
tests:
  build:
    context: .
    dockerfile: Dockerfile.tests
  cmd: rspec

After your main app containers have been brought up, run

docker-compose -f docker-compose-test.yml run tests