Setting up multiple Docker containers and MongoDB

I'm trying to setup a continuous integration flow for a group of several Rails service apps I have that communicate with each other through endpoints. The idea is to configure CircleCI so that when the test flow is triggered, it will pull down preconfigured Docker images for each of the apps, start Docker containers for each one, and then run a test suite that tests that the full-flow integration works from one app through the other. One of the service apps uses MongoDB, so it needs to also communicate with the mongodb that CircleCI automatically installs. The flow should be: client_app -> service_app -> mongodb However, I'm having problems getting the containers to connect.

The Dockerfile for client_app installs Ruby and all dependencies, adds the repo into the image, and then runs:

RUN bundle install

EXPOSE 3000

CMD ["bundle", "exec", "rails", "s", "-e", "development", "-p", "3000"]

The Dockerfile for service_app does the same, then

RUN bundle install

EXPOSE 8080

CMD ["bundle", "exec", "rails", "s", "-e", "test", "-p", "8080"]

Both these Dockerfiles are stored in our private Docker repo. I have built and pulled these Docker images locally and confirmed that they start correctly.

When the flow is triggered on CircleCI, I use circle.yml to pull down each of the images. This is my circle.yml (app names changed):

machine:
  services:
    - docker

dependencies:
  pre:
    - sed "s/<EMAIL>/$DOCKER_EMAIL/;s/<AUTH>/$DOCKER_AUTH/" < .dockercfg.template > ~/.dockercfg
    - docker pull myorg/service_app
    - docker pull myorg/client_app

test:
  override:
    - docker run -d -p 8080:8080 --name service_app myorg/service_app:docker-test
    - docker run -d -p 3000:3000 --env SERVICE_APP_URL=http://localhost:8080 --name client_app myorg/client_app:docker-test
    - docker ps -a
    - bundle exec rspec spec

client_app should be configured to communicate with service_app at SERVICE_APP_URL (internally the app starts the connection to ENV['SERVICE_APP_URL']), so since the service_app container is running on port 8080, I set it to http://localhost:8080, but it doesn't work. When I look in the logs for client_app, in the first view where it is supposed to make a call, it returns:

Connection refused - connect(2) for "localhost" port 8080

I added docker ps -a into the circle.yml to see if the containers were being started correctly. Here's what it output at that step:

CONTAINER ID        IMAGE                           COMMAND                CREATED             STATUS              PORTS                    NAMES
782d09c4f3db        myorg/client_app:docker-test    bundle exec rails s    3 seconds ago       Up 2 seconds        0.0.0.0:3000->3000/tcp   berserk_mcclintock   
64f8af8ab535        myorg/service_app:docker-test   bundle exec rails s    5 seconds ago       Up 4 seconds        0.0.0.0:8080->8080/tcp   furious_wozniak

So it looks as if the containers are started and the right ports are exposed, but client_app still isn't connecting to service_app.

I did look into using Docker's linking functionality, but if I understand it correctly, it requires the app to be internally configured to look for an environment variable setup by Docker, such as DB_PORT_5000_TCP if the linked container is named db, and I would like to avoid having to modify the internal configurations if possible.

Additionally, I will need service_app to talk to a running mongodb. Currently the app is set to connect to localhost:27017, which seems to be what CircleCI starts the Mongo service on, but I'm not sure if the Docker container will be able to see it.

Edit: I've tried to configure my service_app to talk to a running MongoDB container (using the trusted Mongo build using --link but that's not working either. I pulled the latest mongo image, then ran:

docker run -d -p 27017:27017 -p 28017:28017 --name mongodb dockerfile/mongodb mongod --rest --httpinterface

as suggested on that page, then ran

docker run -d -p 8080:8080 --name service_app --link mongodb:mongodb myorg/service_app:v1

In my service_app, before building, I had configured mongoid.yml:

test:
  sessions:
    default:
      database: test
      hosts:
        -  ENV['MONGODB_PORT'] || 'localhost:27017' %>
      options:
        safe: true

My understanding is Docker should set the MONGODB_PORT var when linking containers like this, and so it should connect to the Mongo container. I ran env inside the container and it had set MONGODB_PORT=tcp://172.17.0.95:27017.

However, on my local machine, when I try to connect to service_app to make a query, it returns

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN">
<HTML>
    <HEAD>
        <TITLE>Internal Server Error</TITLE>
    </HEAD>
    <BODY>
        <H1>Internal Server Error</H1>
    Could not connect to any secondary or primary nodes for replica set &lt;Moped::Cluster nodes=[&lt;Moped::Node resolved_address=nil&gt;]&gt;

        <HR>
            <ADDRESS>
     WEBrick/1.3.1 (Ruby/2.0.0/2014-02-24) at
     192.168.59.103:8080
     </ADDRESS>
        </BODY>
    </HTML>

So clearly I don't have this linking setup right. Any ideas?

The problem is related to point of origin. If you access localhost from inside a Docker container, the localhost points to the docker itself, not the host server, and therefore the call is not reaching the host or the other dockers.

If all your dockers are on the same host machine, the easiest way to make them talk with each other via localhost is to share a network between the containers at run. First run the backend normally, then start the other containers with the --net switch:

docker run [other params] -d -p 8080:8080 --name service-app-container service_app_image docker run [other params] -p 3000:3000 --net="container:service-app-container" --name client-app-container client_app_image

Now all the ports exposed and mapped by any of the containers sharing the network should be reachable under localhost from any of the dockers.

To access the service-app-container from anywhere, use machine:8080.
To access it locally from the host server, use localhost:8080.
To access it internally from the service app, use locahost:8080.
To access the service app from the client app, use localhost:8080

The client app will only be exposed over port 3000 to the world if you also expose it in the docker that has the container. The exposed port should not be the same as this confuses the network:

docker run [other params] -d -p 8080:8080 3001:3000 --name service-app-container service_app_image docker run [other params] -p 3000:3000 --net="container:service-app-container" client_app_image

Now you could access the client app from outside using machine:3001.