Docker, Django and Selenium - Selenium unable to c

Question

I have Docker configured to run Postgres and Django using docker-compose.yml and it is working fine.

The trouble I am having is with Selenium not being able to connect to the Django liveserver.

Now it makes sense (to me at least) that django has to access selenium to control the browser and selenium has to access django to access the server.

I have tried using the docker 'ambassador' pattern using the following configuration for docker-compose.yml from here: https://github.com/docker/compose/issues/666

postgis:
  dockerfile: ./docker/postgis/Dockerfile
  build: .
  container_name: postgis

django-ambassador:
  container_name: django-ambassador
  image: cpuguy83/docker-grand-ambassador
  volumes:
    - "/var/run/docker.sock:/var/run/docker.sock"
  command: "-name django -name selenium"

django:
  dockerfile: ./docker/Dockerfile-dev
  build: .
  command: python /app/project/manage.py test my-app
  container_name: django
  volumes:
    - .:/app
  ports:
    - "8000:8000"
    - "8081:8081"
  links:
    - postgis
    - "django-ambassador:selenium"
  environment:
    - SELENIUM_HOST=http://selenium:4444/wd/hub

selenium:
  container_name: selenium
  image: selenium/standalone-firefox-debug
  ports:
    - "4444:4444"
    - "5900:5900"
  links:
    - "django-ambassador:django"

When I check http://DOCKER-MACHINE-IP:4444/wd/hub/static/resource/hub.html I can see that firefox starts, but all the tests fail as firefox is unable to connect to django

'Firefox can't establish a connection to the server at localhost:8081'

I also tried this solution here https://github.com/docker/compose/issues/1991 however this is not working cause I can't get django to connect to postgis and selenium at the same time

'django.db.utils.OperationalError: could not translate host name "postgis" to address: Name or service not known'

I tried using the networking feature as listed below

postgis:
  dockerfile: ./docker/postgis/Dockerfile
  build: .
  container_name: postgis
  net: appnet

django:
  dockerfile: ./docker/Dockerfile-dev
  build: .
  command: python /app/project/manage.py test foo
  container_name: django
  volumes:
    - .:/app
  ports:
    - "8000:8000"
    - "8081:8081"
  net: appnet
  environment:
    - SELENIUM_HOST=http://selenium:4444/wd/hub

selenium:
  container_name: selenium
  image: selenium/standalone-firefox-debug
  ports:
    - "4444:4444"
    - "5900:5900"
  net: appnet

but the result is the same

'Firefox can't establish a connection to the server at localhost:8081'

So how can I get selenium to connect to django?

I have been playing around with this for days - would really appreciate any help.

More Info

Another weird thing is that when the testserver is running not using docker (using my old config of virtualenv etc.) if I run ./manage.py test foo I can access the server through any browser at http://localhost:8081 and get served up webpages, but I can't access the test server when I run the equivalent command if I run it under docker. This is weird cause I am mapping port 8081:8081 - is this related?

Note: I am using OSX and Docker v1.9.1

Answer 1

Whenever you see localhost, try first to port-forward that port (at the VM level)

See "Connect to a Service running inside a docker container from outside"

VBoxManage controlvm "default" natpf1 "tcp-port8081,tcp,,8081,,8081"
VBoxManage controlvm "default" natpf1 "udp-port8081,udp,,8081,,8081"

(Replace default with the name of your docker-machine: see docker-machine ls)

This differs for port mapping at the docker host level (which is your boot2docker-based Linux host)

The OP luke-aus confirms in the comments:

entering the IP address for the network solved the problem!

Answer 2

I ended up coming up with a better solution that didn't require me to hardcode the IP Address. Below is the configuration I used to run tests in django with docker.

Docker-compose file

# docker-compose base file for everything
version: '2'

services:
  postgis:
    build:
      context: .
      dockerfile: ./docker/postgis/Dockerfile
    container_name: postgis
    volumes:
      # If you are using boot2docker, postgres data has to live in the VM for now until #581 fixed
      # for more info see here: https://github.com/boot2docker/boot2docker/issues/581
      - /data/dev/docker_cookiecutter/postgres:/var/lib/postgresql/data

  django:
    build:
      context: .
      dockerfile: ./docker/django/Dockerfile
    container_name: django
    volumes:
      - .:/app
    depends_on:
      - selenium
      - postgis
    environment:
      - SITE_DOMAIN=django
      - DJANGO_SETTINGS_MODULE=settings.my_dev_settings
    links:
      - postgis
      - mailcatcher

  selenium:
    container_name: selenium
    image: selenium/standalone-firefox-debug:2.52.0
    ports:
      - "4444:4444"
      - "5900:5900"

Dockerfile (for Django)

ENTRYPOINT ["/docker/django/entrypoint.sh"]

In Entrypoint file

#!/bin/bash
set -e

# Now we need to get the ip address of this container so we can supply it as an environmental
# variable for django so that selenium knows what url the test server is on
# Use below or alternatively you could have used
# something like "$@ --liveserver=$THIS_DOCKER_CONTAINER_TEST_SERVER"

if [[ "'$*'" == *"manage.py test"* ]]  # only add if 'manage.py test' in the args
then
  # get the container id
  THIS_CONTAINER_ID_LONG=`cat /proc/self/cgroup | grep 'docker' | sed 's/^.*\///' | tail -n1`
  # take the first 12 characters - that is the format used in /etc/hosts
  THIS_CONTAINER_ID_SHORT=${THIS_CONTAINER_ID_LONG:0:12}
  # search /etc/hosts for the line with the ip address which will look like this:
  #     172.18.0.4    8886629d38e6
  THIS_DOCKER_CONTAINER_IP_LINE=`cat /etc/hosts | grep $THIS_CONTAINER_ID_SHORT`
  # take the ip address from this
  THIS_DOCKER_CONTAINER_IP=`(echo $THIS_DOCKER_CONTAINER_IP_LINE | grep -o '[0-9]\+[.][0-9]\+[.][0-9]\+[.][0-9]\+')`
  # add the port you want on the end
  # Issues here include: django changing port if in use (I think)
  # and parallel tests needing multiple ports etc.
  THIS_DOCKER_CONTAINER_TEST_SERVER="$THIS_DOCKER_CONTAINER_IP:8081"
  echo "this docker container test server = $THIS_DOCKER_CONTAINER_TEST_SERVER"
  export DJANGO_LIVE_TEST_SERVER_ADDRESS=$THIS_DOCKER_CONTAINER_TEST_SERVER
fi


eval "$@"

In your django settings file

SITE_DOMAIN = 'django'

Then to run your tests

docker-compose run django ./manage.py test

Answer 3

I've been struggling with this as well, and I finally found a solution that worked for me. You can try something like this:

postgis:
  dockerfile: ./docker/postgis/Dockerfile
  build: .

django:
  dockerfile: ./docker/Dockerfile-dev
  build: .
  command: python /app/project/manage.py test my-app
  volumes:
    - .:/app
  ports:
    - "8000:8000"
  links:
    - postgis
    - selenium  # django can access selenium:4444, selenium can access django:8081-8100
  environment:
    - SELENIUM_HOST=http://selenium:4444/wd/hub
    - DJANGO_LIVE_TEST_SERVER_ADDRESS=django:8081-8100  # this gives selenium the correct address

selenium:
  image: selenium/standalone-firefox-debug
  ports:
    - "5900:5900"

I don't think you need to include port 4444 in the selenium config. That port is exposed by default, and there's no need to map it to the host machine, since the django container can access it directly via its link to the selenium container.

[Edit] I've found you don't need to explicitly expose the 8081 port of the django container either. Also, I used a range of ports for the test server, because if tests are run in parallel, you can get an "Address already in use" error, as discussed here.