Can't reach Django default app via Gunicorn on

Question

I've been struggling with this problem for two days without success. I've created an instance of the default Django (1.6.1) app called "testdj", installed it on an Amazon AWS EC2 t1.micro instance running Ubuntu Server 13.10, and I'm trying to reach the default Django "It worked!" page via Gunicorn (v. 18). When I start gunicorn from the command line:

gunicorn testdj.wsgi:application --bind [ec2-public-dns]:8001

I can see the page when I enter this URL:

http://[ec2-public-dns]:8001

However, if I use a "start-gunicorn" bash script I created after reading Karzynski's blogpost "Setting Up Django with Nginx, Gunicorn, virtualenv, supervisor, and PostgreSQL", I always get an error. When I enter this URL...

http://[ec2-public-dns]

... I get this error:

Error 502 - Bad Request
The server could not resolve your request for uri: http://[ec2-public-dns]

Here is the start-gunicorn script:

#!/bin/bash

NAME="testdj"
DJANGODIR=/usr/share/nginx/html/testdj
SOCKFILE=/usr/share/nginx/html/testdj/run/gunicorn.sock
USER=testdj
GROUP=testdj
NUM_WORKERS=3
DJANGO_SETTINGS_MODULE=testdj.settings
DJANGO_WSGI_MODULE=testdj.wsgi

WORKON_HOME=/home/testdj/venv
source `which virtualenvwrapper.sh`
workon $NAME
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGO_DIR:$PYTHONPATH

RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR

exec gunicorn ${DJANGO_WSGI_MODULE}:application \
  --name $NAME \
  --workers $NUM_WORKERS \
  --user=$USER --group=$GROUP \
  --access-logfile /tmp/gunicorn-access.log \
  --error-logfile /tmp/gunicorn-error.log \
  --log-level=debug \
  --bind=unix:$SOCKFILE

As you can see, I've created a special account on my server called "testdj" to run the app under. I'm running my Django app in a virtual environment. I haven't changed the Django wsgi.py file at all. As I eventually want to use nginx as my reverse proxy, I've installed nginx and put the Django app in nginx's default root directory /usr/share/nginx/html. User/group www-data owns /usr/share/nginx and everything below except that user/group "testdj" owns /usr/share/nginx/html/testdj and everything below it. /usr/share/nginx/html/testdj and all its subdirectories have perms 775 and I've added www-data to the testdj group.

I do have nginx installed but I don't have the nginx service running. I did try starting it up and enabling an nginx virtual server using the following configuration file but the error still occurred.

upstream testdj_app_server {
    server unix:/usr/share/nginx/html/testdj/run/gunicorn.sock fail_timeout=0;
}
server {
    listen 80;
    server_name ec2-[my-public-dns-ip].us-west-2.compute.amazonaws.com;
    client_max_body_size 4G;
    access_log /var/log/nginx/testdj-access.log;
    error_log /var/log/nginx/testdj-error.log;

    location /static/ {
        alias /usr/share/nginx/html/testdj/static/;
    }
    location /media/ {
        alias /usr/share/nginx/html/testdj/media/;
    }
    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect off;   
        if (!-f $request_filename) {
            # This must match "upstream" directive above
            proxy_pass http://testdj_app_server;
            break;
        }
    }
    error_page 500 502 503 504 /500.html;
    location = /500.html {
        root /usr/share/nginx/html/testdj/static/;
    }
}

The problem seems to be with gunicorn because if I replace the "--bind=unix:$SOCKFILE" in my start-gunicorn script with "--bind --[ec2-public-dns]:8000", I can see the Django default page. However, I don't want to bind to my public DNS name on port 8000, I want to run on port 80 and use nginx as my front end reverse proxy.

I initially had AWS inbound security group rules that limited access to the site to HTTP on ports 80, 8000, and 8001 to my laptop but even if I delete these rules and leave the site wide open, I still get the 502 error message.

My gunicorn access log doesn't show any activity and the only thing I see in the gunicorn error log is gunicorn starting up. When I access the default Django page, there are no errors in the error log:

2014-02-03 18:41:01 [19023] [INFO] Starting gunicorn 18.0
2014-02-03 18:41:01 [19023] [DEBUG] Arbiter booted
2014-02-03 18:41:01 [19023] [INFO] Listening at: unix:/usr/share/nginx/html/testdj/run/gunicorn.sock (19023)
2014-02-03 18:41:01 [19023] [INFO] Using worker: sync
2014-02-03 18:41:01 [19068] [INFO] Booting worker with pid: 19068
2014-02-03 18:41:01 [19069] [INFO] Booting worker with pid: 19069
2014-02-03 18:41:01 [19070] [INFO] Booting worker with pid: 19070

Does anyone know what's happening here? It doesn't look like I'm even getting to gunicorn. I apologize for the long post but there seems to be a lot of "moving parts" to this problem. I'd be very grateful for any help as I've tried many different things but all to no avail. I've also looked at other questions here where other people had similar problems but I didn't see anything pertinent to this problem. Thanks!

Answer 1

I repeated my configuration process line-for-line on my Linode server and had no problems at all. I have to assume that this problem has something to do with the way AWS EC2 instances are configured, probably with respect to security.

Answer 2

I was running into the same problem today. Daniel Roseman explained it to me in the comments here:

port 8000 is not open to the outside by default; you'd either need to fiddle with your load balancer/firewall settings to open it, or run gunicorn on port 80 (which will mean killing nginx and starting gunicorn as the superuser). Much easier to just get the nginx settings right; there is a perfectly usable configuration on the gunicorn deploy docs page.

It seems it's possible to run Gunicorn directly, but EC2 isn't set up to do so by default.