Getting error 502 when processing an excel file wi

Question

Getting error 502 when processing an excel file with many rows.

Using Django / Nginx

The problem is not the weight of the file is less than 1Mb.

This page works correctly with files of 200 rows, the problem starts when the file have more rows and then, the page take too long processing this file.

This is the error:

2012/07/28 14:29:54 [error] 18515#0: *34 upstream prematurely closed connection while reading response header from upstream, client: 127.0.0.1, server: localhost, request: "POST /import/ HTTP/1.1", upstream: "http://127.0.0.1:9000/import/", host: "localhost:8080", referrer: "http://localhost:8080/import/"

I am using very large values ​​for the variables, but I keep getting the same error.

This is the configuration of the site:

---

upstream app_server {
    server 127.0.0.1:9000 fail_timeout=3600s;
    keepalive 3600s;
}

server {
    listen 8080;
    client_max_body_size 4G;
    server_name localhost;

    keepalive_timeout           3600s;
    client_header_timeout       3600s;
    client_body_timeout         3600s;
    send_timeout                3600s;

    location /static/ {
        root  /my path/;
        autoindex on;
        expires 7d;
    }

    location /media/ {
        root  /my path/;
        autoindex on;
        expires 7d;
    }

    location / {
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $http_host;
        proxy_redirect off;  

        proxy_connect_timeout       3600s;
        proxy_send_timeout          3600s;
        proxy_read_timeout          3600s;      

        if (!-f $request_filename) {
            proxy_pass http://app_server;
            break;
        }       
    }
}

---

And this is the global configuration:

---

user  www-data;
worker_processes  1;

error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include     /etc/nginx/mime.types;
    access_log  /var/log/nginx/access.log;
    sendfile        on;
    keepalive_timeout  3600s;
    tcp_nodelay        on;

    client_header_timeout       3600s;
    client_body_timeout         3600s;
    send_timeout                3600s;
    proxy_connect_timeout       3600s;
    proxy_send_timeout          3600s;
    proxy_read_timeout          3600s; 

    client_max_body_size 200m;
    client_body_buffer_size 128k;

    gzip  on;
    gzip_disable "MSIE [1-6]\.(?!.*SV1)";

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

---

Can you give me some help?

Best regards

Answer 1

The best option is rewrite the routine to use django-celery but if you want a quick solution you could try upgrading the timeout for your proxy pass in Nginx by adding:

proxy_connect_timeout 300s;
proxy_read_timeout 300s;

You should add this config on /var/nginx/sites-available/[site-config] to a specific site or /var/nginx/nginx.conf if you want to increase the timeout on all sites served by nginx.

If you are using gunicorn, You must add --timeout=300 as well. Example:

gunicorn_django -D -b 127.0.0.1:8901 --workers=2 --pid=/var/webapp/campus.pid --settings=settings.production --timeout 300 --pythonpath=/var/webapp/campus/

References:

• http://wiki.nginx.org/HttpProxyModule#proxy_connect_timeout

• http://reinout.vanrees.org/weblog/2011/11/24/apache-nginx-gunicorn-timeout.html

• Gunicorn Nginx timeout problem