Nginx uwsgi (104: Connection reset by peer) while

问题:

Environment is Nginx + uwsgi.

Getting a 502 bad gateway error from Nginx on certain GET requests. Seems to be related to the length of the URL. In our particular case, it was a long list of GET parameters. Shorten the GET parameters and no 502 error.

From the nginx/error.log

[error] 22113#0: *1 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 192.168.1.100, server: server.domain.com, request: "GET <long_url_here>"

No information in the uwsgi error log.

回答1:

After spending a lot of time on this, I finally figured it out. There are many references to Nginx and connection reset by peer. Most of them seemed to be related to PHP. I couldn't find an answer that was specific to Nginx and uwsgi.

I finally found a reference to fastcgi and a 502 bad gateway error (https://support.plesk.com/hc/en-us/articles/213903705). That lead me to look for a buffer size limit in the uwsgi configuration which exists as buffer-size. The default value is 4096. From the documentation, it says:

If you plan to receive big requests with lots of headers you can increase this value up to 64k (65535).

There are many ways to configure uwsgi, I happen to use a .ini file. So in my .ini file I tried:

buffer-size=65535

This fixed the problem. You can adjust that to taste. Maybe start with the max and work back until you have an acceptable value, or just leave it at the max.

This was frustrating to track down because there was no error on the uwsgi side of things.

回答2:

I was getting the same nginx error and also there was no information in uwsgi log. The problem was that in some cases the application was not consuming the whole request body as advised in http://uwsgi-docs.readthedocs.org/en/latest/ThingsToKnow.html:

If an HTTP request has a body (like a POST request generated by a form), you have to read (consume) it in your application. If you do not do this, the communication socket with your webserver may be clobbered. If you are lazy you can use the post-buffering option that will automatically read data for you. For Rack applications this is automatically enabled.

Of course, this is not a problem in your case, but it may be useful for others who are getting the same nginx error.

回答3:

Thanks, the same solution is adopted in the case of a PHP server. We need just to increase the attribute "output_buffering" value, in php.ini, to a greater value like 65535 or another appropriate value.

回答4:

When we receive a message like (104: Connection reset by peer) while reading response header from upstream, most often, we could blame the upstream side of this kind of error.

As described, the connection was reset by the upstream peer, not by nginx itself. Nginx as a client can barely do anything to make it right.

I'm suspecting if modifying buffer-size will do the magic. Basically the command changes the buffer size where response headers are cached. This would take effect when the response header is too big, of which case we receive a message saying upstream sent too big header while reading response header from upstream, and that is totally different thing from connection reset by peer.

Since this kind of error is trigger randomly, I would suggest you check whether nginx uses keepalive when talking to upstreams. If this was the case, the connection might be reset by upstream server when the idle timed out whereas nginx had no idea that the connection had been dropped, hence forwarding the request using the same connection.

There's no elegant solution to fix it as far as I know. You could do retry or set a keepalive_timeout value to the upstream connection pool in nginx to avoid the problem.

referencing:

Apache HttpClient Interim Error: NoHttpResponseException

http://tengine.taobao.org/document/http_upstream_keepalive_timeout.html

回答5:

--post-buffering 32768 worked for me as suggested (and discouraged) here NGINX + uWSGI Connection Reset by Peer

I don't have time to investigate it further at the moment (quick prototyping mode :), but since it took me a lot of time to find this hack, it might be worth posting here.

回答6:

You need to re-install PHP:

apt-get install --reinstall php5-fpm