Is there a limit to the size when making HTTP GET requests in Node.js? And if so, how can I change this?
var url = "..." // very long, ~50'000 chars
http.get(url, function (res) {
res.pipe(fs.createWriteStream("file.txt"));
});
Gives me this:
<html><body><h1>400 Bad request</h1>
Your browser sent an invalid request.
</body></html>
Same thing using wget in PowerShell or Bash works perfectly fine.
$url = "..."
wget -outf file.txt $url
There is a built-in request size limit enforced by Node. Requested headers + URI should not be more than 80 kb.
As it is defined in http_parser.h#L55:
Asuming that UTF-8 character can be between 1 and 4 bytes, the size of a string with 50 000 characters would be from 50 000 to 200 000 bytes, or from ~48kb to 195kb.