Node.js HTTP Get URL Length limitation

2019-02-21 14:39发布

问题:

Is there a limit to the size when making HTTP GET requests in Node.js? And if so, how can I change this?

var url = "..." // very long, ~50'000 chars
http.get(url, function (res) {
    res.pipe(fs.createWriteStream("file.txt"));
});

Gives me this:

<html><body><h1>400 Bad request</h1>
Your browser sent an invalid request.
</body></html>

Same thing using wget in PowerShell or Bash works perfectly fine.

$url = "..."
wget -outf file.txt $url

回答1:

There is a built-in request size limit enforced by Node. Requested headers + URI should not be more than 80 kb.

As it is defined in http_parser.h#L55:

/* Maximium header size allowed */
#define HTTP_MAX_HEADER_SIZE (80*1024)

Asuming that UTF-8 character can be between 1 and 4 bytes, the size of a string with 50 000 characters would be from 50 000 to 200 000 bytes, or from ~48kb to 195kb.