How to make an HTTP GET request manually with netc

问题:

So, I have to retrieve temperature from any one of the cities from http://www.rssweather.com/dir/Asia/India.

Let's assume I want to retrieve of Kanpur's.

How to make an HTTP GET request with Netcat?

I'm doing something like this.

nc -v rssweather.com 80
GET http://www.rssweather.com/wx/in/kanpur/wx.php HTTP/1.1

I don't know exactly if I'm even in the right direction or not. I am not able to find any good tutorials on how to make an HTTP get request with netcat, so I'm posting it on here.

回答1:

Of course you could dig in standards searched for google, but actually if you want to get only a single URL, it doesn't worth the effort.

You could also start a netcat in listening mode on a port:

nc -l 64738

...and then do a browser request into this port with a real browser. Just type in your browser http://localhost:64738 and see.

In your actual case the problem is that HTTP/1.1 doesn't close the connection automatically, but it waits your next URL you want to retrieve. The solution is simple:

Use HTTP/1.0:

GET /this/url/you/want/to/get HTTP/1.0
Host: www.rssweather.com
<empty line>

or use a Connection: request header to say the server you want to close after that:

GET /this/url/you/want/to/get HTTP/1.1
Host: www.rssweather.com
Connection: close
<empty line>

Extension: After the GET header write only the path part of the request. The hostname from which you want to get data belongs to a Host: header as you can see in my examples. This is because multiple websites can run on the same webserver, so the browsers need to say him, from which server they want to load the page.

回答2:

This works for me:

$ nc www.rssweather.com 80
GET /wx/in/kanpur/wx.php HTTP/1.0
Host: www.rssweather.com

And then hit double <enter>, i.e. once for the remote http server and once for the nc command.

source: pentesterlabs

回答3:

On MacOS, you need the -c flag as follows:

Little-Net:~ minfrin$ nc -c rssweather.com 80
GET /wx/in/kanpur/wx.php HTTP/1.1
Host: rssweather.com
Connection: close
[empty line]

The response then appears as follows:

HTTP/1.1 200 OK
Date: Thu, 23 Aug 2018 13:20:49 GMT
Server: Apache
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html

The -c flag is described as "Send CRLF as line-ending".

To be HTTP/1.1 compliant, you need the Host header, as well as the "Connection: close" if you want to disable keepalive.

回答4:

Test it out locally with python3 http.server

This is also a fun way to test it out. On one shell, launch a local file server:

python3 -m http.server 8000

Then on the second shell, make a request:

printf 'GET / HTTP/1.1\r\nHost: localhost\r\n\r\n' | nc localhost 8000

The Host: header is required in HTTP 1.1.

This shows an HTML listing of the directory, just as you would see from:

firefox http://localhost:8000

Next you can try to list files and directories and observe the response:

printf 'GET /my-subdir/ HTTP/1.1\n\n' | nc localhost 8000
printf 'GET /my-file HTTP/1.1\n\n' | nc localhost 8000

Every time you make a successful request, the server prints:

127.0.0.1 - - [05/Oct/2018 11:20:55] "GET / HTTP/1.1" 200 -

confirming that it was received.

example.com

This IANA maintained domain is another good test URL:

printf 'GET / HTTP/1.1\r\nHost: example.com\r\n\r\n' | nc example.com 80

and compare with: http://example.com/

https SSL

nc does not seem to be able to handle https URLs. Instead, you can use:

sudo apt-get install nmap
printf 'GET / HTTP/1.1\r\nHost: github.com\r\n\r\n' | ncat --ssl github.com 443

See also: https://serverfault.com/questions/102032/connecting-to-https-with-netcat-nc/650189#650189

If you try nc, it just hangs:

printf 'GET / HTTP/1.1\r\nHost: github.com\r\n\r\n' | nc github.com 443

and trying port 80:

printf 'GET / HTTP/1.1\r\nHost: github.com\r\n\r\n' | nc github.com 443

just gives a redirect response to the https version:

HTTP/1.1 301 Moved Permanently
Content-Length: 0
Location: https://github.com/
Connection: keep-alive

Tested on Ubuntu 18.04.

回答5:

You don't even need to use/install netcat

• Create a tcp socket via an unused file-descriptor i.e I use 88 here
• Write the request into it

• use the fd

  exec 88<>/dev/tcp/rssweather.com/80
  echo -e "GET /dir/Asia/India HTTP/1.1\nhost: www.rssweather.com\nConnection: close\n\n" >&88
  sed 's/<[^>]*>/ /g' <&88