How to capture all the HTTP packets using tcpdump

Question

I want to run tcpdump with some parameters (still don't know what to use), then load the stackoverflow.com page.

Output should be the HTTP communication. Later, I want to use it as a shell script, so whenever I want to check the HTTP communication of a site site.com, I just can run script.sh site.com.

The HTTP communication should be simple enough. Like this:

GET /questions/9241391/how-to-capture-all-the-http-communication-data-using-tcp-dump
Host: stackoverflow.com
... 
...

HTTP/1.1 200 OK
Cache-Control: public, max-age=60
Content-Length: 35061
Content-Type: text/html; charset=utf-8
Expires: Sat, 11 Feb 2012 15:36:46 GMT
Last-Modified: Sat, 11 Feb 2012 15:35:46 GMT
Vary: *
Date: Sat, 11 Feb 2012 15:35:45 GMT


....
decoded deflated data
....

Now, which options should I use with tcpdump to capture it?

Answer 1

It can be done by ngrep

ngrep -q -d eth1 -W byline host stackoverflow.com and port 80 
       ^  ^       ^         ^        
       |  |       |         |
       |  |       |         |
       |  |       |         v
       |  |       |         filter expression
       |  |       |         
       |  |       +-->  -W  is set the dump format ("normal", "byline", "single", "none")
       |  |
       |  +---------->  -d  is use specified device instead of the pcap default
       |
       +------------->  -q  is be quiet ("don't print packet reception hash marks")

Answer 2

Based on what you have mentioned, ngrep (on Unix) and Fiddler (Windows) might be better/easier solutions.

If you absolutely want to use tcpdump, try out the following options

tcpdump -A -vvv host destination_hostname

-A (ascii)
-vvv (verbose output)

Answer 3

tcpdump -i eth0 -w dump3.pcap -v  'tcp and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'

see http://www.tcpdump.org/manpages/tcpdump.1.html