Complete reconstruction of TCP Session (HTML pages

Question

I wonder if there is a way in wireshark to reconstruct a complete TCP Session (HTML page(s)) if we have wireshark pcaps, can wireshark do the reconstruction? or is there any tool around that can do the reconstruction? Data streamed from a source could be compressed(Gzip) or uncompressed and the end result of reconstruction should be a valid complete HTML page with all of its contents.

Answer 1

Depending on what version of Wireshark you have, you should be able to do something along the lines of:

1. Filter out the session you care about
2. Do File->Export->Objects->Http
3. Select a folder.

Is there something more you need... this appears to do the gzip decompression etc... won't work if you're running SSL (it MIGHT be able to if you can get the appropiate keys to make the SSL decode work, but that gets trickier and I'd suggest trying fiddler in that case)

HTH

Answer 2

You can also use Bro if you prefer a command-line interface. Simply load it with the contents script:

bro -r trace.pcap -f 'port 80' contents

(You can skip the optional BPF filter expression -f port 80.) This extracts the full TCP stream and writes it to files of the form:

contents.<sourceIP>.<sourcePORT>-<destinationIP>.<destinationPORT>

As Christian mentioned, the reassembly is highly robust and has been tested thoroughly.

Answer 3

TCPTrace has an option (-e) for this:

Extracting: The -e option can be used to extract the contents (TCP data payload) of each connection into a separate data file.

For example,

Beluga:/Users/mani> tcptrace -e albus.dmp

generates files a2b_contents.dat, b2a_contents.dat; c2d_contents.dat, d2c_contents.dat if the file albus.dmp had 2 traced TCP connections. tcptrace is pretty smart in generating these contents files. It does not commit trivial mistakes like saving retransmissions multiple times in the file for example, and is aware of sequence space wrap-arounds. However, if you want the entire contents of the traffic, please make sure that packets are captured in their entirety (give suitable snaplen value with tcpdump for example).

Answer 4

I suggest tcpflow, a full-featured tcp/ip session reconstructor. It is very fast, will handle very large sessions, automatically decompresses gzip'ed connections, automatically breaks out MIME objects sent by HTTP, creates an XML file of what it's done, runs on MacOS, Linux and Windows, and more. It's a command-line tool.

Answer 5

Use justniffer-grab-http-traffic .It is based on justniffer and it is an excellent tool for rebuilding tcp streams.