Problem - Want to see your network traffic

Let's say you are running a web server on your local machine and it is communicating with another programme, or you are communicating with another web server. And something is going wrong. Your webserver isn't sending the right data. You are getting some strange problem reported somewhere. The first thing you should try to do is to look at what is being sent to/from your server. If this was a normal webbrowser / web server communication, you could just look at the HTML source. But you can't.

The Solution - use tcpflow!

There's a great programe tcpflow (ubuntu install link), that will capture all/some tcp traffic and put it in an easy to read file.

For example this command captures all traffic on port 8000 (the django debug server ip address) on your local machine

sudo tcpflow -i lo port 8000

You should start this command in an empty directory. It will create files of the format x.x.x.x.y-a.a.a.a.z (where x.x.x.x and a.a.a.a are the source/destination IP addresses and y and z are the source/destination port numbers). When you are done, just Control-C that command to stop it.

The port 8000 part is the standard wireshark/tcpdump expression

You have to run this as root (i.e. use use sudo ) because non-root users are not allowed to look at network traffic

Since I am running my django debug server locally, I use -i lo (a.k.a. Local interface). You can make it listen to traffic that goes to the wider internet with other choices of -i , e.g. -i eth0 .

Why not use a packet sniffer like wireshark?

Wireshark, being a proper packet sniffer, can do all of this and more. However tcpflow is easier for my cases since it's a level above wireshark. Wireshark shows you individual packets (and the TCP options and flags). tcpflow puts all the TCP packets into order and assembles them into the file, so you never see anything about the packets. Wireshark can assemble TCP packets into a linear stream, however this is easier to use.

In the case of HTTP traffic, you can see the Headers, then the content all in one file. This maeks it easy to copy and paste, email etc. the actual traffic that is being sent

If you use tcpflow (instead of wireshark), you can use all your standard unix text processing tools to work with the data. You don't have to use libpcap tools. This might be easier for what you want to do.