Would it help if you could see how Outlook, Internet Explorer or IIS works?
If I judge from posts in discussion forums and from mails that I receive, I am sure it would help. Many developers encounter problems with networking code that would be trivial to solve, if they could see how well-known applications implement the protocols.
The good news is that you can. It’s not even difficult: you can record the exchange as it goes over the network.
Debugging protocols
When working on a networked application, it is essential to review actual exchanges for two distinct but complementary applications:
- studying a well-known implementations of the protocol. When working with Internet protocols, often you will find another application that does what you want to do, by studying how it works, you’ll learn more about the protocol and how to implement it yourself;
- finding problems in your code. Servers and clients often return cryptic error messages. Only the actual exchange has the information you need to solve problems.
Tools of the trade
There are two solutions to listen to network traffic for debugging/learning purposes: a network sniffer or a proxy.
A network sniffer, such as the open source tcpdump package, is the most powerful of the two. As shown in figure 1, a network sniffer listens to the traffic at the lowest level as it is transmitted over the wire.
Figure 1: A network sniffer
A sniffer is versatile because it can record anything. It is also complex because it returns so much data that you will need to implement filters. Sniffers require administrative privileges and are prohibited on some corporate network. A friend of mine was fired for using one against the corporate policy; you have been warned.
Although sniffers are powerful, I tend to prefer proxies. Proxies are servers that listens to a port and redirect the traffic to another machine, see figure 2. For debugging purposes, you will want the proxy to record the traffic as well.
Figure 2: A debugging proxy
Filtering is built into the proxy design because the client connects to the proxy. Consequently proxies are easier to use. Furthermore coding a proxy is trivial: open two sockets and copy the data from one into the other. It is so easy that I often write custom proxies that will modify the traffic for testing purposes.
Using proxies: an example
To prepare my last article on JavaMail (JavaMail more efficiently), I had to review the exchange between a POP3 mail client and a server and duplicate the exchange with JavaMail.
I fired up a home-made proxy (which you can download) on my workstation configured to redirect all traffic from port 5001 to my regular mail server on port 110 (110 is the POP3 port). Next I went into the mail client configuration and changed the server to localhost:5001.
When I hit the “Get Mail” button, the client accessed my mailbox through the proxy so I could review the exchange. This gave insights into how a commercial mail client performs certain operations efficiently. The next step was to reproduce those with JavaMail.
To validate the JavaMail implementation, I again used the proxy. This time to record the exchange between JavaMail and the server and compare it with the commercial implementation.
Conclusion
I’m amazed that proxies and sniffers are not more commonly available in development tools. XMLSpy is the only IDE I know of that offers a debugging proxy (limited to SOAP). Don’t despair, it’s easy to hack your own.
When in doubt about any network code, fire a proxy or a sniffer and records the exchange. It’s a life-saver.
About the Author
Benoît Marchal is a Belgian writer and consultant. He is the author of XML by Example and other XML books.
He works mostly on e-commerce, XML and Java.
