March 1, 2021
Hot Topics:

The myth of open source security

  • By John Viega
  • Send Email »
  • More Articles »

"The speed with which the TCP wrappers Trojan was discovered in early 1999 can lull the open source movement into a false sense of security. "
Even if you get the right kinds of people doing the right kinds of things, you may have problems you never hear about. Security problems are often incredibly subtle and may span large parts of a source tree. It is not uncommon to have two or three features spread throughout a program, none of which constitutes a security problem alone, but which can be used together to perform a security breach. For example, two buffer overflows recently found in Kerberos version 5 could only be exploited when used in conjunction with each other.

As a result, doing security reviews of source code tends to be complex and boring, since you generally have to look at a lot of code and understand it pretty well. Even many experts don't like to do these kinds of reviews.

And even the experts can miss things. Consider the case of the popular open source FTP server wu-ftpd. In the past two years, several very subtle buffer overflow problems have been found in the code. Almost all of these problems had been in the code for years, despite the fact that the program had been examined many times by both hackers and security auditors. If any of them discovered the problems, they didn't announce it publicly. In fact, the wu-ftpd has been used as a case study for vulnerability detection techniques that never identified these problems as definite flaws. One tool was able to identify one of the problems as potentially exploitable, but researchers examined the code thoroughly for a couple of days, and came to the conclusion that there was no way that the problem identified by their tool could actually be exploited. Over a year later, they learned that they were wrong, when an expert audit finally did turn up the problem.

In code with any reasonable complexity, it can be very difficult to find bugs. The wu-ftpd is less than 8,000 lines of code long, but it was easy for several bugs to remain hidden in that small space over long periods of time.

To compound the problem, even when people know about security holes, they may not get fixed, at least not right away. Even when identified, the security problems in Mailman took many months to fix, because security was not the the core development team's most immediate concern. In fact, the team believes one problem still persists in the code, but only in a configuration that we suspect doesn't get used.

An army in my belly

The single most pernicious problem in computer security today is the buffer overflow. While the availability of source code has clearly reduced the number of buffer overflow problems in open source programs, according to several sources, including CERT, buffer overflows still account for at least a quarter of all security advisories, year after year.

Open source proponents sometimes claim that the many-eyeballs phenomenon prevents Trojan horses from being introduced in open source software. The speed with which the TCP wrappers Trojan was discovered in early 1999 is sometimes cited as supporting evidence. This too can lull the open source movement into a false sense of security, however, since the TCP wrappers Trojan is not a good example of a truly stealthy Trojan horse: the code was glaringly out of place and obviously put there for malicious purposes only. It was as if the original Trojan horse had been wheeled into Troy with a sign attached that said, "I've got an army in my belly!"

Well-crafted Trojans are quite different. They generally look like ordinary bugs with security implications, and they are very subtle. Take, for example, wu-ftpd. Who is to say that one of the buffer overflows found recently was not a Trojan horse introduced years ago when the distribution site was hacked?

The open source movement hasn't made the problem of buffer overflows go away. But eventually, newer programming languages may; unlike C, modern programming languages like Java or Python never have buffer overflow problems, because they do automatic bounds checking on array accesses. As with any technology, fixing the root of the problem is far more effective than any ad hoc solution.

Is closed source any more secure?

Critics of open source software might say that providing source code makes the job of the malicious attacker easier. If only a binary is available, the bar has been raised high enough to send most such people looking for lower-hanging fruit. But as the many well-publicized security holes in commercial software make clear, attackers can find problems without the source code; it just takes longer. From a security point of view, the advantages of having the source code available for everyone to see far outweighs any benefit hackers may gain.

There are many benefits of open source software unrelated to security. And the many-eyeballs effect does have the potential to make open source software more secure than proprietary systems. Currently, however, the benefits open source provides in terms of security are vastly overrated, because there isn't as much high-quality auditing as people believe, and because many security problems are much more difficult to find than people realize. Open source programs which appeal to a limited audience are particularly at risk, because of the smaller number of eyeballs looking at the code. But all open source software is vulnerable, and the open source movement can only benefit by paying more attention to security. //


1. The Mailman Web site
2. The ITS4 security scanner for C code
3. Software security for developers
4. The CERT Web site

John Viega is a Senior Research Associate in the Software Security Group at Reliable Software Technologies, and an Adjunct Professor of Computer Science at the Virginia Polytechnic Institute. He is the author of Mailman, the open source GNU Mailing List Manager, and ITS4, a tool for finding security vulnerabilities in C and C++ code. He has authored over 30 technical publications in the areas of software security and testing, and is responsible for finding several well-publicized security vulnerabilities in major network and e-commerce products, including a recent break in Netscape's security.

Page 2 of 2

This article was originally published on June 1, 2000

Enterprise Development Update

Don't miss an article. Subscribe to our newsletter below.

Thanks for your registration, follow us on our social networks to keep up-to-date