Open-source versus proprietary software Is one more reliable and secure than the other?

IBM Systems Journal,  June, 2005  by A. Boulanger

• E-mail
• Print
• Link

Additional information comes from companies that develop automated software-inspection services. Some background will help explain the role of these companies in the software industry. In large projects the people responsible for maintaining a system are often not the same people who originally developed the system. Unless the maintainers are careful and fully understand the system, it becomes very easy to make a mistake that can affect the overall quality of the system code. One of the ways to increase the reliability of a system is to review the source code for defects and remedy them before the system is released. Typically the inspection process is performed through formal code reviews and evaluations. This process is very labor-intensive and time-consuming. Historically, as systems grew and it became more expensive to perform formalized code reviews, researchers developed ways to automate this process and make code reviews less labor-intensive than manual inspection. Several companies now offer automated software-inspection services, allowing software publishers to outsource their code reviews. One such company, Reasoning, Inc., has been assisting organizations to improve the quality of their systems through automated software inspection for almost 20 years and is considered a leader in this field.

In 2003, Reasoning conducted a study of the implementation of the Internet protocol code in the 2.4.19 version of the Linux kernel and in five proprietary operating systems. (15) The purpose of the study was to use automated code inspection techniques to compare the quality and defect rate of each implementation of the TCP/IP (Transmission Control Protocol/Internet Protocol) net working software. Reasoning discovered that the defect rate for the Linux code was 0.1 reported defects per 1000 lines of code (KLOC). The defect rate for proprietary implementations was reported to be 0.55 defects per KLOC. Reasoning concluded that the FOSS implementation of TCP/IP had a significantly lower defect density compared to the implementations in the five proprietary operating systems. The study also concluded that the overall quality of the FOSS package rated in the top third of all source-code projects that had been inspected by Reasoning.

In July of 2003, Reasoning analyzed the popular Apache Web server software package. (16) The Apache Web server is a FOSS system developed and maintained by the Apache Software Foundation, a membership-based not-for-profit corporation. (5) The Apache server is the dominant HTTP (Hypertext Transfer Protocol) server package on the Internet today, according to a recent survey by Netcraft. (17) This survey, conducted in June 2004, reported that of the 51.6 million identifiable servers on the Internet at that time, Apache had over 67 percent of the market, followed by Microsoft with a 21 percent market share. With so many organizations relying on FOSS technology for their Internet presence, it would obviously be valuable for IT managers to have a vendor-neutral software-quality metric to assist in a decision whether to deploy FOSS or proprietary systems. Reasoning concluded in their study that the defect density for the 2.1 release of the Apache system was 0.53 defects per KLOC. To put that figure into perspective, Reasoning compared the defect density of the Apache system to the 200 other projects Reasoning had analyzed at that time, both FOSS and proprietary, involving a total of 33 million analyzed lines of code. The top third of these 200 projects showed defect densities of less than 0.36 defects per KLOC; defect densities of the middle third ranged from 0.36 to 0.71 defects per KLOC; the bottom third had defect densities greater than 0.71 defects per KLOC. Given these statistics, the defect rate for the Apache system falls somewhere in the middle compared to the rest of the industry and slightly above the average defect density Reasoning has round for proprietary software (0.51 defects per KLOC).