Open-source bug hunt results posted

GCN.com ^ | 3/06/06 | Joab Jackson, GCN Staff

[N3WBI3]

Coverity Inc. of San Francisco has released the results of a Homeland Security Department-funded bug hunt that ranged across 40 popular open-source programs. The company found less than one-half of one bug per thousand lines of code on average, and found even fewer defects in the most widely used code, such as the Linux kernel and the Apache Web server.

...

Most of the 40 programs tested averaged less than one defect per thousand lines of code. The cleanest program was XMMS, a Unix-based multimedia application. It had only six bugs in its 116,899 lines of code, or .51 bugs per thousands lines of code. ...

Overall, the average defect density of all the programs was .43 bugs per thousand lines of code. The most widely used programs scored well under this average. The 3 million lines of code that make up the Linux Kernel had an average of .33 bugs per thousand lines of code. Apache has .25 bugs per thousand lines of code. The open-source LAMP stack (consisting of Linux, Apache, MySQL and a scripting language of either Perl, PHP or Python), had a defect density of .29 bugs per thousand lines of code.

...

The maintainers of the source codes can register with Coverity to see the full results. (End users cannot see the bug lists themselves; they will be able to see how buggy a particular program may be.)

Bruce Momjian, who oversees development of PostgreSQL has used Coverity reports before and has found them useful, if not absolutely essential. The results of a previous study pointed to “a few unusual cases that weren't exploitable bugs, but were something we wanted to clean up,” he said.

...

[N3WBI3]

Interesting albeit it light read..

[N3WBI3]

OSS PING

[zeugma]

I've seen some articles about their methods that aren't entirely flattering. It's an interesting report, in that their reports point up code that can be cleaned up a bit. I don't know if Coventry has found any exploitable code before. Wouldn't me particularly suprised if they had, but I can't recall offhand if that is the case.

 You might be interested in the following Link sent to me by a co-worker today regarding "ESX Server Modified Source" at VMware considering the discussion we had about this a month or so ago.

I haven't had a chance to look at exactly what it includes closely, especially in light of a page I'd found on their site after much digging that stated the ESX kernel was something of their own making, rather than being a Linux derivative. Perhaps it's just a kernel tuned for running in a VMWare container or something. Just thought I'd pass it along.

[ThePythonicCow]

Coverity catches some bugs that automatic analysis can get, from tracking data and code flow. Some of its complaints aren't bugs at all, just the result of code flow that is, perhaps intentionally in the case of some performance critical code paths, too obscure for it to track.

Almost none of the really interesting, or really dangerous, bugs are caught by it, and while it reports alot of bugs (kind of a nuiscance) few of them are more than mildly interesting.

Unlike 'real' bugs that start with a symptom - such as the system generates an error if such-and-such is done, Coverity bugs start with the specific code complaint, such as this variable doesn't seem to be initialized before use on this code path. This makes Coverity bugs less useful, because one can't see what, if any, impact that alledged bug has on actual system behaviour, and so can't really tell what is the severity of the bug or the impact of the change.

[Golden Eagle]

Why is the government spending our tax dollars on this?

Isn't the open source community supposed to be doing this on their own?

You always claim they are. Next you'll be backing Richard Stallman's call for a tax increase to pay for this. If you aren't already.

[N3WBI3]

"As more and more business transactions take place on the Internet, governments and businesses must cooperate to find innovative ways to regulate and derive tax revenue from Internet commerce without interfering with the economic benefits it can provide." -- Bill Gates

[GladesGuru]

“governments and businesses must cooperate to find innovative ways to regulate and derive tax revenue from Internet commerce”.

How about reducing the size/powers of agencies to their original Constitutional bounds?

Tax monies are the essential nutrient for government. Cut the budgets and the bureaucrats WILL go away.

[N3WBI3]

Glades,

Im all for that I was pointing out to a local troll who give MS a pass on such things a reality check.

I think bug hunts in open and closed source code by the government is appropriate because, well, they use it. The LAMP stack is one of the most used platforms for websites in the world and that has to include com government work. I also think the Government should be doing this for closed source software they use.