Three Unexpected Results in Empirical Open-Source Software Engineering

On Thursday Tyson Dowd attended a talk by Stephen R Schach from Vanderbilt University.  He talked at UTS in Sydney about some empirical research into OSS

Slides for this talk are at:https://www.vuse.vanderbilt.edu/~srs/three.unexpected.ppt

The results were very interesting.

Unexpected result 1: Linus’s Law is not applicable to open-source software development

Linus’s Law: As formulated by Raymond [2000] in his essay The Cathedral and the Bazaar

4     “Given enough eyeballs, all bugs are shallow” 

In other word

4     If there are enough individuals examining an open-source software product, for each fault there should be someone who is able to locate that fault and explain how to fix it. 

If there is a core group of developers and a vast periphery of users who are supposed to fix bugs because they have access to the source code, we should expect to see the majority of bugs fixed by the periphery, not the core.

This is not the case at all:

The major role the periphery plays is finding and reporting bugs, much like in closed source software.  Most of the fixes they suggest are rejected and the core provides a “correct” fix.

Also as an extra result, enhancements come from the core, not the periphery – and decisions regarding which enhancements are made come from the core.  Core enhancements outnumber periphery 17:1.

Unexpected result 2: Linux is unmaintainable

Common coupling = shared global variables = bad software engineering.  It leads to unmaintanable code because the coupling introduces dependencies of otherwise unrelated modules.

Results:

• The number of instances of common coupling grows exponentially with version number
• The number of lines of code grows linearly with version number
• Conclusion: As Linux grows in size, it will become unmaintainable

Linux was found to have > 10 times the couplings (and all of them are bad, just that some are much worse than others) than OpenBSD, FreeBSD and NetBSD.

When presented for the first time these results caused a riot amongst Linux supporters – a riot largely aimed at the presenter.

A more accurate study was done, identifying definitions and uses (lvals and rvals for compiler lovers) of variables.  It turns out that most of the couplings are of the worst variety – non-kernel modules that modify global variables used and modified by kernel modules.  This means every optional module could conceivably break the kernel.  It makes quality control extremely difficult and means configuration testing becomes very hard (can’t test each module individually).   The results make seasoned software engineers gasp.

Unexpected result 3: The LST result is false

By examining bugzilla databases they find that the popular Lientz, Swanson and Tompkins result that 17.4% of maintenance was corrective, 18.2% was adaptive, 60.3% was perfective and 4.1% was other.  They authors assert that this result is wrong, chances are the people surveyed had no data (the study was from 1978 – few companies even had data) so they made up something that sounded nice.

See the slides for more info on this at: https://www.vuse.vanderbilt.edu/~srs/three.unexpected.ppt