Judging the quality of open source projects

It seems to me that people who want to use open source projects have a big problem. Several problems actually, but perhaps the most insidious of these is quality. How do you know if that really cool project you heard about is good quality design and code or is perhaps something that is going to be more trouble than it is worth in the long run?

For most open source projects, the quality is simply a guessing game. There is word of mouth, but you never know just how deeply someone has looked into a project. And what is more, perhaps they never found the bug that you will run into.

Perhaps the most important measure of quality for a project is the response of the community.

  • When a bug is posted, are people assigned and are they fixing it quickly?
  • Are questions responded to in a timely fashion?

If the community is active and supportive, you do get a sense that even if you do find a problem, someone is going to fix it and in a timely fashion.

What I want to see is the principles of test driven development being fully exploited in the community. This would involve a full set of unit tests that achieve substantial code coverage. Of course, code coverage doesn’t tell you everything you need to know, but it does tell you something.

My bottom line is that you need to know the quality of the project you want to use. An ideal community infrastructure would measure both the quality of the testing but also the quality of the team in terms of responsiveness.

I don’t know of anyone who has built a community like that yet, but I would sure love to see it.