TCO : Visual Studio Team System or Open source ?
/ Disclaimer: remove the emotion while reading this post and be as cold in the analysis as an excel spreadsheet can be /
/ Disclaimer #2: TCO include more than just the cost of the solution. Keep in mind that 80 to 85% of a devteam cost is based on people (salary plus full cost for a position). So anything that can increase the productivity will positively impact your activity /
/ Disclaimer #3: Decision are sometimes based on risk and perception and not only on rational facts /
/ Disclaimer #4 : the following post is only investigating the cost dimension /
You know that I sort of like ROI and TCO.
Let’s give you a quick update on a case I have spent some time on
Company A is looking at Team Foundation Server (TFS) or a mix of Open source solutions (Subversion, Jira...) for structuring their development efforts
At this stage, I’m not even talking about software quality. So I’ll add this important topic at the end.
Company A got 2 teams and 10 developers in total.
We spent some time on the benefits of TFS and they agreed that TFS is offering the same level of functionality (if not more) than a mix a “best of breed” open source solutions
So, still without the quality tools, what is the effort to install and use TFS versus Open source
TFS solution (cost is in AU$) over 3 years
· Cost of Team Foundation Server + 10 CALs (client Access Licenses): $ 10 000 (including the maintenance/SA)
· Installation and personalisation: 5 to 10 days, let say 7.
o Average cost of a developer in NSW : $ 1000 per day
o So effort for setting up TFS is $ 7k
· TFS will be updated once during the 3 years: 4 days -> $ 4k
· Administration : 3 days per year -> $ 9k over 3 years
· Total cost
o Over 3 years : $ 30 000
o Annual cost : $ 10 000
Open Source solution over 3 years
· Acquisition cost: $ 0
· Source control/versioning installation and personalisation : 3 days -> $ 3k
· Project and issue management - installation and personalisation : 3 days -> $ 3k
· Continuous integration – installation and personalisation : 2 days -> $ 2k
· Source control and project management integration: 5 days -> $ 5k
· Continuous integration and Source control integration : 10 days -> $ 10k
· continuous integration and Project/issue management integration: 7 days -> $ 7k
· Build the reports on top of the integration: 9 days -> $ 9k
· update of Source control solution over the 3 years – installation and personalisation : 5 days -< $ 5k
· update of Project and issue management: 5 days -> $ 5k
· update of continuous integration solution over the 3 years – installation and personalisation: 3 days -> $ 3k
· update of the integration between Source control / project management / continuous integration after the update : 6 days -> $ 6k
· update of the reports: 5 days -> $ 5k
· administration over the 3 years : 20 days -> $ 20 k
· Total cost
o Over 3 years: $ 83 000
o Annual cost: $ 27 000
Now, what about testing integration?
I am maybe repeating myself when I say that testing itself is useless. You need to know what to test and when to ensure that you are measuring correctly the level of quality. Testing will not improve the quality but reveal your current level.
And if you are turning it the other way, source control + project management + continous integration is useless too! What’s the point to have a crystal clear visibility on the status of the project if you can’t trust what you are tracking? How could you be sure that you are on time and that the job was done properly if you don’t have the testing results?
PREDICATBILITY = VISBILITY + QUALITY checks
So, if we are back to the TFS versus Open source debate, we need to add the testing tools and the integration
TFS + Team Suite
· TCO for TFS : $ 10 000 per year (see previous)
· Team Suite for the developers : $ 44 000 per year
o One Update during the 3 years : $0 as the price include the maintenance/SA
· Integration between Team Suite and TFS : $ 0
· Integration of the results in the reporting : 3 days per year -> $3k (mainly to build the reports)
· Total annual cost is $ 57 000
Open source
· Source control + project management + continuous integration + reports : $ 27 000 per year
· Integration of the quality tools in the process
o Code quality (unit test, code coverage) : 10 days -> $ 10k
o Profiling tools integration : 4 days -> $ 4k
o Code analysis integration : 9 days -> $ 9k
o Manual test : 1 days -> $ 1k
o Functional testing : no “good enough” open source solution
o Load testing: no “good enough” open source solution
o Architecture tools + database management : 9 days -> $ 9k
· Integration of the results in the reports: 20 days per year -> $ 60k
· We agreed (like for Team System) to have one update over the 3 years. We divided the integration efforts by half as the team is more familiar with the tooling
o Integration of the quality tools updates : 15 days -> $ 15k
o Update of the reports : 5 days -> $ 5k
· Total annual cost $ 64 000
And you know what?
· Issue 1 – You still have no functional and load testing
· Issue 2 – You will suffer integration pain and have less granularity and flexibility in the reports
· Issue 3 – The integration you will be build will lock down the way you are working and will rigidify your development efforts
· Issue 4 – You’ll have to keep up with Team System as Microsoft will continue to add value in the solution
o Just look at CAMANO in Team System 2010 for example, where is the open source solution for testing labs and tools to replay a defect scenario (using virtual machine, videos ...). And what about the integration again?
o And camano will be part of Team Suite with no additional cost J
Keep in mind that TCO is not equal to Acquisition cost.
Many techies don’t care about integration and reporting ... manager and CIO do.
To regain credibility we need to be more predictable by using a strong foundation built on process and integration and having automated quality check to ensure that we are delivering the right level of expectation.
Open source solutions are far from being free and only ideologist or “young” architect/project manager could think so. At the end of the day, we are all responsible of our choices and I strongly recommend Team Foundation Server and Team Suite as a good candidate to support the application development best practices.
Again, remember my initials disclaimers. Cost is only a dimension and the challenge of building software is not that simple. What is the risk (or the cost) of not moving? What is the risk of not increasing the software quality? If your website generate $ 100 000 per hour and the site is down because of a bug, are you still happy only about the dev cost? If your top customer is complaining about a defect or a low performance and is looking at the competition or you are losing credibility in his eyes, are you still happy only about controlling the project’s cost?
Look at my post as a light on a specific dimension but the challenge is far from being simple (and If it was, someone would have found a good way to resume it before me J).
Cheers,
Jiji