Migration Toolkit and TFS to TFS updates
I’ve seen quite a lot of people having problems with and ultimately abandoning use of the Migration Toolkit and the TFS to TFS migration tool over the past year or so. In my mind, there are two categories of reasons for this 1) People are expecting the tool (particularly the TFS to TFS tool) to solve a problem that it does not solve and 2) There have been quite a few bugs that have made it a very frustrating experience. We have just released an update to both tools and you can read about the details on the TFS Migration Blog post. This release was designed to address the most egregious bugs that have been frustrating people: Making the tool install more easily, making the samples actually work, supporting TFS 2008, etc. If you are using the migration toolkit for what it is good at, then I believe you will find the updates valuable. This release, however, really does not address the problems with mismatched expectations. We are embarking on an effort to address those issues but those improvements will probably roll out over the next year.
Mismatched expectations
The TFS migration toolkit was designed to serve as a starting point for people who want to move data between TFS and another system (version control, bug tracking, help desk, or other). It was designed with two modes in mind – a “one time conversion” mode and and ongoing “data synchronization” mode. It has, at least, one big issue with the “one time conversion” mode and that is that the dates and times for all operations that you end up with in TFS are set to the date/time that the action was migrated and not the date/time that it was originally performed. This causes history to look funny and reports to be compressed because the entire history of the migrated project is compressed into the few hours that the conversion took. This issue is not such a big problem for the “data synchronization” mode because actions are synchronized soon after they happen and although the dates/times don’t match exactly they are close enough that in practice no one cares. In case you care, the fundamental underlying reason that we don’t just “fix” this issue is that it is a hard problem. The core issue is with version control and stems from the fact that we have a single monotonically increasing changeset number that updates anytime any checkin in performed. There is a direct correlation between the changeset number and the time that an operation occurred. An invariant of the system is that if one changeset number is greater than another, then its date/time is also greater. So, imagine a server you have up and running and you want to import some data from another system. The actions from the other system occurred in the past and the TFS server has been in active use. All of the changeset numbers “in the past” are already used. There’s no way to “insert” a changeset number between say number 300 and 301 if the imported operation happened temporally between them. Hence all imported operations are imported at the time of the migration operation and not at the time of the original operation. Now, all hope is not lost. We have some ideas on things we could do to improve this but they aren’t easy to do and involve changing this pretty important invariant of the system in some creative ways. So out of my rat hole and back to the larger point… The TFS to TFS migration tool has even larger mismatched expectation problems. In fact, we’ve considered changing the name because the name is almost misleading. People see it and assume, for example, it’s the solution for moving TFS from one server to another or for moving a project from one sever to another. Moving TFS from one server to another is a totally different thing and is described in detail in our admin documentation. The TFS to TFS migration tool has nothing what-so-ever to do with it. Ideally, the TFS to TFS migration tool would help you with the moving a project from one server to another but due to the time compression problem and others, it is not a very good choice for this. So what is it for? Well, if you’ve got some data on one TFS server and you desperately want to move a fraction of it into another project somewhere (maybe on another server) and are ready to start typing it by hand if you have to, then the TFS to TFS migration tool may help you with it because you are probably desperate enough to overlook the time compression problem, etc.
The Future
Going forward, we are trying to address these shortcomings. The first area we are going to focus on is the TFS to TFS synchronization “mode”. I suspect this is not how most of our customers would prioritize this but let me explain why. We are constantly building new features in TFS and improving existing ones. It is very important that we use the product so that we have a very good feedback loop so that we can tell when the features we are building work well and when we have missed the target. Our own dogfood use is an incredibly important part of this feedback loop. However, as the dogfood use of TFS within DevDiv has grown (to nearly 3,000 users), it has become clear that there are competing priorities. Most of the division needs stability, reliability, up-time and lack of churn so they can focus on getting their own work done. We (mostly the Team System team) need frequent updates of our dogfood system so we can get feedback on our work. These two priorities have created quite a conflict. Our short to medium term solution to this problem is to move the VSTS team on to a separate TFS server and synchronize it with the “main” DevDiv server. That way we can update our server however frequently we need without constantly creating churn for the rest of the division. We will probably only update the main DevDiv server once or twice per product cycle. So, while this is not generally the highest priority scenario for our customers, it is a very important scenario that will enable us to continue to deliver great TFS features to you. Once we get the synchronization scenario nailed, we will do another public update to these tools and then we will begin working on the one time migration scenario. As for the scenarios that the TFS to TFS synchronization tool was never really designed to solve, we are working on other solutions for those as well. As I mentioned above, the server to server move case is covered in our admin docs today and we continue to improve it. In Rosario we are working on features to “split” TFS servers into multiple sets of independent “project collections” and the ability to freely move project collections between TFS servers. I expect you will start to see early versions of this work sometime in the first part of next year. In the meantime, I hope you will fine the recent improvements to the Migration Toolkit helpful and, as always, we’re listening for feedback. Thanks,
Brian
Light
Dark
0 comments