Migrating from VSS to Hatteras


Buck posted the preliminary plan on VSS->Hatteras migration, be sure to check it out if you currently use VSS and might be looking to make the change.

 

The comments are mainly focused on the drawbacks – you lose VSS sharing and pinning.

 

I think between the label and locking capabilities of Hatteras, people that use pinning should still be ok. However, if we could hear an example or two of how people use pinning, we can either show that Hatteras fulfills the same need, or we’ll have a nice feature request šŸ™‚

 

As far as sharing, there seems to be two main scenarios that people are using sharing to provide:

 

1) Common or 3rd-party code or libraries that are used by many projects, but in such a way that either none of the various projects are updating it, or they’re all taking the same updates (e.g. when a new version of the library comes out).

 

2) Common code that any of a variety of projects might update.

 

I’ll talk about number 2 first (naturally). This scenario is what Albert Einstein and others call “spooky action at a distance”. Granted, they’re talking about particle physics and we’re talking about source control, but the parallels are actually fairly strong.

 

The underlying assumption is that all of those projects sharing the object(s) don’t mind having new versions show up, either because the people touching the shared objects know of all of the clients of the object, or are reasonably certain that their changes won’t negatively impact any of them. This, in our experience at least, is not true, but that does NOT mean it isn’t true for you. Ask yourself if you’ve ever had project A break because someone updated the shared code in project B – nothing wrong with project B still, but all of a sudden, project A won’t build, or has some kind of new ‘bug’ that turns out to be due to a change in that shared code.

 

If you’ve never had this happen to you, then you’re the perfect customer for VSS-style sharing. If you HAVE been a victim of this, then I will make the claim that you don’t want sharing, you want a branch and merge model instead. I also believe the #1 folks want branch-and-merge, so let’s move on to that.

 

Branching and merging between branches lets you decide when the various branches pick up changes (either from the original ‘trunk’ to the branch, or the reverse – even across branches with a “baseless” merge, but that’s another blog entry all by itself).

 

So, for the #1 folks, you’d branch your shared code into each project tree (using disk space that scales on the order of a few bytes per file branched, not to the size of the files branched, mind you). Then, when a new version becomes available, you’d change the trunk, make sure it’s happy, and check in. Then, in each branch, you can pend the merge of the changes, ensure everything in that project is still happy, and then check in – making any changes to that project’s sources, if necessary. If you find yourself having to change the branched bits to make things work (this represents some segment of the #2 crowd, the question is how big a segment), you can then decide whether and how to merge those changes back to the trunk, if you want to keep the shared code as identical as possible across all the projects.

 

While sharing works very well for those it is ideally suited to, it does something between frustrate and completely fail for those it is NOT ideally suited for. Branching and merging are somewhat more intimidating, but is significantly more flexible, letting you choose when, and tailor how, shared items are updated.

 

Obviously, neither system is universally ideal – there are clearly those of you out there that will never run into the potential “spooky action” that sharing can lead to. I’m curious to know, from among those of you that use sharing, how many of you fall into category #1 above vs. category 2 (or still other categories I haven’t been made aware of). I can envision a way to make branch-and-merge act like sharing, in terms of a policy you’d turn on (like exclusive checkout) – that is, if you check in a change on a branched item, you would want to automatically merge that change throughout the branch tree for that item.

Comments (16)

  1. Steve Perry says:

    We fall into catagory 2. We have 3 builds of a product for the most part are identicle. 1 is the base product that most people run. the 2nd one is a striped down version that the data entry temps use and is specifily optimized for fast data entry. the 3rd build is an specialized version for our audit department that has specialized searching and auditing modules. We have classes and forms that are shared across all three. Since we don’t always build all 3 for every release (if the new feature does not affect one or more of the others). We are well aware that changes to the sare modules usualy break the other builds but fixing those is part of the preperation for releasing the new build. I have never used branching and merging but the big issue I see is that the person doing the merging must be very familar with all the changes, and if these changes have take place over a long period of time and by multiple developers, you are back to the same issue as sharing. To me sharing is very usefull and would be a huge loss.

  2. Chris says:

    Thanks for the feedback, Steve.

    I will make the ‘claim’ that you’d like the branch and merge model better once you got used to it, though I’m not saying I’m so sure of it that you’d be crazy not to do it. You’re in that gray area where it’s hard to say, in a vacuum, whether you’d spend less time fixing breaking changes with sharing, or less time tweaking a merge when applying it to your "one-off" releases.

    If you don’t mind my asking, about how many devs do you have (either touching the shared code, or across the 3 projects – whatever ballpark number you don’t mind sharing)? I think people don’t really have problems with sharing until the number of people touching the codebase passes some certain size, but pinning down that size isn’t an exact science, since there are a lot of other variables, e.g. how much of the codebase is shared vs. project-specific, how many projects include the shared code, etc.

    To contrast the lack of sharing, what features of Hatteras are you most excited about? Or, what limitations in VSS would you most like to move away from?

  3. Steve Perry says:

    There are currently 6 developers working on the project(s). Since this application has been around for 5 years (vb6) there have been as many as 10 developers. The core project contains 197 files of those only about 20-30 are not share with one or both of the other projects. The main project is on 1-2 month build/release cycles, the other 2 project can go anywhere from 3-6 months between build/releases.

    As far as what features of Hatteras I’m excited about? The only thing I’ve read about is the intergration with the bug/task tracking. Intergrating the association with the files that change and the task associated with the change will be a great thing.

    Thanks for the terrific blog, and fantastic information šŸ™‚

  4. Chris says:

    Steve,

    I’m pretty excited about the Work Item/Source Control integration as well. In fact, I worked on Currituck, the work item tracking project, in the earlier stages before moving to Hatteras, and I’ve "rolled my own" bug tracker at a previous company, so I can tell first hand how nice it’ll be.

    I’m still convinced that helping move people interested in the Team System suite is the Right Thing To Do, but hearing people pining for pinning (heh) and missing sharing makes me realize we need to make that transistion as painless as possible. Part of that clearly needs to be a "So you like sharing…" document that helps ease the transition into the branch/merge model. I think branch/merge has a reputation as an intimidating way to handle shared source code (in general, not just in MS land), and that’s something we’ll have to tackle.

    When I discussed the sharing issue today, someone else mentioned reorganzing the source tree such that you only *need* one copy of the code that’s shared. I assume you’ve considered and rejected that for one reason or another. I’m curious, if so – maybe we can address that blocking issue instead? The build system in Whidbey has some considerable changes from the previous releases, and so I’m wondering if there’s a new flexibility there that might help.

  5. Steve Perry says:

    I guess the one of the main problems with branching and merging today is that you have to start off sharing anyway. I also ran into a problem a couple of days ago, after reading you comments, I was presented with a problem in which I needed to update a web service that is in production, but we have made lots of changes that are in testing, and this change needs to go out right away. So I attempted to branch from a labeled earlier version. So Long story short, I did’nt do it right, got confused and ended up losing (deleted and purged) the most recent version of the web service. So what I’m trying to say is branching is not an easy to use or understand.

    I will come around, but its just going to take a while.

    Once again thanks for your time.

  6. Chris says:

    Yeah – the fact that Hatteras doesn’t use the sharing model will feel different for people coming from VSS, but your scenario is a typical reason why you *would* create a branch instead of a share – you can safely work on fixes to the production in a "release" branch while doing new feature work in the main branch (for example).

    So, in your case, even if you didn’t create it at the time, in Hatteras you can branch from a non-tip version (the one you labelled as the release version). Checkins to each branch do not affect the other (as they would with sharing); instead, you merge from one to the other to move changes between the branches.

    For this specific situation, that sounds like that would be preferable to sharing, but there are plenty of situations where sharing is preferable (when you *know* you *always* want the changes in one branch to immediately take effect in the other branch).

    I need to get a list of common usage scenarios worked up, and show how VSS and Hatteras treat each one – maybe that will help people decide which is right (or "more right") for them.

  7. David Chapman says:

    We use shared files far more than we probably should but there is one scenario where it’s invaluable: when we share a common file defining our file format. We have multiple products that read the same proprietary file format. If someone makes a change to this file in one product we WANT the other products to break if they haven’t been updated.

  8. The scenario David Chapman identified (common header files) is one we use heavily too…and one I personally don’t think suits the branch/merge model at all. How does Hatteras cope with that scenario??

    That said, in every other case where we’ve used sharing/pinning it’s been because branch/merge is so poorly supported in VSS. Pinning is a nightmare in the VSS IDE.

    One question: Does Hatteras truly support branched projects, or is it file orientated in the same way VSS is? For example, I’d like to see files added to a development branch automatically propagated to production branches when they are raised to the next project version…

  9. Chris says:

    David,

    Makes you almost wish Windows had symlinks, eh? šŸ™‚

    The scenario you put forward is one where sharing makes perfect sense, if pointing the multiple projects at the same *one* file just isn’t possible or feasible (and I know that can happen).

  10. Chris says:

    Anna-Jayne: Tough, specific questions – my favorite šŸ™‚

    Re Common files: Off-hand, it looks like Whidbey supports this by adding existing items with "Add as Link" from the add menu in addition to the normal Add (Add, as you probably know, copies the item even if you added it as an existing item).

    A linked item stays wherever you added it from, but otherwise is treated as any other file in the project, *as far as I can tell* (I’m no Solution Explorer expert), can be compiled against, etc. You can only remove the link from the solution explorer, rather than delete or rename the item, but that’s probably A Good Idea anyway, given that it’s a linked item.

    The answer to your project branch question is mostly yes and partly no. Bear with me.

    We branch and merge items on a per-file basis, in terms of how we track item history. However, you can perform branch and merge operations from any level in a tree of files recursively. So, if the *server* tree structure of your project is such that the new items are under the project root, then yes, any additions would be merged (as branched items) when you merged the project’s changes.

    The reason I emphasized the "server" tree is the fact that you can remap the way the files are located on your client workspace to be different than they’re organized on the server. Generally, that’s something you wouldn’t want to get too fancy with; it’s a feature that (to use a popular saying in MS land) gives you just enough rope to shoot yourself in the foot with.

    But, it can come in handy to help address your need (project-oriented branch/merge) – say you version some binaries that are stored in $/ProjectFoo/lib ($/ is the semantic for server paths in Hatteras), but the way your build is setup, you want them in c:foobinlib on your machine, instead of C:sccProjectFoolib, where they’d end up by default. No problem – you’d just remap $/Project/lib to c:foobinlib in your client workspace (similarly to the way you’d override a child node’s local/working folder in VSS). They’d still get branched/merged appropriately when you did a branch or merge of $/ProjectFoo to $/ProjectBar (for example).

    My assumption, of course, is that the single-rooted tree makes sense for the server organization, and you’d only need to reparent items on the workspace for some practical reason (the nature of your build system, for example).

    I have to admit that I have very limited experience with pinning, or (more accurately) the situations that prompt people to need it. I’ve found myself getting a specific version of a file and making it writable, so I don’t sync newer versions – but I assume that’s a pretty limited example of a pin-like situation, and a hack to boot. What’s a better example of a situation/problem where pinning seemed like the solution?

  11. Chris says:

    Oh, one other possibility regarding shared headers –

    I don’t know which version of Visual Studio introduced this, but Whidbey (at least) has a type of reference called a "Project Reference" – basically, referring to another project in the same solution, instead of a (usually external) .dll.

    Would putting your commmon headers in one project, and creating project references to it from your dependent projects, work for you? I can imagine situations where it wouldn’t, so it’s not a loaded question by any means.

  12. "Would putting your commmon headers in one project, and creating project references to it from your dependent projects, work for you? I can imagine situations where it wouldn’t, so it’s not a loaded question by any means."

    Arguably that’s the way to do it, but I would say that a team shouldn’t *have* to change its project structure just to integrate a new source code control tool. As a VSS admin I know I’d not be popular if I suggested this!

    Surely it wouldn’t be too difficult for Hatteras to support reasonably close equivalents to the VSS share/pin capability – even if largely for the purpose of migrating existing projects/databases?

  13. Chris says:

    Well, I realized later that the project re-org has all sorts of problems depending on what exactly you’re trying to do (for example, now they’re in a separate assembly, which probably isn’t "better" and would quite likely be worse), and it gets more complicated still if we’re talking about native instead of managed code, etc.

    I fiddled around with the "add as link" approach yesterday, and that seems to work, though I’m sure there are drawbacks there, too (off hand, for example, I remember how contrived C++ include paths can be even without having to contend with links).

    It’s not really a question of whether it’s "too hard" to provide sharing/pinning; it has more to do with the branch/merge model and the various ways that sharing "breaks" the model, in ways that end up frustrating users. It’s something you don’t tend to run into until team size grows. Well, that’s indirect – in truth, it’s the number of people touching shared items and the number of times and item is shared that governs how likely you are to get bitten by it.

    Having said that, I wonder if there’s a way to meet in the middle, perhaps using policies (that is, create a way to say "you can’t share files normally" but allow users with certain permissions to share specific items). This is just stream-of-thought here, mind you. The problem is that one user’s life might be made easier by having two items shared rather than just branched (to be merged between later); while another would find himself bitten by the "spooky action at a distance" aspect of sharing. To an admin or to either user, it’s usually pretty easy to see which they’d prefer on a case-by-case basis; but the system lacks that context and insight.

    It’s something we’ll keep working on, to be sure.

  14. This has come up on my blog before, but since it came up in the forums again, I thought Iā€™d point people…