During my recent brownbag on TDD one comment was that having a lot of unit tests makes it harder to refactor code. First of all the word refactoris in my opinion misused as much as mocking. There is a difference between refactoring and rewriting. Refactoring means you change the code to be "better" without changing any functionality. Rewriting means you potentially change things completely including changing interfaces in APIs. So first of all, if you refactor your code and that means you have to change your unit tests your unit tests are too tightly coupled with your code. Lesson learned; somebody screwed up when writing the tests in the first place. This happens all the time when you learn TDD. It is just a matter of biting the bullet and learn form the mistake you made.
So back to the original statement; it should really be unit tests makes it harder to rewrite code. Think about it. You want to change an API and some functionality and as a result of that you have to update hundreds of unit tests. That's a big pain right? But wait... All these tests that now fail means I have manually verify that things I didn't want to change still works and things I wanted to change are changed to the correct thing. That is all good isn't it? All the failing unit tests are really like a buddy asking; Did you really want to change this? Is it really a good idea to change this? Consider the alternative with no unit tests... Sure you don't have to "waste time" updating old unit tests but neither do you know if your code works as expected.
So if you're ending up changing tests when you refactor code; somebody screwed up. And changing unit tests when you rewrite code is a good thing!