How I Learned to Program Manage an Agile Team after 6 years of Waterfall


Agile is the single greatest things a team could do to significantly improve the user experience and quality of their site. Now having said that…

If I could go back in time… 3 things I would tell myself about Agile

For a long time, I’ve said that my biggest job on the CodePlex team is to learn agile. Now I find myself saying “if only I could go back in time, here’s what I would tell myself about agile:”

1. Design and plan for the very next step, instead of designing and planning for the very next feature.

2. Break down work into smallest possible sets. Adding work is fun and rewarding; removing work is painful and risky.

3. Design and plan 80% of the way as the very next step. Use feedback to solve the remaining 20% in the very next step after that.

And there’s probably a 4th point that’s only relevant to a Microsoft Program Manager going from multiple 3-year product cycles to 3 week agile releases:

4. Forget everything you’ve been told about what makes a Program Manager successful at Microsoft. Thinking about this will only drive you insane and make you completely miserable on a real agile team.

Agile: A love / hate relationship

Usually at this point, someone asks me, “so how did you convince your team to do agile?” I have to laugh and say, “It’s the other way around.” The team already consisted of agile people. I was the person with zero agile experience entering the team with my traditional Microsoft software development ways.

To illustrate how clueless I was about agile, during my first week on the team, I had asked a specification question, like how many characters do we allow for a password? And Jim says, “we’ll have to look it up in the test cases.” And I said very proudly from my Visual Studio experience, “Gotta love it when the test cases are the specs.” (yes, I said this to Jim Newkirk, my manager.) To which Jim very calmly replied, “That’s the way it is supposed to be.” And I laughed, and then after a pause, I said in slight horror, “you’re serious, aren’t you?” It was like finding out there was no Santa Claus or you really don’t vote for the President of the United States. The mind just simply can’t comprehend.

Needless to say, the CodePlex team had a lot of “untraining” to do. They estimated it would take about 18 months for me to chill out, and it actually took 18 months for me just to see that agile could actually work well. I still have my moments when my “VS-side” (the traditional program management training) takes over l, but this really stems from me not having enough info to replace #4. After 6 years of being told how everyone views your role in the world, it’s impossible to see it in any other way, especially when you still work in the same company.

And just to get this out of the way now so there’s no confusion… if you asked me personally what i thought of agile, I would say that Agile drives me crazy for reason #4. Why? Well, I was hired straight out of college by the Visual Studio team, so traditional waterfall software development is all I’ve known for 6 years. And since there were so many people in the VS org, everything had to be well-defined, from what was expected from us, how we performed our jobs, etc. At any given point in time, I knew what was expected from me, my dev, my PM (since I was mostly a SDET / tester during my VS career) and how they would do it.

Moving to agile, I knew what they wanted me to do (have meaningful work for the devs to do, and make sure they always have the right work to do), but had no clue how they wanted me to do it. I called it “navigating a labyrinth that had invisible walls outlined with electric fences.” I only learned my way around by making one agile mistake after another agile mistake. It wasn’t really until I decided to try for one week, “I don’t care how I’m evaluated. #4 never existed. And I’ll just see if Rome burns down…” that I actually felt true happiness at work. I’m only sharing my personal feelings to prove that I’m not just drinking from the kool-aid (or anti-kool-aid, in the case of Agile at Microsoft). I really think agile is awesome (see next paragraph), even if it comes to me at great personal cost (see reason #4).

But if you ask me as Sara Ford, the Program Manager of CodePlex, and the author of this blog post, I will tell you that agile is the single greatest things a team could do to significantly improve the user experience and quality of their site. I honestly believe this to be true after everything I’ve seen, even if reason #4 drives me crazy and makes me miserable inside. But as I tell myself to get out of the car each morning, “it’s a chance of a lifetime to fight the evil empire from within!” (yes, you can quote me on that.)

How we build the CodePlex software

Before I can explain the above 3 things, it’s important to understand how we build the CodePlex software, especially if there are any other MSFT Agile PMs out there reading this. (We should start a support group, btw.)

We deploy every 3 weeks using 5 week deployment cycles, as shown below. There’s roughly 2 weeks of new feature work, then 1 week of bug fixing. Then we “cut the RC” (Release Candidate) where we fork the code so Test can start the full test pass (regression) on the RC bits, and the devs can start new feature work on the “main” code branch.

5 Week Deployment Cycle overview

Besides having a very good “very next step” plan ready to go, another reason why agile works for us is that we all sit in a team room, as effective communication is key. Got a question? Ask the room. Never be blocked due to communication. Another inside joke here of the “untraining of SaraF” is how long it took the team to break me of the “don’t interrupt the dev,” a strong motto from my Visual Studio days. Somewhere in month #2 or #3 on the team, I asked someone “so, um, how am I supposed to ask a question out loud to the room.” The answer was, “um, just ask the room.” (see #4)

Now, once again, me as Program Manager thinks a team room is excellent, but it took a little (okay, a lot) to get used to working in a single room of so many people, after years of fighting for a window office. The solution we found was to put me facing the corner of the room – out of sight, out of mind, while giving me the sense of “an office”. Significant job satisfaction improvement with that single tweak.

As show below, all the devs sit together at the pairing stations (to the right), and over on the left is where the test team sits. I later moved to the desk in the bottom left FTW!

clip_image003

We use a variation of “Extreme Programming,” but are 90% following XP. Other agile aspects include

  • One week iterations
  • Test driven development
  • Continuous integration
  • Pair programming
  • Shared workspace
  • Collective ownership of codebase among devs

Why those 3 things are so significant

Now back to me. =D

Jim and Jonathan (bless their hearts) tried their best to explain agile to me, while putting up with all my whining. But I didn’t believe them that Rome isn’t burning (and still don’t to some extent, but to lesser degrees now compared to back then). I’m still expecting the world to end (see #4), but just less often, but there are still some special cases out there that just freak my inner waterfall self out “ARGH!!! WHY ARE YOU MAKING THAT CALL AND NOT ME?! Rome is burning! I’m not doing my job righT And!!!  Oh wait, what’s that? I’m on an agile team and this is my VS-side showing? Everything is fine? Oh, okay, um, nevermind. Let’s open the next bug to triage.”

But, it’s like anything else, until you figure it out for yourself (like leading a horse to water…), listening to instructions over and over is just white noise. (no offense Jim or Jonathan). Or better yet, like I say in karate, you really don’t know a technique until you can teach it to someone else.

Let’s take a look at those 3 things again, but this time in detail, so I can prove (or be disproved) that Agile has clicked for me.

1. Design and plan for the very next step, instead of designing and planning for the very next feature.

In traditional product groups, specifications can be as long as 60 pages. Every scenario must be known at design time and figured out, because you only get one shot of getting it right. The analogy here is firing a gun. You could spend a significant amount of time planning, calculating, aiming to achieve the perfect shot. Or using the agile approach, you could aim and fire, recalculate aim and fire, recalculate aim and fire. You could say that traditional product groups do this to some extent, but a few years is a long time to wait for a recalculated shot.

What I didn’t know when I first joined was that it was “okay” (recall #4) to break down the feature work. I honestly though features had to be designed end-to-end and then squeezed into a deployment. Once I had that glorious light bulb moment of “plan the deployment based on the very next step, not the very next feature,” everything else fell into place.

This first light bulb moment occurred when I wanted to do a particular feature, but the way the designs ended up would have taken us a deployment cycle of 6 months. This of course was unacceptable, and I accidentally found myself saying, “if they would have just scoped it to the very next step I could get started with this…” and that was the moment I realized that agile actually had a structure to it.

2. Break down work into smallest possible sets. Adding work is fun and rewarding; removing work is painful and risky.

Maybe it was light bulb moment #1 going off that enabled the “on” switch for light bulb #2.Towards the end of last summer, I was so exhausted from the book, trying to figure out agile, doing major speaking events, and of course not to mention promoting open source within Microsoft, I decided to “give up” for one particular release and just do the minimum amount of work possible to get the deployment out. I was just too exhausted to worry about enduring the potential wraith of my manager (see #4) for deploying such a small amount of work.

And of course, it turned out that was the ideal amount of feature work for the deployment. I discovered this simply based on the number of bugs that had to be fixed in the 3rd week of the cycle, and how there was no feature work that needed to be carried over to the next deployment cycle. In other words, when I finally “failed” in the traditional program management sense, agile clicked for me. And people wonder why I was a nervous wreck all the time (see #4).

And to explain the second half of this light bulb moment: If there were any feature work we couldn’t squeeze in, it had to be carried over, but yet it wasn’t “planned” this way. Then it was a question of do we finish a half-written feature, or do we pull it out, not fully understanding what other aspects of the feature sets it could affect in this state? We would have to make a best guess decision right before deployment.

Another analogy is like Legos. Break all the pieces down into 1×1 blocks. Then it is easy to plan, add, remove. But the larger the pieces, the more connection points, the more cost, the more complexity. I’m sure there’s some analogy to the connection points (the little bumps on the lego square) and writing the code, but I’m not a developer on an agile team, so I’ll leave that open to interpretation.

3. Design and plan 80% of the way as the very next step. Use feedback to solve the remaining 20% in the very next step after that.

Immediately after light bulb moment #2 clicked, I saw the very reason why agile rocks for customer quality. Using agile, you actually can recalibrate, aim, and fire in a time period that is reasonable for customers (compare 3 weeks to 3 years.)  I’m not talking about overall feature set comparison, but a comparison of the quality of the features being released. It was finally at this point in time I could allow myself (see #4) to take advantage of this “aim fire recalibrate repeat” concept by actually planning the recalibration time period. In other words, I could say “okay, we’re going to go with this, and based on feedback, we’ll tweak as needed in the next deployment.”

In Conclusion

Once these three wonderful light bulbs went off, I demanded from Jim a whiteboard. I had this awesome idea that I could use sticky notes to track all of the individual 1×1 (or 2×2) Legos (as shown below). Then someone (I forget who) said, “That’s how agile teams typically track work.” Part of me wanted to rejoice that I had finally graduated to some level of “I’ve figured out agile,” and part of me wanted to scream, “Why didn’t you tell me this before!?” But they probably tried, and had I listened, I would have had just a big sticky note called “Feature”, and we’re back to day 1 all over again. 

clip_image004

Just some final thoughts about #4, since I refer to it so often. I just want to do the best job I can, because I’m here to fight the culture to make Microsoft better for customers. (I know what it is like to use Visual Studio in college (graduated in 2001) and Microsoft asking me for 250 dollars (my month’s rent) to report a bug about the IDE, hence the “evil empire” reference.) But I’ve been so trained by #4 on how software development should work, that it’s been an uphill battle to learn Agile, even if it think the process produces higher quality results at the end. Jim must spend at least 30 minutes on average with me each week trying to get me to accept that I am doing a good job, even though all of my traditional waterfall training says everything to the contrary. (we should do a podcast of my 1-1s.)

I guess #4 is to a Program Manager like trying to convince a developer to write unit tests or to do pair programming. It’s hard cultural change to accept.

So, to all my fellow real agile PMs at Microsoft, I hope this posts helps in some way and I’m serious about the support group. And to the rest of you, I hope this has been an interesting read to say the least of what’s it like to change the culture at Microsoft, especially when you are quite literally the cultural change.

Comments (27)

  1. Great post Sara. I’ve not read your blog before but came across this from Google Reader. You provide a great description of the mindset shift that occurs when making the leap from Waterfall. Welcome to the dark side 😉

  2. Brice Busselman says:

    Great post.  Thanks for sharing your experiences.

  3. Brice Busselman says:

    Great post.  Thanks for sharing your experiences.

  4. Max Battcher says:

    Hey Sara,

    Thanks for a wonderfully candid post. I’m currently outside of MSFT, but hopefully there are those inside that appreciate your insights in this post. Always interesting to read stories of those working Agile methodologies inside MSFT.

  5. Thank you for submitting this cool story – Trackback from DotNetShoutout

  6. TurkeySub says:

    Could you mention what specifically from #4 you found most difficult to let go of?

  7. Jim Holmes says:

    Awesome post!

    I fled a heavy CMM/Six Sigma/ISO 9000 process environment because I’d heard about Agile and I wanted be in it. Letting go of those nearly useless 60 page spec docs, and learning to be OK with late design decisions, was a hard thing. I laughed when I read your references to these.

    Good on you for your great journey, and good on you for fighting the good fight at Microsoft!

  8. saraford says:

    @TurkeySub Everything i’ve ever been told about what a good PM does got flipped upside down, so i’m confused by little daily things like, "if i do ‘x’, is it because it is the right thing to do? or is it because i’ve been brainwashed into thinking that it is right?" I have a strong tendency to over-analyze, and it is only making #4 worse for me.

    It’s almost like dying from a 1000 papercuts. None of the individual ones are bad at all, but the sum is overwhelming.

    I hope this makes sense.

  9. Jeff says:

    There’s a great podcast where Sara talked about this several months ago.

    http://deepfriedbytes.com/podcast/episode-15-visual-studio-tips-and-running-an-agile-team-with-sara-ford/  

    Check it out if you want some additional insight.

    -J

  10. saraford says:

    @Jeff ah yes, thanks for adding the deep fried bytes podcast! That was quite a day. Started at 5am in Northern Alabama, drove 4 hours to Northern Miss to say hi to college friends and advisor, then drove 3 hours to Southern Miss for the podcast. There just wasn’t enough coffee after that many days driving my rental car on the road.

    Nothing like getting to the airport and saying "Yeah, i put 2000 miles on the car. Might want to check the oil." =D

  11. Seth Rowe says:

    Terrific article! It’s very nice to read about you sticking through the tough aspects of adopting Agile for the betterment of your team and Microsoft in general.

  12. Jason Welch says:

    Great read – thanks for sharing your insights!

  13. YvesHanoulle says:

    I think this is a very good publicity for agile, especially with your hate feelings about what agile does to you persnally. It explains very nicely why it is so difficult to unlearn PM stuff.

    It is also a very nice publicity for you and MS, that you really tried for a long time to do this even when it was hard for you.

    Thanks

  14. Peter Stephens says:

    Thanks for the great article Sara.

    I’m been doing waterfall for a number of years (14??). But Agile keeps on coming up. I can see that it is a superior method and the wave of the future.

    What books do you recommend reading to get up to speed on Agile?

    Thanks,

    Peter Stephens

  15. nils says:

    so Sara, if given a choice to you for next assignments, which methodology would you prefer.. waterfall or agile 🙂

    After the Agile experience, do you think that any of your earlier projects would have been better executed with agile than waterfall?

  16. Mike Lim says:

    Great article!  I’m going thru a similar struggle myself.  I worked on big gov’t programs for 6 years using very traditional processes.  And for the last 3 years we’ve been using agile development.  Not very well to be honest but it’s certainly allowed us to have a lot of success.  The biggest problem is that many of my developers have the same process experience as me and we’re all having problems unlearning.

  17. Eduardo says:

    Sara,

    Great post.

    Now that you experienced both worlds do you think that agile is doable and valuable in a "boxed product" team like VS, Office? I mean Codeplex is more like a service where new releases are easier to apply.

    I would love to hear your impression.

  18. As I was going through postings from our internal agile discussion forum (yes, we do have one at Microsoft

  19. Edmundo Andrade says:

    Sara, you must feel like a winner because you has survived after all that Agile learning battle.

    Congratulations for that. I hope Microsoft can be transformed by persons like you, as IBM and some other huge companies are doing.

  20. Debadatta says:

    A great article for future Manager to understand the importance of Agile development Cycle

  21. Ian Nelson says:

    This is a really good post, Sara.

    Thanks for taking the time to share these experiences.

  22. saraford says:

    @Edmundo it’s more like I’m trying not to lose =D

    I have to say it has been fascinating being inside Microsoft over the past 7 1/2 years watching the culture change.

  23. Max says:

    Have you read "Software On A Leash" by David C. Birmingham? If you think Agile is different, you need to look at this methodology. Completely different and supposedly more agile.

  24. Karl Hering says:

    Fantastic post…just had a similar but less successful conclusion on a project as the team here just couldn’t do away with the old waterfall approach.  The only thing that saved it was ultimately a redirection back to agile precepts plain and simple.  Really wonderful post!

  25. Jerome says:

    Great read!

    Hello, I‘m a Chinese student. It’s very glad to read your blog and thanks for sharing your insights.

  26. stevie says:

    Thanks for your honest perspective on your journey from waterfall Program Management to agile.  Much of what you described I see first-hand as an agile coach.  Now I better understand the struggles the PM’s I work with are having in making the switch to agile.