Only Human


I’m sure others have talked about this, but I haven’t had a chance to check out the other blgos, so I’m going to throw in a little bit of information about what’s going on work right now.

 

On the road to shipping beta2 we’ve now entered a time called “ask mode.”  The name comes from the fact that before we’re allowed to make any changes to any part of the IDE we have to “ask” for permission from up on high.  Actually “ask” is a rather generous term.  What you’re actually doing is pleading, cajoling, bribing, or just pain arguing that the issue you’re trying to resolve is necessary for beta2.  In order to judge this the “ask mode committee” attempts to determine where the “bar” is in terms of the quality of VS and whether or not you’re currently below that bar and if the fix you have will raise you above it.  Of course, as you get closer to shipping this bar gets higher and higher as the damages that will happen if something goes wrong will get more critical.  Less time to fix bad bugs means a better chance of shipping crap out to you.  So they try to find a careful balance of determining which bugs are worth fixing versus the potential for worse bugs to be introduced.

 

Each team has their own individual “bug bar” as well which they’ll use to triage incoming bugs to decide if they should be fixed for beta2 or if they should be fixed for the final product.  Each team has their own set of criteria, i.e. hangs, crashes, critical functionality broken, but it’s pretty safe to say that the “bar” is defined as: something so bad as to be known to be completely wrong while *at the same time* making the IDE so unusable that we won’t be able to get good feedback about everything else because this will be bothering users all the time.  For example, a crash in a mainline scenario (like typing) is really3 bad.  It’ll make the product unusable and will prevent users from actually getting to try out more than 5% of its functionality.  However, a crash in some very bizarre and difficult to get into scenario might not meet the bar.  Something like a crash in the immediate window when you type a very special type of expression that has an associated debugger visualizer on it that needs to cross security boundries to a sql machine that then executes a stored proc with a managed-to-native transition.  That’s just something that we can deal with since it will probably hit a low percentage of users and one could still effectively use the rest of the product. 

 

What’s interesting is that one bug we brought to “ask mode” that was actually accepted was a problem with type colorization where the colors had accidentally been redefined so that teal was now showing up as light gray.  “What!!!” you exclaim “how could that possibly meet anyone’s bar???!”  Well, in this case we felt (and argued) that the experience would be so god awful for anyone typing that the literally would hate every moment of it and would feel so negatively against the product that they wouldn’t want to use it now or in the future.  So while it wasn’t something so obviously bad as a crash it was so completely wrong for the C# customer that we felt that that was the only issue they would give us feedback on at the expense of everything else we’ve worked on.

 

Now, for a dev this time is incredibly hard to deal with.  For months and months you’ve been happily coding at a great clip.  Your features have been getting better and better and you’re totally psyched that they’re going to get into the hands of the customers who’ve been waiting.  And all of a sudden it comes to a screeching halt.  Issues that would have once take 5-10 minutes to fix now take on the order of 4-5 days.  No, that’s not an exaggeration.  Now, a fix which you could have coded up, had reviewed and checked in after having all tests pass has to go through the following gauntlet:

 

1)      You have to have your individual team triage and sign off on fixing the issue for beta2

2)      You have to prepare the fix

3)      You have to the fix reviewed

4)      You need to create a build of the fix that you can hand off to QA where a far more complete test pass will be done over it including integration testing where it’s banged on a real world scenario to make sure that nothing has regressed

5)      Once it’s been signed off by dev and QA you need to place it in escrow where it cannot be touched again. (if you touch it again you need to start at the beginning

6)      You now need to bring the bug to DevDiv shiproom and convince then that it’s worth fixing.  The dev who reviewed it and the QA who tested it need to be there to confirm signing off on it as well.

7)      You need to bring snacks as well to bribe shiproom.  There’s about 30+ people from multiple teams that you have to convince.

8)      Sacrifice a virgin to the gods (optional)

 

Sigh…

 

Step 7 is why I often blogged about issues and asked you guys to tell me what you felt.  We really needed to understand the customer perspective on this so that we could convince others it was worth doing.

 

So productivity for a dev sloooooooows down by several orders of magnitude.  But I guess that’s ok.  We’re usually the ones causing the problems in the IDE so it makes sense to allow us to change as little as possible so that the product can truly stabilize and get into good shape for you.

 

Now, the title of my post comes from the fact that ended up completely screwing up this process.  Because of the drastic shift in policies I misunderstood the different roles that were played (especially the difference between the individual team triage, and the entire division “ask committee”), and I ended up checking in a bug fix without doing everything that I was supposed to.  This caused a whole lot of grief for many parties and ended up making the rules more stringent and more understandable so that other silly devs like myself wouldn’t rock the boat anymore.  In the end it was felt that the mistake was forgivable.  It was fixing a pri0 failure in Japanese Visual Studio and Pri0s are not considered “ask” bugs rather they’re “tell” bugs (i.e. you fix them and then tell the committee why you did it.

 

Anyways, it’s an interesting time now.  Working a glacial pace to get beta2 out of here.  Working on the bugs that have been pushed to the RTM release, and also putting a lot of thought and energy into the plans for the next release after this one.  We’re working on some kick-ass stuff and I think you’re going to love it once you hear about it.


Comments (5)

  1. Jeff Parker says:

    Hey Cyrus, hopefully you will get some more time to start blogging like you used to during some of this down time. And no, nobody really talks much about the dev process there. I know it is pretty though.

    I can’t wait to see all the new toys you may for us. A little bug BTW to make you aware of, I can’t report it, I can’t reproduce it, I have no clue what so ever happened. But you never know when you may run across something that could cause it.

    Anyway Built a very simple collection called SubMenu in C# using IEnumerable which would enumerate and Index another class called SubMenuItem. Overrode the ToString method and my IEnumerator was on a class inside my collection. Nothing too out of the ordinary. I then went to another new empty class Called MainMenu to new this collection up because I wanted to use it right away. So I start typing like this

    Submenu menu = new<hit space here> Intellisense came up but never fully populated or really don’t know for sure just seen it come up but was all white. Then a grey message box popped popped up. Litterally looked like MessageBox.Show and said Null reference Exception. I clicked the OK button and VS went bye bye.

    Unfortunately I hadn’t saved any of those classes cause as soon as I newed it up I was going to hit compile which would save for me. So I just shrugged fired VS back up and went right back in. Recreated the collection and never had the problem again. Why did I get it I have no clue, I had 2 other copies of VS open never affected them. So I really do not know what happened and I can’t reproduce it. But if this by chance might help you find a bug somewhere sometime so someone else does run into this. Then thats the only reason I am telling you about it.

  2. Jeff: Ack!

    Sounds like a handle leak to me! I’ll check into if we’ve seen that or fixed something in that area. Thanks for the info!

  3. JavaKid says:

    Don’t sweat it Cyrus, we are all "Only Human". If you work for the Man I guess you have to play by the Man’s rules… I know it sucks… ::Sigh::

    Fortunately some of you guys are showing signs of hope.

    Hopefully Microsoft will drop some of their old habits and allow you guys to do what it is you do best. Write some Kick ass software.

    If its of any consolation I’m behind you guys.

    JavaKid.

Skip to main content