As you probably know, the first Visual Studio “Whidbey” beta was released a few months ago, and we are hard at work finishing the product for release sometime… soon. ish.
As you also probably know, Microsoft is now threat-modelling all new components that go into products as a way to identify and address potential security flaws during the design phase, where it is much easier and cheaper to fix problems than in the “let’s get it out the door” phase.
Anyway, recently our team has been going through updates for the threat models that were originally done for our components to make sure they still map to the product after bug fixes, DCRs (Design Change Requests), and customer feedback has been taken into account.
Someone who is fairly new to the team (and had not been in the initial round of threat models before we started building “Whidbey”) remarked that the threat model meetings we had were not very useful since no bugs or problems seemed to be found as a result.
But that’s exactly the point!
During the initial threat modelling exercises, we did find design problems with various components and many changes were made to the way the product was built — some small, some not so small — and the reviews we are doing now are just to ensure that nothing slipped through the cracks with the inevitable churn that comes from a multi-year product cycle with lots of customer feedback.
There is still value in the threat model meetings, as the people present (devs, testers, and PMs) often learn details about the product they may not have known, and the QA team gets more ideas about how to write test cases to attack potential weak spots in the product (eg, to verify that the mitigations we list as being in place really are in place 🙂 ). But the fact that the meetings are for the most part a rubber-stamping committee makes me feel good, not bad, about the status of the product.