One of the requirements for designing software at Microsoft these days is that every (and I do mean EVERY) feature in every product needs to have a threat model.
Threat modeling isn't new, it's been a part of good design for years. Heck, the IETF has a threat model for the entire internet 🙂
Note that I didn't say good security design, IMHO, threat modeling isn't necessarily about security design. Threat modeling's really all about understanding your design, at a level that design documents don't necessarily cover.
Threat modeling is almost more of a discipline than an analysis tool- if it's done right, it enables you to gain a much deeper understanding of your components and how they interact. And by applying that methodology, you learn a great deal about your component.
We're currently running through the threat modeling process in my group. Since I'm the security nut on my team, I'm the one who volunteered to run the process. And it's really been an interesting exercise..
The big thing that writing the threat model for your feature (or component, or protocol, or whatever) is that it really forces you to understand the interactions between the various pieces of your feature.
When threat modeling, you need to concentrate on a bunch of things. First, there's your assets (also known as protected resources). All threats relate to assets that need to be protected. If you don't have assets, you don't have threats. But sometimes the things that are your assets can be quite subtle. For example, the audio samples being sent to a sound card might be an asset (especially if the audio samples came from DRM'ed files). So might the privileges of the LocalSystem (or the current user) account. The contents of a file on the disk might be an asset for a filesystem, similarly, the contents of an email message is an asset for an email system (like Exchange). Understanding your assets is key - assets tend are attacked in different ways, the attack strategies mounted against an email message probably won't work for transient data like an audio sample.
The next thing you need to determine is the entry points to your components. The entry points to your components are what the attacker is going to use to gain access to your component.
Related to entry points, one other thing that you need to determine are the trust boundaries in your component. A trust boundary is a boundary (physical or virtual) across which there is a varied level of trust. For example, when you RPC from one machine to another (or one process to another), that forms a trust boundary. Another example of a trust boundary is the network - almost by definition, "the wire" is a trust boundary.
For some systems, there are no real trust boundaries - an interactive application running as the user that only outputs data may have no trust boundaries. Similarly, an API that processes data and returns it to the caller may have no trust boundaries.
Related to trust boundaries are trust levels - trust levels indicates how much you "trust" a portion of the system. For instance, if a user is an administrator, they are trusted to do more than normal users. When data flows between entities whose trust level is different, by definition, there is a trust boundary between those two entities.
Once you've identified your entry points, assets, trust boundaries, and trust levels, the next major part of a threat model is the data flow diagrams. A data flow diagram indicates the flow of data into and out of the various parts of your components.
All of this is the up-front work. It lays the foundation for the "meat" of the threat model, which, of course are the threats. I'll write more about threats tomorrow..
One of the things that I realized while writing down all the stuff above is that getting all this stuff written down provides a really cool backup for your design documents. Much of the time, design documents don't necessarily include all the interactions of the various pieces of the system (often they do, but...). Threat modeling forces you to write those aspects of the design down. It also forces you to think about (and write down on paper) the design of your component in terms of the DATA manipulated by your component, instead of the CODE that makes up your component.
Btw, Frank Swiderski and Window Snyder have an excellent book on threat modeling, it's a bit dry reading, but it's really invaluable for learning about the process. Microsoft has also provided a threat modeling tool (written by Frank) here that can be used to help guide you through the process of developing the threat model. Microsoft's been working hard internally at making the threat modeling process require less effort - the ultimate goal of the team is "Do the DFD, turn the crank!".
There are also lots of fascinating resources on the web available, including Dan Epp's article here, and Krishnan Ranganathan's threat model checklist here. In addition, Michael Howard's written a bunch about TM, here and here.
Edit: Included some additional information and links for Michael Howard's TM articles.
Edit2: Corrected possesive on Dana Epp's name 🙂