Threat Modeling, part 1

One of the requirements for designing software at Microsoft these days is that every (and I do mean EVERY) feature in every product needs to have a threat model.

Threat modeling isn't new, it's been a part of good design for years.  Heck, the IETF has a threat model for the entire internet 🙂 

Note that I didn't say good security design, IMHO, threat modeling isn't necessarily about security design.  Threat modeling's really all about understanding your design, at a level that design documents don't necessarily cover.  

Threat modeling is almost more of a discipline than an analysis tool- if it's done right, it enables you to gain a much deeper understanding of your components and how they interact.  And by applying that methodology, you learn a great deal about your component.

We're currently running through the threat modeling process in my group.  Since I'm the security nut on my team, I'm the one who volunteered to run the process.  And it's really been an interesting exercise..

The big thing that writing the threat model for your feature (or component, or protocol, or whatever) is that it really forces you to understand the interactions between the various pieces of your feature.

When threat modeling, you need to concentrate on a bunch of things.  First, there's your assets (also known as protected resources).  All threats relate to assets that need to be protected.  If you don't have assets, you don't have threats.  But sometimes the things that are your assets can be quite subtle.  For example, the audio samples being sent to a sound card might be an asset (especially if the audio samples came from DRM'ed files).  So might the privileges of the LocalSystem (or the current user) account.  The contents of a file on the disk might be an asset for a filesystem, similarly, the contents of an email message is an asset for an email system (like Exchange).  Understanding your assets is key - assets tend are attacked in different ways, the attack strategies mounted against an email message probably won't work for transient data like an audio sample.

The next thing you need to determine is the entry points to your components.  The entry points to your components are what the attacker is going to use to gain access to your component.

Related to entry points, one other thing that you need to determine are the trust boundaries in your component.  A trust boundary is a boundary (physical or virtual) across which there is a varied level of trust.  For example, when you RPC from one machine to another (or one process to another), that forms a trust boundary.  Another example of a trust boundary is the network - almost by definition, "the wire" is a trust boundary.

For some systems, there are no real trust boundaries - an interactive application running as the user that only outputs data may have no trust boundaries.  Similarly, an API that processes data and returns it to the caller may have no trust boundaries.

Related to trust boundaries are trust levels - trust levels indicates how much you "trust" a portion of the system.  For instance, if a user is an administrator, they are trusted to do more than normal users.  When data flows between entities whose trust level is different, by definition, there is a trust boundary between those two entities.

Once you've identified your entry points, assets, trust boundaries, and trust levels, the next major part of a threat model is the data flow diagrams.  A data flow diagram indicates the flow of data into and out of the various parts of your components.

All of this is the up-front work.  It lays the foundation for the "meat" of the threat model, which, of course are the threats.  I'll write more about threats tomorrow..

One of the things that I realized while writing down all the stuff above is that getting all this stuff written down provides a really cool backup for your design documents.  Much of the time, design documents don't necessarily include all the interactions of the various pieces of the system (often they do, but...).  Threat modeling forces you to write those aspects of the design down.  It also forces you to think about (and write down on paper) the design of your component in terms of the DATA manipulated by your component, instead of the CODE that makes up your component.


Btw, Frank Swiderski and Window Snyder have an excellent book on threat modeling, it's a bit dry reading, but it's really invaluable for learning about the process.  Microsoft has also provided a threat modeling tool (written by Frank) here that can be used to help guide you through the process of developing the threat model.  Microsoft's been working hard internally at making the threat modeling process require less effort - the ultimate goal of the team is "Do the DFD, turn the crank!".

There are also lots of fascinating resources on the web available, including Dan Epp's article here, and Krishnan Ranganathan's threat model checklist here.  In addition, Michael Howard's written a bunch about TM, here and here.

Edit: Included some additional information and links for Michael Howard's TM articles.

Edit2: Corrected possesive on Dana Epp's name 🙂

Comments (12)

  1. Anonymous says:

    What level of granularity is a "feature" defined at? Is it every exposed function in a library? Is it every button in a GUI? How do you relate those to a class’s private data and public member functions? Does it change for every type of programming being done?

  2. Anonymous says:

    I’ve been on the road for the last year teaching a workshop on .NET security programming best practices to ISVs.

    In the first part of the workshop, I spend a good 4 hours reviewing Thread Modelling and even demo Frank’s tool.

    What’s amazing is the collective yawn of cynisism that spreads through the entire room. It’s gotten so bad that I now save these discussions for the end of the workshop, and I’ve reduced my TM content.

    What I have discovered is that while most programmers and architects are whole-heartedly behind threat modelling, most project managers and business managers don’t understand why it is necessary. Most developer project managers believe that security is up to the "networking" folks and don’t budget the time up front for security threat modelling. As a result, it never gets done.

    I think selling threat modelling to programmers is the wrong audience.

    It seems that we need to be evengelising threat modeling in MSF to project management folks rather then programmers.

  3. Anonymous says:

    Good post! I’m trying to get people to understand that TM should be part of any design. Hey, even really small designs would benefit from a TM. For some reason people think that any TM would take months to complete and involve a whole team of security people. I guess my view is that you only need a small TM for a small design. Hey, surely a small TM is better than none! =)

    Thanks for the links!

  4. Anonymous says:


    That’s a good question. In general, products are built from features, features are built from components, components are built from files and libraries.

    But it’s your call where you want to do your threat model, it’s what "makes sense".

    You don’t want to do something huge. For example, the threat model for "Windows" would be unmanagably large, while the threat model for "Windows CIFS Filesystem" might be more reasonable (maybe not, I’m not on that team).

  5. Anonymous says:

    Good post Larry. And thanks for the link. Even if you did spell my name wrong 😉

    Andrew, I’m sorry to hear of the troubles you are having with TM at that level. It’s really too bad though. I can agree to an extend that more education to PM is required for TM, but I disagree that it shouldn’t be programmers who drive the content in the modeling session.

    There is a reason for that. When looking from the outside in, to properly utilize threat modeling you have to think like the attacker. And its much easier to do that when you understand the design. And you will only be able to mitigate risks once you know what they are. Hard to do that when you are a manager at such a high level. The guys in the bowels of the architecture will be far more informed and interested in making it happen.

    Sometimes I joke about it, but having the programmers doing the threat model has an added benefit of covertly changing the development mentality of the company. It’s hard to do a top down approach for secure software engineering when the higher level managers rarely see the ROI in doing so. (even though it has been shown in numerous studies over the years) Yet if it’s just another task like doing functional design specs or checking in code, it is somewhat more accepted. In this day and age responsibility in secure design should come from everyone. But the buck stops with the developers, and the threat model should be made mandatory at that level.

    The fact you are finding development PM believing threat modeling is for the network folks show me they don’t truly understand their own responsibilities in the mix of things. Have them thumb through the "Threat Modeling" book from Microsoft Press and tell me that the examples there don’t clearly cover off the fact that a model can be applied to ANYTHING, from hardware to software applications, and communications in between. If you are teaching this stuff I know you get this. I wonder what the right way would be to get this across to them though.

    Michael Howard uses a good approach of using good real-world examples of how things have failed in the past, and how threat modeling may very well have found the vulnerability before it became a problem. Nothing wakes a crowd up like hearing about real world blunders in the world. Heck I should know… I talked about my own screw ups in the post Larry linked up.

    Good luck in finding how to deliver the content to the PM. If all else fails, beat them with the Threat Modeling book. Maybe they will learn through osmosis.

  6. Anonymous says:

    the band plays in dealing truth to the liars & criminals universally.

  7. Anonymous says:

    A threat model can literally be applied to anything. Well if it has all of the ingredients, assets, entry points, trust levels, and attackers. The main problem I’ve found in threat modeling is the assets. Things one person considers an asset might not be considered by another person. Assets also can creep into your code from another location like a file on the file system. Sure your component looks for files but you typically don’t think about things like that.

    I’m in small business IT and I’m using threat modeling to help me with security in our network. I’ve started a model for our CRM application we use (SalesLogix) which has a couple of vectors that can be exploited rather easily. This helps me understand what security measures are in place and what I can do to make them a little stronger.

    Confining threat modeling to just programming practices put it inside a very small box. More professionals should be using threat modeling than currently do. If you’re a dba you should have a threat model of your databases, servers, and software. If you’re a network admin you should have a threat model of your network. If you’re a coder you should have a threat model of your components and applications. Threat modeling simply causes you to think about the security implications on what you’re doing and even if your company follows security a threat model will get you thinking about what is important. It’s by no means a holy grail for security but it comes pretty dang close.

  8. Anonymous says:

    Absolutely Jeremy, you’re totally correct – I tend to get caught up in my world without seeing the bigger picture.

  9. Anonymous says:


  10. Anonymous says:

    Way back in the day, back when we were doing the initial threat models for the vista audio stack, I wrote…

  11. Anonymous says:

    About 2.5 years ago, I wrote a series of articles about how we threat model at Microsoft, about 18 months

Skip to main content