Threat Modeling without Security Subject-Matter Expertise


With this release of the process and the accompanying tool, there were 4 high-level objectives we are trying to fulfill:


 



  1. Provide a consistent methodology for objectively identifying and evaluating threats to applications.
  2. Translate technical risk to business impact.
  3. Empower a business to do effective application risk management during the SDLC and beyond.
  4. Create awareness among teams of security dependencies and assumptions.

I wrote about the challenges that lead us to our first objective (at the end of the post) and will be writing about the other three later. What I wanted to talk about here was a theme central to our process. This being the ability to produce feature-rich threat models with a process that meets these objectives without requiring security subject-matter expertise.


 


We’ve been doing security consulting work (code assessment, training, design reviews, etc.) internally in Microsoft for almost half a decade now. Threat modeling is one of the more recent activities we pushed out in an effort to make application teams more pro-active about security. One of the key observations we’ve made (especially with v1 of threat modeling) is that application development teams do not want to be security subject-matter experts (SME) – it’s not their job! (before you gasp for air, allow me to finish… J).


 


One doesn’t have to be a security SME to write secure software.


 


The first thing to point out here is that there is a fundamental difference between becoming a security SME and being security aware. When building a house, you have various professionals involved in the process. Here, you wouldn’t expect the carpenter to be able to lay the bricks or the either of them to be able to do the plumbing. But you still expect everyone involved in the process to build a structurally sound house at the end without any of them being structural engineers. Structural soundness is just one attribute of the house along with energy efficient, appealing, etc.


 


When building a software application, the same kind of thing applies. Development teams have to worry about a lot of things such as scalability, usability, performance, accessibility, cost, etc. and they have various professionals involved (architects, developers, testers, etc.). Security is just another attribute they have to worry about. Granted that depending of the application being built, security may take greater or lesser (yes, believe it!) significance than other attributes. It’s our job, as security SMEs, to empower the development teams in their quest to build secure applications by making them more security aware.


 


With our threat modeling, we’re trying to empower the business to do effective application risk management. Threat modeling is essentially about creating specific instances of the following structure:


 




 


Now defining the threat is the job of the development teams (because it’s their application and their business) but they can’t define or know what kind of attacks are going to be used to realize those threats (or the vulnerabilities used to materialize these attacks or the countermeasures used to mitigate these vulnerabilities). There is a separation of duties here and what we needed to do (and did in V2) is acknowledge that. The development teams need to model their application and identify the threats inherent in it. After that, they utilize these known attacks, in the form of the Attack Library, that is created and verified only by security SMEs. This Attack Library is a very fundamental feature of our process and is meant to be extensible (more on this in a later post).


 


Threat Modeling can’t replace other security activities like code reviews, design reviews, training, etc. What it can and does do is to complement these activities. Our job, as security SMEs, is to ensure that the threat models are optimized and are complete and that appropriate up-to-date guidance (through Attack Libraries) is being followed. But at the end of the day, threat models need to be created by application development teams in order to define their threat profile that are not – and will never be because it’s not their job – security subject matter experts.


 


-Talhah Mir


Comments (4)

  1. axleyjc says:

    I agree at a high level with the relationship you have defined above between Threat, Attack, Vulnerability, and Countermeasure.  However, I don’t think that your v2.0 tool embodies this model.

    The new tool has a completely flat threat hierarchy (ripping out the attack tree model in v1) and within a particular threat, there is no definition of anything other than the Attack Countermeasures.  To be consistent, each threat should be decomposed into multiple possible Attacks that can realize the threat (the Tree part of an Attack tree) so that you can then define whether or not there is a Vulnerability associated with the Attack (if not, then that attack vector is irrelevant) and from there, what the possible countermeasures are.  The attack tree model allows for chains of countermeasures or vulnerabilities to be defined as well so that you can detect a weakness if any part of the chain does not hold.  The current tool’s flat model does not have this richness so is left up to guessing to know that a) you have all of the possible attacks for a given threat and b) you have all of the possible countermeasures for each attack defined.

    How was it that this model got lost in the implementation of the new tool?  And where did threat trees go?

    I understand not wanting to put so much complexity in front of the business, but the current alternative is to handcuff the security person trying to actually _model_ the threats.  You don’t _model_ the threats in the new tool so much as you _list_ them.  I call it an "attack pile" instead of an "attack tree".

    There is a lot of potential with the tool but these core decisions concer me as a security SME wanting to use the tool to model threats and countermeasures in ways that is comprehensive and easy to communicate to the business why something is or is not to be worried about.

  2. talhahm says:

    Hi axleyjc,

    The threat-attack-vulnerability-countermeasure relationship is there and can be viewed form the Threat Tree Visualization. In it, for each threat, you’ll see the relationships that bind each threat to a given countermeasure. We did this cause from an app team’s perspective, the vulnerabilities and attacks are not important – what they care about is that they have a potential problem that the business wants to avoid (threat) and they need to know how they can potentially do this (countermeasure). We didn’t get rid of the relationship, just pushed it to the background.

    The attack library we define in the tool is more of a blueprint for an attack rather than an attack tree. Attack trees are very valid but as you correctly pointed out, they can really only be correctly built by security SMEs with a good understanding of the underlying architecture and technology in the application.

    Here’s another way you wanna look at it: a SQL Injection attack can be realized through many MANY attack vectors all depending on the underlying technology and architecture. But from a defensive perspective, if you don’t have dynamic SQL in your code, you can’t have SQL injection (regardless of the vector). That’s what we’re doing, looking at the problem from a defensive perspective.

    But there is value in looking at the problem from an adversarial perspective as well (especially when you are doing penn testing) to help prioritize countermeasure and get a better analysis or the security posture and this is a piece we’ll be adding on top of v2 to enable risk analysis & modeling (what I referred to in on my blogs as “Typhoon”). Also, keep in mind it’s next to impossible to build effective attack trees from just business requirements and a high-level architecture – good attack trees require an in-depth understanding of the implementation.

    Thanks.

    -Talhah

  3. Weddings says:

    With this release of the process and the accompanying tool, there were 4 high-level objectives we are trying to fulfill: Provide a consistent methodology for objectively identifying and evaluating threats to applications. Translate technical risk to busines

Skip to main content