Application Threat Modelling

Being truly ‘Secure-by-Design’

Prashanth B.P., Security Architect (prashanth@fortian.com.au) | August 23, 2021

Introduction

Imagine developing a new mobile application for your company. Going through the entire development cycle of identifying functional and non-functional requirements, coding, and testing. And then in one of the post-development security tests (say, in a SAST scan), the scrum team ‘discovers’ that an important use-case has been left open to potential vulnerabilities, which leads to a major re-design (read ‘re-work’) of the application. Such a scenario is fairly common. In fact, traditionally, the application security domain has focused on “secure-by-test” approach by employing a plethora of scanning tools: Static Application Security Testing (SAST), Dynamic Application Security Testing (DAST), and the latest Interactive Application Security Testing (IAST). While these tools are very important to detect potential security vulnerabilities in the application, it is important to adopt an approach which helps to prevent these vulnerabilities occurring in the first place.

So, let’s take a look at how we can make the transition to “secure-by-design” rather than “secure-by-test”!

Firstly, what does “secure-by-design” mean?

While “secure by design” can cover a host of aspects like governance, policies and enterprise security architecture tools, in this article we will specifically focus on how to implement the concept of ‘secure-by-design’ in the application security domain, with a focus on threat modelling.

Threat modeling is the core to the concept of secure by design. In my view, it is one of the best ways of assessing the security posture of applications. In this blog post, I will outline what threat modelling is, its benefits and an example of how to develop a threat model for an application.

What is threat modeling and its advantages?

Threat modeling is a structured approach to identify, quantify and address the security risks associated with an application. It ensures that applications are developed with in-built security right from the inception phase of SDLC.

Creating a threat model for an application during the design phase has several advantages:

  • Threat(s) identification in the early stages of development lifecycle.
  • The ability to clearly visualize and understand risk, including:
    • The possible attack vectors on an application and the data;
    • The risk level of each potential threat; and
    • Action that needs to be planned to remediate the threat.
  • An increase in security awareness within the development teams.

Popular Threat Modeling Frameworks

A threat model can be created in multiple ways. There are many frameworks that have been developed over the years. The choice of framework depends on the focus of the project such as risk, security and privacy. Here are some of the popular frameworks used to create threat models:

  1. STRIDE and its associated derivations

  2. STRIDE (which stands for Spoofing, Tampering, Repudiation, Information disclosure (privacy breach or data leak), Denial of service and Elevation of privilege) was invented in 1999 and adopted by Microsoft in 2002. It is one of the most mature threat modeling methods. It has an open-source version and is widely adopted in the developer community.

    More information on the STRIDE framework.

  3. Attack Trees

  4. One of the oldest ways and mostly widely applied techniques is using attack trees to model threats. Here the root of the tree are the goal of the attack and leaves are ways to achieve that goal. Each tree represents different goal.

    More information on attack trees

  5. PASTA

  6. The Process for Attack Simulation and Threat Analysis (PASTA) is a risk-centric threat-modeling framework developed in 2012. It employs an attacker-centric perspective to produce an asset-centric output in the form of threat enumeration and scoring.

    More information on the PASTA framework

  7. LINDDUN
  8. LIDDUN (Linkability, Identifiability, Nonrepudiation, Detectability, Disclosure of information, Unawareness, Noncompliance), focuses on privacy concerns and can be used for data security.

    More information on LINDDUN framework

  9. Persona non Grata (PnG)
  10. PnG focuses emulates the behavior, motivation and skill of a human attacker. The idea is to introduce the potential attacker and their point of view.

    More information on Persona non Grata framework

  11. Security cards
  12. Security cards are more of a brainstorming approach with the help of a deck of 42 cards to facilitate threat-discovery activities.

    More information on security cards

And there are many other techniques…

No matter what tool you choose or approach you take, developing a threat model will generally involve the following steps:

Let’s dive into each of these steps and understand it in some detail.

Creating a STRIDE Threat Model

Let’s take the STRIDE threat modeling technique and emulate the steps involved in the process of creating a threat model as this is one of the most popular and mature threat models used.

The STRIDE technique provides classification of threats in six categories.

STRIDE means IMPACT DESCRIPTION
Spoofing Authentication Posing as something or someone you’re not.
Tampering Integrity Modifying something you’re not supposed to modify. E.g - malicious code identification: It can include packets on the wire (or wireless), bits on disk, or the bits in memory.
Repudiation Non-Repudiation Claiming that you didn’t do something (regardless of whether you did or not). Can the actions be traced?
Information Disclosure Confidentiality Exposing information to people who are not authorised to see it. Can the information be protected?
Denial of Service Availability Attacks designed to prevent a system from providing service, including by crashing it, making it unusably slow, or filling all its storage.
Elevation of Privileges Authorisation A program or user is technically able to do things that they’re not supposed to do.

For this blog, I will use the ‘Microsoft Threat Modeling Tool’ as an example, since this is the one of the oldest and most tested open-source threat modeling tools in the market. It is based on the STRIDE technique and uses the standard notation for visualizing components, data flows, and security boundaries.

Download the Microsoft Threat Modeling Tool

As an example, lets create a threat model for a school with the requirement that students and staff can login remotely and access their workstations. Student gain access through a VPN and staff log in from a dedicated browser interface. The internal financial application is in a separate network segment, accessible to only authorised individuals.

Step 1: Identify components

A draft model that identifies components is often developed during the solution design phase during a brainstorming session between the security architects, senior developers and project manager (and other relevant stakeholders).

Symbols used in creating a threat model in STRIDE:

1. Start by identifying the most interesting elements of the solution

  • Assets, entry and exit points

In our example the various entities would be:

2. Identify the Trust Boundaries

3. Identify all process flows between the various elements

  • Identify all the data flows and data-stores where the data will finally reside.

4. Enumerate assumptions and dependencies

5. Create a high level Data Flow Diagram (DFD), after detailed discussion about the above entities.

Step 2: Identify threats

Once a high-level DFD is created, the Microsoft Threat Modeling Tool can be used to provide a detailed report on ALL POSSIBLE threats whether mitigated or not.

The report should be analysed with security architects, senior developers and relevant stakeholders.

Some of the possible threat scenarios are as follows:

Spoofing: Impersonating another person/process
Threat scenario Possible threats by an attacker
Spoofing a process on the same machine Creates a file before the real process
Renaming/linking
Renaming
Spoofing a file Creates a file in the local directory
Creates a link and changes it
Creates many files in the expected directory
Spoofing a machine ARP Spoofing
IP Spoofing
DNS Spoofing
DNS Compromise
IP Redirection
Spoofing a Person Sets e-mail display name
Takes over a real account
Spoofing a role Declares themselves to be that role

Tampering: threat scenarios
Threat scenario Possible threats by an attacker
Tampering with a file Modifies a file they own and on which you rely
Modifies a file you own
Modifies a file on their file server
Tampering with memory Modifies your code
Modifies the data they can control to an API
Tampering with network Redirects the flow of data to their machine
Modifies data flowing over the network

Repudiation: threat scenarios
Threat scenario Possible threats by an attacker
Repudiating an action Claims to have not clicked
Claims to have not received
Claims to be a fraud victim
Uses someone else’s account
Uses someone’s payment instrument (PoS) without authorisation
Attacking the logs Notices you have no logs
Put attacks in the logs to confuse logs or corrupt the logs

Information disclosure: threat scenarios
Threat scenario Possible threats by an attacker
Information disclosure from a process Extracts secrets from error messages
Reads the error messages from username/passwords to entire database tables
Extracts machine secrets from error messages
Extracts business/personal secrets from error cases
Information disclosure from data stores Takes advantage of inappropriate or missing ACLs
Takes advantage of bad database permissions
Finds files protected by obscurity
Finds crypto keys on disk (or in memory)
Sees interesting information in filenames
Reads files as they traverse the network
Gets data from logs or temp files
Gets data from swap or other temp storage
Extracts data by obtaining device, changing OS
Information disclosure from a data flow Reads data on the network
Redirects traffic to enable reading data on the network
Learns secrets by analyzing traffic
Learns who’s talking to whom by watching the DNS
Learns who’s talking to whom by social network info disclosure

Denial of Service: threat scenarios
Threat scenario Possible threats by an attacker
Denial of service against a process Uses memory (RAM or disk) or CPU
Uses a process as an amplifier
Denial of service against a data store Fills data store
Makes repeated requests to slow down the system
Denial of Service against a data flow Consumes network resources

Elevation of privileges: threat scenarios
Threat scenario Possible threats by an attacker
Elevation of privilege against a process by corrupting the process Send inputs that the code doesn’t handle properly
Gains access to read or write memory inappropriately
Elevation through missed authorization checks Identifies a loophole in the authorization where authorization is not implemented or is not sufficient
Elevation through data tampering Modifies bits on disk to do things other than what the authorized user intends

Step 3: Address threats

The identified threats can be dealt with in different ways based on the risk appetite of the organisation. Possible ways to address the identified threats include:

  1. Take no action
  2. Inform stakeholders about the risk (potentially in combination with other actions)
  3. Mitigate the risk
  4. Accept the risk
  5. Transfer the risk
  6. Eliminate the risk

Step 4: Validate

Once a detailed analysis of all the threats has been done and list of threats to be mitigated has been identified, the development/validation teams can validate the threats in different ways during the various phases of SDLC. The developers perform code reviews and, once the code is ready, testers develop specific tests to check if the application code addresses the threats identified.

Let’s see this in bit more detail:

Code Review

Code review can be done in either as a peer-review or by scanning the application source code for security vulnerabilities. As mentioned earlier, there are various application scanning tools (SAST/IAST) which can be used for this purpose. If the application code addresses the threats identified earlier, the scanning tools will not highlight them as vulnerabilities.

Test Planning

Test planning should ensure that the list of threats is used as a base for test planning. It’s important to ensure that tests cover all the possible scenarios highlighted in the threat model.

Penetration Testing

Penetration Testing is a manual testing process in which pen testers simulate the attack against the application in real time. The threats identified during the design phase act as the input for this process.

Conclusion

A true shift-left in application security is achieved by adopting threat modelling as a necessary step early in the SDLC. This is crucial for critical applications that handle the core business requirements of an organization, for instance financial/critical transactions, confidential data etc. It is important to carefully select the threat modelling tool and this choice is influenced by the threat modelling framework which the organization wants to focus on. During the design and planning phase, threat modelling encourages defence-in-depth and structurally sound security controls. A high-level threat model should be applied while designing the application and refined throughout the lifecycle. This helps the team to address the threats continuously while the project is ongoing.