Threat Modelling Machine Learning Systems
As companies progressively seek to employ new forms of technologies into their operational frameworks, it also exposes them to vulnerabilities. While vulnerabilities depend upon the use case, how does one evaluate them?
There exist a host of scoring systems that can help quantify the impact of a potential attack. These Threat modelling Machine Learning systems act as the benchmark to evaluate threats, based on the use case. Based on the results of the benchmark, companies can make sound quantitative decisions to mitigate these risks.
Understanding Threat Modelling
Threat Modelling involves adopting a structured approach to identifying and assessing vulnerabilities. It is imperative to adopt a structured and drawn-out process to move past vulnerabilities. Threat modelling begins by first identifying the threat actors that act as the source of the vulnerabilities. Threat actors range from hackers to nation-state activists. These threat actors have different agendas to orchestrate attacks of such nature. It could be based on revenge for a past incident or to obtain confidential information that isn’t meant for everyone to access.
An optimal defense strategy for Machine Learning systems has to be based off of the CIA triad. The CIA triad is an information security model which is inclusive of the following: -
The CIA triad mandates the kind of attacks that could be administered and safeguards can be put in place based on this.
Attack types on ML systems have an evolving nature. Do note that there are designated attack types that have been successful in infiltrating these systems, such as Evasion attacks and Poisoning attacks. However, as ML models evolve to combat these attacks, so do the attacks themselves.
The question that arises here is how can entities using ML models decide which vulnerabilities need immediate attention and which one don’t?
Common Vulnerability Scoring System (CVSS)
CVSS is an open framework that quantifies the features and depth of severity of a vulnerability. The CVSS evaluates the vulnerability based on 3 metric groups, namely Base, Temporal and Environmental. These 3 metric groups make the CVSS results quantifiable. Here is a brief overview of the 3 metric groups: -
1. Base Metrics:
The Base metrics evaluates the qualities of the vulnerability. These metrics produce a score ranging from 0 to 10, which is most relied upon by companies. Base metrics is further split into Exploit-ability metrics and Impact metrics. Since the features of the vulnerability do not evolve over time, base scores provide a starting point for patching prioritization.
2. Temporal Metrics
Temporal CVSS Metrics pertain to vulnerabilities that evolve over time. Temporal metrics focus on how the vulnerability exploits the current framework. Based on this, it assists companies in evaluating the availability of remediating controls.
3. Environmental Metrics
Environmental Metrics build up on the scores obtained from the Base metrics. Based on the Base Metrics score, companies can modify these Base CVSS scores as per security requirements and modified base metrics.
Practical Examples of CVSS Models
Let’s take a lot at some practical examples of CVSS models: -
1. Malware Classifiers against Poisoning
In an attack of this nature, data sets first become corrupted and are classified as malwares. This directly affects the Integrity of the system as it misclassifies malware. The malware is assumed to be a legitimate file, which allows it to spread as it goes undetected.
A breakdown of the CVSS score of this attack will look like this: -
2. Patient Systems In Healthcare Against Model Extraction / Inversion
Threat of this nature directly affects the Confidentiality of proprietary models used to handle patient data in healthcare. An attack of such nature is more plausible to be used by competitors to get their hands on sensitive patient data. Once they break into the healthcare ML models of their competitors, they can gain a competitive edge by leveraging the sensitive information for nefarious purposes.
A breakdown of the CVSS scores of this attack would look like this: -
From the above examples, while they clearly had the same total CVSS score, their separate use cases clearly indicate that protecting the system from malware classifiers is more important than poisoning.
Final Thoughts
No vulnerability has to be taken lightly. As systems get more sophisticated, a grasp of the flaws and the system itself is imperative in developing good information security governance. The CIA triad’s universal applicability is something that no enterprise should overlook and it is essentially a pre-requisite to mitigate threats. Therefore, while companies might see implementing ML models as the ultimate step in gaining a competitive edge, it is clearly not as it is the first step.