Statistical methods of interest: Sensitivity Analysis

Consider a situation where you are studying a model with an unknown function (often referred to as a black box function) and input many variables, and you want to understand how those input variables affect the output. Sensitivity Analysis is something you should then definitely consider using. 

In this blog post, we'll dive into the core concepts behind sensitivity analysis and explore some common methods used in the field.

Sensitivity Analysis examines how the uncertainty of the input variables contribute to the uncertainty of the output. Typically, it's conducted once uncertainties in all variables have been precisely quantified.Through this analysis, you can gain deeper insights into the relationship between input and output, helping identify the most impactful variables. Removing the least relevant variables will reduce dimensionality and simplify the model. 

There are several type of approach to sensitivity analysis (Iooss, Bertrand, and Paul Lemaître., 2015). Here are some of the most commonly used:

• One-at-the-time (OAT):  the most commonly used methods. Here the sensitivity is computed by running the model several times, each time one variable varies and the others are kept fixed. Running those types of methods can often be computationally demanding, especially for large numbers of input variables. The Morris method offers a way of dealing with high dimensionality. By sampling the model's input space more systematically, it provides a more comprehensive understanding of input-output relationships.

• Variance based methods: These methods decompose the variance of the output to determine the contribution of each input variable. They offer an intuitive framework for understanding how uncertainties in the inputs are propagated to the outputs. Sobol indices are a well known variance based method that offers a way to compute not only the effect of a variable on the output (main effect), but also the effect of the interaction between input variables.

• Distance based sensitivity analysis:  It is used to determine the influence of input variables on a model's output by examining the distances or differences between input-output pairs.This approach assesses how small changes in input parameters lead to changes in model outputs by comparing the "distance" between them in the input-output space. The method allows for the evaluations of main effects and interactions.

• Monte Carlo filtering: The method relies on monte carlo simulation to generate random samples of the input variables, and then evaluate which input has the largest impact on the output.

The choice of the right method really depends on the type of problem we are trying to solve and on the information we have available.

At Praxis Security Labs, we are always finding innovative solutions for our customers, and sensitivity analysis is one approach we use to help them get the most out of the data collected.

To know more about what we do and how, schedule a meeting with our CEO, Kai Roer, or with our Director of Research, Thea Mannix.

References: Iooss, Bertrand, and Paul Lemaître. "A review on global sensitivity analysis methods." Uncertainty management in simulation-optimization of complex systems: algorithms and applications (2015): 101-122.