The whole is greater than the sum of the parts, say Aristotle and Gestalt psychology. The whole effect of combined factors of a model in terms of sensitivity may also be greater than the effect of any individual effect of a factor. In other words, if you’re trying to find out what contributes the most uncertainty to a model, you may be missing something if you only look at the effect of a single factor. Yet given that models and their variations need to be understandable to different audiences, how complicated should you make your sensitivity analysis anyway?
Image source: 8020world.com
What is sensitivity good for?
The short version is model understanding, model development, decision making, and communication. By varying one or more factors, deeper understanding can be developed of the relationship between input and output. The model can also be developed to become more correct, simpler, and more robust if data is missing or of poor quality. Priorities can be assigned in terms of data acquisition. Decision making can be improved by identifying how and when optimal solutions change, or producing flexible recommendations that take account of changing circumstances. Communication of these decisions using the results of sensitivity analysis can be made more understandable or more credible.
One at a time variation in sensitivity analysis
The simplest way to approach sensitivity analysis is to vary each factor one at a time (OAT). The big advantage is that any changes (including model failure) observed can be ascribed to the change in that one factor. While one factor is being varied, the others are usually kept at their nominal or baseline values, so that each OAT change is effected with reference to a certain model ‘base space’.
Limitations of an OAT approach
Image source: commons.wikimedia.org
Varying one factor at a time does not allow for identification of correlation between variables. In some contexts, such correlation or interaction may be considerable. In business for instance, a supply chain is often an interlinked system of input factors with certain individual and collective constraints. Likewise, revenue models will depend on sales volume and sales pricing, but these two factors are also often interlinked. The elasticity of a market means that a business will sell more products at a lower price per product, subject again to certain constraints such as the need to make a per-product profit.
Combinations of factors
Depending on the size of the model and the number of factors, testing different combinations of factors for sensitivity may be realistic. When running simulations of models using Monte Carlo techniques, Latin hypercube sampling may represent the best compromise between efficient coverage of different combinations and realistic assessment of their combined effect on sensitivity. Alternatively, sensitivity analysis can also be performed on sub-models of an overall model to see the effect of particular combinations of factors.
Models of models
Moving up a level, different models can be run in response to the same problem or question, and yield different results. Some models may yield very different conclusions from the rest of the models. The sensitivity of overall conclusions to such models can in turn be assessed: for instance, by observing the effect of including or excluding the model in the overall spectrum of results obtained.
If you’d like to know how Analytica, the modeling software from Lumina, can help you do sensitivity analyses and automatically rank factors in order to sensitivity importance, then try the free edition of Analytica to see what it can do for you.
