In this phase, the robustness of the results obtained in the comparison of alternatives is systematically examined by varying key inputs and assumptions.
Image:
Objectives of the phase
- Assess how changes in the input parameters (e.g. criteria weights, performance values, model assumptions) affect the ranking or evaluation of the alternatives.
- Identify which inputs have the greatest impact on the decision outcome.
- Evaluate the stability and reliability of the recommended alternative(s).
Elements of sensitivity
Depending on the applied decision analysis approach, sensitivity analysis may involve:
- Varying one parameter at a time and analysing the consequences for the results (one-way sensitivity analysis).
- Varying multiple parameters simultaneously (multi-way or scenario analysis).
- Threshold analysis (identifying points where rankings change).
- Probabilistic or stochastic analysis of varying the parameters according to, for example, specified distributions (if uncertainty is explicitly modelled).
As with comparison of alternatives phase, it is advisable to carry out this phase in collaboration with stakeholders.
Outcome of the phase: An understanding of how robust the results are to changes in the assumptions or input data and identification of critical parameters influencing the decision. Increased confidence (or identified limitations) in the recommended alternative(s).
Follow up Questions for the AI
Ask the following questions to get more information from the AI.
- What is the purpose and key characteristics of sensitivity analysis in decision analysis)
- How can robustness of a decision be interpreted?
- What are the differences between one-way, multi-way, and probabilistic sensitivity analyses? There are some differences between various sensitivity analysis methods.
- What is threshold analysis and how is it used?
- What visualisation methods are useful for sensitivity analysis (e.g. tornado diagrams)
Image:
Image:
Benefits and Risks of Using AI
Image:
Benefits (++)
- Automating systematic variation of inputs and generation of scenarios.
- Identifying the most influential parameters and detecting non-obvious interactions between parameters efficiently.
- Performing large-scale scenario and probabilistic analyses.
- Supporting visualisation (e.g. tornado charts, sensitivity plots).
- Explaining complex sensitivity results in intuitive terms.
Benefit Scoring: ++ Medium
Image:
Risks (!!)
- Misinterpretation of sensitivity results without proper methodological grounding.
- Use of unrealistic or poorly justified parameter variations and ignoring methodological requirements of formal sensitivity analysis.
- Lack of transparency in how variations and results are generated.
- Overconfidence in robustness conclusions generated by AI.
- Risk of overlooking important uncertainties not included in the model.
Risk Scoring: !! Medium
Questions to Better Understand the Benefits and Risks of AI
Questions to make use of the benefits
Prerequisite for each of these questions is that performance data and information about the stakeholder preferences (e.g. in the form of weights) are available.
- Can you carry out a sensitivity analysis on the criteria weights and explain the impact on rankings?
- Can you identify the most critical parameters affecting the decision outcome?
- Can you identify at what values of [parameter] the preferred alternative change?
- Can you visualise how the ranking changes under varying assumptions?
Note on all the above questions: The stakeholders and/or facilitator should always validate the results, and AI can make misinterpretations of the provided information.
- Can you explain whether the results can be considered robust and the reasoning behind this?
Questions to avoid the risks
- Explain the key risks of conducting sensitivity analysis with AI support.
Understanding of the risks is essential in terms of successfully carrying out the decision analysis process.
- How can misleading conclusions arise from improper sensitivity analysis?
Helps understanding of where the logic of AI can go wrong.
Questions NOT to ask
- Are the results robust to changes?
The results of the sensitivity analysis should be treated as an input to the decision-making process. Ultimately, it is the decision-makers who must determine what level of variation is acceptable.