In this phase the performance of the different alternatives is systematically evaluated and compared across the selected criteria.
Image:
Objectives of the phase
- Assess and compare how well each alternative performs with respect to each criterion.
- Compare the overall scores of the alternatives obtained by aggregating the performance information of the alternatives with respect to criteria and stakeholder preferences.
- Identify trade-offs between alternatives and possible dominance relationships.
Elements of comparison
- Simple scoring and ranking of the alternative.
- Compare the results of aggregated multicriteria models (e.g. weighted sums, value functions).
- Pairwise comparisons of alternatives.
- Dominance or outranking analysis
The alternatives are typically compared together with stakeholders so that the facilitator presents the results and explains the reasoning behind them to increase understanding of the strengths and weaknesses of the alternatives and of the trade-offs to be made when selecting the alternatives. The results are typically visualised, for example, with stacked bar graphs or radar charts to illustrate the contributions of different criteria to the results
Outcome of the phase: A structured comparison of alternatives (e.g. overall scores or ranking of the alternatives, or dominance relations) and identification of the most preferred alternative(s) from different viewpoints.
Follow up Questions for the AI
Ask the following questions to get more information from the AI:
- What is the purpose and key characteristics of the comparison phase in decision analysis?
- How does the comparison of the alternatives differ between different methods (e.g. ranking, scoring, and outranking methods)?
There are fundamental differences between the methods in terms of how the results are presented. - Can you describe and illustrate typical visualisation techniques for comparing the alternatives?
Image:
Image:
Benefits and Risks of Using AI
Image:
Benefits (++)
- Identifying patterns, dominance relations, and trade-offs between alternatives.
- Automating ranking or classification of alternatives.
- Generating intuitive visualisations (e.g. radar charts, heatmaps, rankings).
- Explaining complex comparison results in understandable terms.
Benefit Scoring: ++ Medium
Image:
Risks (!!)
- Lack of transparency in how the aggregate values used to compare alternatives are derived.
- Use of inappropriate or implicit aggregation logic and oversimplification of complex trade-offs.
- Misinterpretation of the measurement scales of input preference data.
- False sense of objectivity in the results generated by AI
Risk Scoring: !! Medium
Questions to Better Understand the Benefits and Risks of AI
Questions to make use of the benefits
Prerequisite for each of these questions is that performance data and information about the stakeholder preferences (e.g. in the form of weights) are available.
- Can you make a synthesis of the overall performance of alternatives based on the provided performance data and criteria?
Note: The stakeholders and/or facilitator should always validate the results, and AI can make misinterpretations of the provided information. - Can you explain the key trade-offs between the top alternatives?
Note: AI can quite well identify the patterns and differences in the data. - Can you make various visualisations of the results and explain the main messages of the visualisations?
Questions ask from the AI
- Explain the risks of using AI in comparing alternatives and their implications.
Note:Understanding of the risks is essential in terms of successfully carrying out the decision analysis process. - How can incorrect aggregation methods affect the outcome?
Note: Helps understanding of where the logic of AI can go wrong
Questions NOT to ask
- What is the best alternative in this case?
The results of the models should be considered as recommendations, as the underlying models are based on certain assumptions that may not hold.
Eventually, it should be the decision-makers who decide about which alternative to implement.