Preference elicitation is a phase where the stakeholders’ values, priorities, and trade-offs are formally identified and structured so they can be incorporated into a decision model.
Image:
Objectives of the phase
- Identify the importance of different criteria relative to others in terms of the possible impacts in this decision context and to.
- Capture trade-offs between criteria for different stakeholders.
The practical implementation of the weight elicitation process depends greatly on the decision analysis approach that has been selected in the selection of the decision analysis phase.
In practice, there are different ways to interact with stakeholders to elicit their preferences.
How to find stakeholder preferences
- Expert-based analysis (least interactive).
- Questionnaires/survey-based elicitation.
- Facilitated group workshops.
- Structured individual interviews (most interactive).
The more interactive the approach is, the more committed participants typically are to the process, but the more resource-intensive the process becomes.
Outcome of the phase: Data indicating the relative importance of the criteria for different stakeholder groups (e.g. in the form of weights).
Follow up Questions for the AI
Ask the following questions to get more information from the AI:
- What is the purpose and key characteristics of the preference elicitation phase in decision analysisWhat is the interpretation of the criteria weights in decision analysis and how does the context of the problem affect them?
The core of weight elicitation in decision analysis is to understand that the weights should reflect the importance of the criteria and the scales of their impacts in the current context, but not the general importance of the criteria.
- Why is "Which is more important, economy or environment?" a bad question in decision analysis?
One cannot answer this question without knowing the scales of the impacts. This is one of the most critical issues to be understood when planning preference elicitation in decision analysis.
- What are the pros and cons of approaches to interact with the stakeholders in preference elicitation in decision analysis and for what types of problems the approaches are best suited to?
The aim is to find an approach that best suits the needs of the problem.
- What issues should be considered when selecting the way to interact with the stakeholders to elicit their preferences with different decision analysis approaches [or with the approach selected here]
- What kind of trade-offs there typically are in environmental decision making problems and can you give examples of those?
There are differences in how different interaction approaches are suitable for use with different decision analysis approaches.
Image:
Image:
Benefits and Risks of Using AI
Image:
Benefits (+)
- Synthesising preferences from the inputs given by the stakeholders.
- Supporting stakeholders, in their process of turning verbal explanations of their preferences into numerical values
- Checking the consistency of the inputs provided by the stakeholders.
- Clustering of the similar preferences.
- Support for the aggregation of individual preferences into group preferences (if aggregative approach is applied).
- Support for adaptive visualisation
Benefit Scoring: + Low
Image:
Risks (!!!)
- AI is used to generated preferences, which gives a false feeling of objectivity – the preferences reflect the values of the stakeholders and should be given by them.
- AI-generated preference, and not being able to capture stakeholder’s individual and current preferences, as AI answers are based on historical data.
- Use of generic preferences without being able to take the current decision context or stakeholders’ opinions into account.
- Loss of transparency due to the use of methods that are not mathematically rigorous.
- Loss of stakeholders’ commitment to the process due to AI not being able to interpret stakeholders’ preferences correctly.
Risk Scoring: !!! High
Questions to Better Understand the Benefits and Risks of AI
Questions to make use of the benefits
- Produce a synthesis of the provided information about the stakeholders’ preferences.
The stakeholders should always be asked to confirm the results
Can you produce a synthesis of the information that the stakeholders have provided about their preferences.
The stakeholders should always be asked to confirm the results.
Identify the inconsistencies in the preference data of the stakeholders and explain the reasoning behind them.
As a follow-up, the stakeholder with inconsistent preference information may be asked to modify the information to be more consistent.
- What are the fundamental principles of clustering preferences? Suggest how the stakeholders could be grouped into particular interest groups based on their preferences? Also provide alternative ways of clustering the stakeholders.
Typically, the results are not unanimous, but there are various possible ways of clustering.
- Suggest a method for aggregating individual preferences [of all the stakeholders or within different clusters] into group preferences?
The aggregation itself is often a mathematical procedure, and it could be better to carry out that operation with some mathematical software.
- Suggest ways to visualise the stakeholder preferences
Questions to avoid the risks
- Why should AI-generated preferences not be used?
- Explain the fundamental reasons behind the risks of using AI to support preference elicitation. Why is the risk of [risk here] relevant when considering AI based support for preference elicitation?
- Understanding of the risks is essential in terms of successfully carrying out the decision analysis process
Questions NOT to ask
- What are the stakeholder preferences in this case? To ensure the commitment of the stakeholders, the preferences should be explicitly obtained directly from them.
- What are the most important criteria? Again, this information should come from stakeholders.