Evoking information in probability assessment: Knowledge maps and reasoning-based directed questions

To assess probabilities in decision analysis, and for decision making in general, decision makers must evoke and apply relevant information. Decision analysts have developed a variety of structuring tools to aid decision makers in these tasks, including influence diagrams and knowledge maps. However, despite their pervasive use in practice, there have been no reported empirical tests of these tools. One goal of the present research was to provide an empirical test of the evocative knowledge map methodology. Second, a theoretical analysis of probability assessment was used to develop a new prescriptive elicitation technique. This technique uses a theoretically-grounded set of directed questions to help decision makers evoke information for probability assessment. Experimental results showed that both the knowledge map and the new directed questions methodology elicited a higher quantity and quality of information from decision makers engaged in probability assessment tasks than did a control condition. Further, the information elicited by the two techniques was qualitatively different, suggesting that the two methods might profitably be used as complementary elicitation techniques.