Will Google destroy western democracy? Bias in policy problem solving

Democracy requires students to choose policy positions based on evidence, yet confirmation bias prevents them from doing so. As a preliminary step in building a policy reasoning tutor, this study identifies where bias occurs during the search and analysis of evidence in a policy reasoning task. 60 university students played an on-line game in which they chose which of four policies would increase school performance. The between-subjects design compared a free search group who searched for evidence in google-like environment, to a sequential presentation group who read all available evidence, and manipulated whether the evidence confirmed or disconfirmed students' prior beliefs. The study measured the impact on students' evidence-based recommendations, their change in beliefs, and their recall of the evidence. Results showed that students did not cherry-pick evidence nor discount disconfirming evidence. However, students' extreme confidence in their initial beliefs usually prevented them from changing position, and they mistakenly recalled the evidence as confirming their beliefs. The results suggest that a policy tutor should focus on evidence synthesis and making recommendations based on explicit evidence.