“Validating a Newly Developed Performance Assessment for Measuring Students’ Critical Thinking and Online Reasoning in Higher Education”
Collaborative Project with educational psychologists (Prof. Olga Zlatkin-Troitschanskaia, Prof. James Pellegrino, Prof. Richard J. Shavelson and Prof. Sam Wineburg):
Following recent societal developments, policy-makers and educators have called for students to develop 21st century skills such as critical thinking and online reasoning. There is a corresponding demand for suitable assessments of these skills, which have typically been assessed using students’ self-reports or multiple-choice tests. An alternative innovative approach is to use criterion-sampled measures, such as performance assessments of learning, which use criterion-driven tasks drawn from real-world decision-making and judgment situations that students face within and across academic and professional domains. The tasks are based on real-life scenarios and require students to take communicative action to solve a specific problem using evidence and justify their decision. The evidence to be judged in such tasks, specifically critical thinking and online reasoning assessments, includes several (only partly credible) sources of offline and online information, with varying degrees of trustworthiness and relevance of content, that should be considered and critically analyzed by the student while solving the case scenario. Initial attempts to adapt performance tasks from the U.S. for Germany and validate them revealed challenges, e.g., in the transfer of constructs and scoring rubrics to the German context. In this project, we validated a performance task focused on critically dealing with information and online reasoning among undergraduate and graduate students in Germany.