Concurrent and Retrospective Metacognitive Judgements as Feedback in Audience Response Systems: Impact on Performance and Self-Assessment Accuracy

Papadopoulos, Pantelis M.; Obwegeser, Nikolaus; Weinberger, Armin (2021). Concurrent and Retrospective Metacognitive Judgements as Feedback in Audience Response Systems: Impact on Performance and Self-Assessment Accuracy Computers and Education Open, 2, p. 100046. Elsevier 10.1016/j.caeo.2021.100046

Preview

Text 1-s2.0-S2666557321000173-main.pdf - Published Version Available under License Creative Commons: Attribution-Noncommercial-No Derivative Works (CC-BY-NC-ND). Download (2MB) | Preview

Asking questions in classrooms can produce metacognitive judgements in students about their confidence in being able to answer correctly. In audience response systems (ARSs), these judgements can be elicited and used as additional feedback metrics. This study (n = 79) explores how online concurrent item-by-item judgments (OCJ) and retrospective composite judgments of performance accuracy (RJPA) can enhance students’ performance and self-assessing accuracy (i.e., calibration – as measured by sensitivity, specificity, and absolute accuracy index). In each of eight weeks, the students answered a multiple-choice quiz and had to denote their level of confidence that their answers were correct (OCJ) and estimate their final score (RJPA). The quizzes followed the voting/revoting paradigm according to which students answer all the quiz questions, receive feedback, and answer the same questions again before the correct answers are shown. The students were randomly grouped into two conditions based on the feedback they received in the ARS: the OCJ group (n = 41) received the percentage distribution and peers’ OCJs as feedback metrics, while the RJPA group (n = 38) received the percentage distribution and peers’ RJPAs. Data analysis showed a systemic underconfidence that affected students’ OCJ judgments. As a result, students in the RJPA group scored significantly higher than the ones in the OCJ one, were more accurate in self-assessing in the revoting phase, and felt overall more confident in the revoting phase. The study also discusses the relationship between the two judgments employed and the calibration variability between the two study phases.

Item Type:	Journal Article (Original Article)
Division/Institute:	Business School > Institute for Digital Technology Management Business School
Name:	Papadopoulos, Pantelis M.; Obwegeser, Nikolaus0000-0003-3404-1989 and Weinberger, Armin
ISSN:	26665573
Publisher:	Elsevier
Language:	English
Submitter:	Nikolaus Obwegeser
Date Deposited:	20 Sep 2021 15:42
Last Modified:	29 Sep 2021 02:18
Publisher DOI:	10.1016/j.caeo.2021.100046
Uncontrolled Keywords:	Audience response systems Calibration Metacognition Self-assessment Peer Instruction
ARBOR DOI:	10.24451/arbor.15312
URI:	https://arbor.bfh.ch/id/eprint/15312