Concurrent and Retrospective Metacognitive Judgements as Feedback in Audience Response Systems: Impact on Performance and Self-Assessment Accuracy

Papadopoulos, Pantelis M.; Obwegeser, Nikolaus; Weinberger, Armin (2021). Concurrent and Retrospective Metacognitive Judgements as Feedback in Audience Response Systems: Impact on Performance and Self-Assessment Accuracy Computers and Education Open, 2, p. 100046. Elsevier 10.1016/j.caeo.2021.100046

[img]
Preview
Text
1-s2.0-S2666557321000173-main.pdf - Published Version
Available under License Creative Commons: Attribution-Noncommercial-No Derivative Works (CC-BY-NC-ND).

Download (2MB) | Preview

Asking questions in classrooms can produce metacognitive judgements in students about their confidence in being able to answer correctly. In audience response systems (ARSs), these judgements can be elicited and used as additional feedback metrics. This study (n = 79) explores how online concurrent item-by-item judgments (OCJ) and retrospective composite judgments of performance accuracy (RJPA) can enhance students’ performance and self-assessing accuracy (i.e., calibration – as measured by sensitivity, specificity, and absolute accuracy index). In each of eight weeks, the students answered a multiple-choice quiz and had to denote their level of confidence that their answers were correct (OCJ) and estimate their final score (RJPA). The quizzes followed the voting/revoting paradigm according to which students answer all the quiz questions, receive feedback, and answer the same questions again before the correct answers are shown. The students were randomly grouped into two conditions based on the feedback they received in the ARS: the OCJ group (n = 41) received the percentage distribution and peers’ OCJs as feedback metrics, while the RJPA group (n = 38) received the percentage distribution and peers’ RJPAs. Data analysis showed a systemic underconfidence that affected students’ OCJ judgments. As a result, students in the RJPA group scored significantly higher than the ones in the OCJ one, were more accurate in self-assessing in the revoting phase, and felt overall more confident in the revoting phase. The study also discusses the relationship between the two judgments employed and the calibration variability between the two study phases.

Item Type:

Journal Article (Original Article)

Division/Institute:

Business School > Institute for Digital Technology Management
Business School

Name:

Papadopoulos, Pantelis M.;
Obwegeser, Nikolaus0000-0003-3404-1989 and
Weinberger, Armin

ISSN:

26665573

Publisher:

Elsevier

Language:

English

Submitter:

Nikolaus Obwegeser

Date Deposited:

20 Sep 2021 15:42

Last Modified:

29 Sep 2021 02:18

Publisher DOI:

10.1016/j.caeo.2021.100046

Uncontrolled Keywords:

Audience response systems Calibration Metacognition Self-assessment Peer Instruction

ARBOR DOI:

10.24451/arbor.15312

URI:

https://arbor.bfh.ch/id/eprint/15312

Actions (login required)

View Item View Item
Provide Feedback