Questionnaire experience and the hybrid System Usability Scale: Using a novel concept to evaluate a new instrument

Baumgartner, Juergen; Ruettgers, Nicole; Hasler, Annigna; Sonderegger, Andreas; Sauer, Juergen (2021). Questionnaire experience and the hybrid System Usability Scale: Using a novel concept to evaluate a new instrument International Journal of Human-Computer Studies, 147(147), p. 102575. Elsevier 10.1016/j.ijhcs.2020.102575

[img]
Preview
Text
Baumgartner_2021_Questionnaire experience and the hybrid System Usability Scale.pdf - Published Version
Available under License Creative Commons: Attribution-Noncommercial-No Derivative Works (CC-BY-NC-ND).

Download (2MB) | Preview
[img] Text
manuscript_hsus_revision_01.11.2020.pdf - Accepted Version
Restricted to registered users only until 1 April 2023.
Available under License Creative Commons: Attribution-Noncommercial-No Derivative Works (CC-BY-NC-ND).

Download (1MB) | Request a copy

This article presents the concept of questionnaire experience (QX), intending to add a new element to the psychometric evaluation of questionnaires, which may eventually help increase the validity and reliability of instruments. The application of QX is demonstrated in the development of the Hybrid System Usability Scale (H-SUS), making use of items comprising pictorial and verbal elements to measure perceived usability. The H-SUS was modelled on the verbal version of the System Usability Scale (SUS). Since previous research showed advantages of pictorial scales over verbal scales (e.g., higher respondent motivation) but also disadvantages (e.g., longer completion times), we assumed that hybrid scales would combine the advantages of both scale types. The goal of this study was to compare the two instruments by assessing traditional psychometric criteria (convergent, divergent and criterion-related validity, reliability and sensitivity) and respondent-related aspects of QX (respondent workload, respondent motivation, questionnaire preference, and questionnaire completion time). An online experiment was carried out (N = 152), in which participants interacted with a smartphone prototype and subsequently completed the verbal SUS together with the H-SUS. Results indicate good psychometric properties of the H-SUS. Compared to the SUS, the H-SUS showed similar workload levels for questionnaire completion, higher levels of respondent motivation, but longer questionnaire completion time. Overall, the H-SUS is considered a promising alternative for the evaluation of perceived usability. Finally, QX can be considered a useful concept for identifying potential problems of psychometric instruments in a respondent-centred way, which may help improve the quality of future scales.

Item Type:

Journal Article (Original Article)

Division/Institute:

Business School > Institute for New Work
Business School > Institute for New Work > New Forms of Work and Organisation

Name:

Baumgartner, Juergen;
Ruettgers, Nicole;
Hasler, Annigna;
Sonderegger, Andreas0000-0003-0054-0544 and
Sauer, Juergen

Subjects:

B Philosophy. Psychology. Religion > BF Psychology
B Philosophy. Psychology. Religion > BH Aesthetics

ISSN:

1071-5819

Publisher:

Elsevier

Submitter:

Andreas Sonderegger

Date Deposited:

20 Jan 2021 10:01

Last Modified:

24 Jan 2021 01:34

Publisher DOI:

10.1016/j.ijhcs.2020.102575

ARBOR DOI:

10.24451/arbor.13934

URI:

https://arbor.bfh.ch/id/eprint/13934

Actions (login required)

View Item View Item
Provide Feedback