Simulating diverse patient populations using patient vignettes and large language models

Denecke, Kerstin; Reichenpfader, Daniel (2024). Simulating diverse patient populations using patient vignettes and large language models Proceedings of the First Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC-COLING 2024, pp. 20-25. ELRA, ICCL

Text 2024.cl4health-1.3.pdf - Published Version Restricted to registered users only Available under License Publisher holds Copyright. Download (193kB) | Request a copy

Ensuring equitable access to digital therapeutics (DTx) is essential to avoid healthcare inequalities in an era of increasing digitization. This requires DTx to be tested with users from diverse populations, which is often not realistic due to time and resource constraints. In this paper, we propose the use of large language models (LLMs) to simulate diverse patients. Specifically, we manually create a patient vignette that characterizes a specific population group. Variations of this vignette are used for role-prompting a commercial LLM, GPT-4, instructing the LLM to take on the role described in the patient vignette and act accordingly. We investigate if the LLM stays in its given role. To do this, we simulate a medical anamnesis interview with the role-prompted LLM and analyze its responses for compliance, coherence, correctness, containment, and clarification. Our results show that GPT-4 generates compliant, coherent and clinically valid responses, including information that is not explicitly stated in the provided patient vignette.

Item Type:	Journal Article (Original Article)
Division/Institute:	School of Engineering and Computer Science > Institute for Patient-centered Digital Health School of Engineering and Computer Science > Institute for Patient-centered Digital Health > AI for Health School of Engineering and Computer Science
Name:	Denecke, Kerstin0000-0001-6691-396X and Reichenpfader, Daniel0000-0002-8052-3359
Subjects:	R Medicine > R Medicine (General) T Technology > T Technology (General)
Publisher:	ELRA, ICCL
Submitter:	Kerstin Denecke
Date Deposited:	31 May 2024 12:00
Last Modified:	31 May 2024 12:00
Uncontrolled Keywords:	Large Language Models Inclusive Design Accessibility Patient Vignettes Simulation
ARBOR DOI:	10.24451/arbor.21921
URI:	https://arbor.bfh.ch/id/eprint/21921