The Impact of Prompting Strategies on the Quality of LLM-Generated Biomedical Explanations

Abstract

Recent advancements in Large Language Models (LLMs) have great potential for understanding and reasoning within the biomedical field. Yet, the ability to craft interpretable and dependable medical explanations is predominantly determined by the design and arrangement of prompts. This research evaluates three styles of prompting (structured, role-based, and a hybrid combining both) and their influence on the quality of explanations from the GPT-5 model on the DDXPlus medical dataset. Significant differences in clarity and confidence were observed across all prompt techniques in the selected cases. Role-based referrals achieved the highest clarity (5.0 out of 5.0) and confidence scores (81.2%), while hybrid referrals explained symptom-disease relationships in a detailed and structured manner. Furthermore, the inclusion of clinician assessment in the study enhances the importance of research into real-world clinical applications in terms of comparing ground-truth and model outcomes. Overall, the findings demonstrate that strategic referral design is crucial for optimizing LLM outcomes in clinical practice and enables a transition from accurate diagnoses to truly interpretable and clinically useful explanations. © 2026 IEEE.