Accuracy, completeness and hallucinations in large language model generated statements and references for common rhinologic disorders
Volume: 0 - Issue: 0
First page: 0 - Last page: 0
I. Margulis - M.A.M. Slim - E.C. Sommer - Y. AAlammar - A.B. Kassam - S. Khalife - L. Macias-Valle - D.D. Sommer
DOI: 10.4193/Rhin26.460
Large language models (LLMs) are increasingly adopted for medical documentation, clinical decision-making, and scientific writing. Although these applications demonstrated impressive capabilities, concerns persist regarding their accuracy, fabricated references, and ethical implications (1-3). The Massive Multitask Language Understanding (MMLU) test was introduced as a comprehensive benchmark to evaluate how well LLMs perform across a diverse range of tasks (4). This prospective comparative two part study evaluated the accuracy and completeness of ChatGPT (CGPT)-3.5-generated 100 word summaries of four common rhinologic disorders (epistaxis, allergic rhinitis, odontogenic sinusitis, and chronic rhinosinusitis) including background, investigations, and magement as assessed by international rhinology society members. Furthermore, we evaluated the accuracy of references generated by contemporary LLM versions for the same prompts (CGPT-5 thinking (5), non-thinking (6), Claude Sonnet 4.5 non-thinking (6) and Gemini 2.5 (7)). Otolaryngologists evaluated each statement using standardized scales for accuracy, completeness, and overall reference quality.
Rhinology 0 - 0: 0-0, 0000
To see the issue content and the abstract you do not have to login
Please login to download the full articles
If you do not have a subscription to Rhinology please consider taking one.
