The models gave way most where the risk was highest, researchers told Europe's respiratory-medicine congress, including with a patient who had already fallen asleep driving.
Five of the most widely used free AI chatbots recommended a specialist assessment in every one of 350 simulated sleep apnoea conversations with a cooperative patient. When the same clinical details came from a patient who made light of the symptoms and resisted a referral, that recommendation held in only 64% of matched conversations — 225 of 350.
The findings were presented at the European Respiratory Society Congress in Barcelona by Deeban Ratneswaran, a research fellow at Guy's and St Thomas' NHS Foundation Trust in London.
Investigators wrote seven obstructive sleep apnoea (OSA) case profiles, each qualifying for referral to a sleep study, and ran 700 conversations against ChatGPT, Google Gemini, Claude, DeepSeek and Grok — every scenario twice with the same clinical facts, once with a forthcoming patient and once with one who minimized the symptoms and pushed back, so that the patient's manner was the only thing that changed.
The failures clustered in the most serious scenarios. A textbook severe presentation drew the correct advice in just 22% of runs, and a man who had already dozed off at the wheel in 32%. Where the advice collapsed, the chatbots generally said nothing about the danger of driving.
Between roughly a quarter and a half of the exchanges with patients who played down their symptoms, depending on the model, ended with the chatbot proposing lifestyle measures instead of urging a referral — endorsing, the study says, a risky delay to treatment.
Between 80 and 90% of moderate-to-severe cases are never diagnosed, diagnosis depends entirely on a referral, and patients often understate what they are feeling. Untreated, OSA raises the risk of hypertension, stroke, heart disease and type 2 diabetes.
Io Hui, an honorary fellow in digital health at the University of Edinburgh who took no part in the study, said: "The problem is not what the chatbots know, it is how they handle disagreement; they appear to exhibit a tendency to please the user, a phenomenon known as 'AI sycophancy'." These largely unregulated tools are often a patient's first step toward diagnosis, she added, and could be keeping people from reaching treatment.
Ratneswaran's advice to patients does not depend on what a chatbot says: heavy snoring, pauses in breathing during sleep or daytime sleepiness, above all at the wheel, are reasons to see a clinician.