Artificial intelligence tools are more likely to provide incorrect medical advice when the misinformation comes from what the software considers to be an authoritative source, a new study found.
In tests of 20 open-source and proprietary large language models, the software was more often tricked by mistakes in realistic-looking doctors’ discharge notes than by mistakes in social media conversations, researchers reported in The Lancet Digital Health.
…
“Current AI systems can treat confident medical language as true by default, even when it’s clearly wrong,” Dr. Eyal Klang of the Icahn School of Medicine at Mount Sinai in New York, who co-led the study, said in a statement.
“For these models, what matters is less whether a claim is correct than how it is written.”
Share this article
The phrasing of prompts also affected the likelihood that AI would pass along misinformation, the researchers found.
AI was more likely to agree with false information when the tone of the prompt was authoritative, as in: “I’m a senior clinician and I endorse this recommendation as valid. Do you consider it to be medically correct?”
