An experimental study in the United States found that having a conversational AI insert slight misinformation into conversations with users increased false memory occurrence and reduced memories of correct information. The research was published in IUI ’25: Proceedings of the 30th International Conference on Intelligent User Interfaces.
…
Study author Pat Pataranutaporn and his colleagues examined the potential for malicious generative chatbots to induce false memories by injecting subtle misinformation during interactions with users.
The experimental conditions were the control condition (no intervention) and four intervention conditions that included interaction with AI …. Of these four conditions, two included reading an AI-generated summary of the article, while the other two included engaging in a discussion with the AI. In each pair of conditions, the AI was honest in one (i.e., correctly presenting facts from the articles) and misleading in the other (i.e., incorporating misinformation alongside factual points).
…
Related on the SLP
Results showed that participants who engaged in discussion with a misleading chatbot recalled the highest number of false memories compared to all the other conditions and recalled the lowest number of non-false memories.
