"Just Asking Questions": Doing Our Own Research on Conspiratorial Ideation by Generative AI Chatbots

Abstract

Interactive chat systems that build on generative artificial intelligence frameworks - such as ChatGPT or Copilot - are increasingly embedded into search engines, Web browsers, operating systems, or available as standalone sites and apps. In a communication ecosystem where information disorder is a persistent threat (Wardle & Derakhshan, 2017), there is the potential for users to utilise chat systems to seek information about conspiracy theories and false claims. Conducting a systematic review of seven AI-powered chat systems, this study examines how these leading products respond to questions related to conspiracy theories. The nine theories chosen for analysis range from historical - such as the JFK assassination conspiracy theories, which have long been debated and debunked - to false claims related to more recent events, such as the idea that Hurricane Milton was geoengineered by Democrats. The chat systems were presented with preset questions which adopted a "casually curious" persona, requesting further information about the chosen conspiracy theories. Our findings to date suggest that AI chat systems are less likely to implement strict safety guardrails around historical conspiracy theories, such as the JFK assassination theories. By contrast, the chat systems were more sensitive to conspiracy theories involving certain minority groups. AI chat systems were also less likely to engage with conspiracy theories related to developing stories and breaking news. In this study, we consider how these patterns affect the role of AI in the information and media ecosystem and explore how AI chat systems may better respond during periods of political transition or division.