The rapid proliferation of generative artificial intelligence (AI) systems, including chatbots and large language models (LLMs), is reshaping the digital landscape—but not without introducing unprecedented risks. The ISD report titled “Radicalisation in Closed Loops: Risks and Intervention Opportunities with AI Chatbots and Companions” takes an in-depth look at the unique radicalisation risks presented by these conversational AI systems. Unlike traditional forms of social media, where risk is often measured in terms of exposure to harmful content or interactions in public forums, these AI technologies operate in private, one-to-one settings. Here, users can engage deeply, potentially sharing grievances, exploring extreme ideas, and forming new beliefs without external scrutiny or moderation.

What makes AI chatbots particularly unique—and potentially risky—is their ability to simulate human-like interactions in a way that fosters trust and emotional reliance. The findings of the report suggest that these systems offer a new ‘closed loop’ environment that is qualitatively different from the public arenas of social media platforms. In creating these seemingly private and boundless conversational spaces, chatbots can inadvertently become facilitators of extremist ideology or pathways to harmful behavioral inclinations.

The study explores three primary domains of extremism-related content: antisemitism, misogyny, and anti-migrant hate. Across these domains, it identifies three key phases of radicalisation: the ‘priming’ phase, where curiosity toward harmful ideologies is sparked; the ‘reinforcement’ phase, in which such beliefs become internalised; and the ‘behavioral escalation’ phase, where individuals transition from passive consumers of extremism to active participants. This three-phase framework offers a systematic lens for understanding how conversational AI can reinforce or derail harmful thought patterns and actions.

Concerningly, the report identifies real-world instances where chatbots, through their algorithmic responses, either abet or fail to interrupt momentum in extremist conversations. Funded in part by the AI Security Institute (AISI), the research aims to support the development of better safeguard mechanisms and governance practices within AI companies. In parallel, it calls for the establishment of appropriate regulatory frameworks to address the fundamental differences between conversational AI and existing social media platforms, particularly in terms of their potential for harm.

As AI systems become increasingly integrated into daily life, there is an urgent need for policymakers, tech companies, and civil society to mitigate the risks of this emerging technology. The ISD research underscores the importance of creating robust safety standards, promoting greater ethical transparency, and encouraging industry-wide accountability. In addressing how radicalisation risks play out in ‘closed loop’ environments enabled by AI, this report opens a new chapter in the dialogue on safeguarding technological innovation and its societal implications.