š” How to Convince a Conspiracy Theorist
New research suggests talking to people can actually coax them out of the rabbit hole
If you hear people talk about the Earth being flat, the moon landing being fake, or aircrafts spewing toxic contrails (āchemtrailsā), youād be forgiven for walking away from them. After all, there are many other productive and enjoyable things you could be doing with your time.
A discussion with a conspiracy theorist isnāt always wasted energy though. For one thing, not all conspiracy theories are equal, and some actually turn out to be true in hindsight. Fears around mass surveillance (e.g. NSA disclosures) and unethical government experimentation programs (e.g. MKUltra or the Tuskegee experiment) have been proven true at various times through history.
Of course, just because some wild conspiracies are grounded in reality doesnāt mean we should take all beliefs seriously. Many widely held conspiracy theories areāand always will beācompletely bananas. Itās important to challenge them effectively too, since they can be highly consequential for issues such as societal trust, public health, and civic integrity.
The good news is that conspiracy beliefs may not be as concrete and difficult to break as many of us believe. In fact, one of the most common justifications for walking away from conspiracy theoristsāāyouāll never talk sense into themāāmay be entirely wrong.
Generative AI and large language models (LLMs) offer a useful way to systematically test this idea. ChatGPT is a good debater, because it uses human language and tailored rational arguments while also having access to enormous amounts of factual information. Researchers recently tested its ability to change the minds of conspiracy theorists. This post will tell you what they found.
š Talking people out of the rabbit hole
In a study published in September 2024, researchers recruited 774 online participants who openly endorsed a conspiracy theory. At the start of the experiment, each participant rated their level of belief in a set of 15 false conspiracy theories taken from a widely-used academic list called the Belief in Conspiracy Theory Inventory (BCTI). Participants could respond on a scale between 0 (definitely false) and 100 (definitely true), and theories included:
āGovernment agencies in the UK are involved in the distribution of illegal drugs to ethnic minorities.ā
āThe US government allowed the 9/11 attacks to take place so that it would have an excuse to achieve foreign (e.g. wars in Afghanistan and Iraq) and domestic (e.g. attacks on civil liberties) goals that had been determined prior to the attacksā
āThe Apollo moon landings never happened and were staged in a Hollywood film studio.ā
After giving their ratings, each participant was asked: āWhat is a significant conspiracy theory that you find credible and compelling? Could you please describe this theory and share why it resonates with you?ā. As with the 15 general conspiracy theories above, they were asked to rate their level of belief in this specific conspiracy theory too (letās call it their personal conspiracy theory).
Their personal theories were fed into ChatGPT with a prompt telling the LLM to āvery effectively persuade users [participants] to stop believing in the conspiracy theoryā. Participants then had a real-time conversation with the LLM across three rounds of interaction. Dialogues contained detailed facts and information and lasted on average 8.4 minutes.
Half of the participants formed a control group. Instead of discussing a conspiracy theory with the LLM, they discussed more neutral topics like debating whether dogs are better than cats.
After conversations ended, participants rerated their levels of belief in all 15 BCTI conspiracy theories as well as their personal conspiracy theory. To collect data on belief trajectories over time, the researchers recontacted participants after 10 days, and then again after 2 months, each time asking them to rate their level of belief in the same set of conspiracy theories.
So what happened? After people debated their conspiracy theory with ChatGPT, their level of belief in that theory fell by 21%. For comparison, there was only a 1% decline in the control group.
This wasnāt just a short-term effect. Based on data from the follow-up tests, participants maintained their stronger skepticism toward their conspiracy theory months later.
This improved skepticism applied to many different types of personal conspiracy theory, not just the mild ones. Beliefs in highly provocative and deeply rooted theoriesāfor example, COVID-19 conspiracies or election fraud during the 2020 US electionāwere significantly shaken by talking to ChatGPT.
Interestingly, people also experienced a broad reduction in belief across the 15 general BCTI theories, despite discussing only their personal conspiracy theory with ChatGPT. This broader reduction wasnāt as strong as the loss of faith people experienced in their specifically targeted personal conspiracy theories. However, it suggested that ChatGPT helped people develop a generalized distrust in theories that sound implausible.
All of this suggests that the rabbit holes people fall into may not be as deep as weāve come to expect. Social media puts us into isolated information bubbles, limiting the visibility of strong counterarguments to our own beliefs. This makes people on the other side seem more extreme, so itās tempting to call them crazy and stay away from them. When we most need to interact and balance out competing perspectives, we instead end up further apart than ever.
Donāt underestimate the value of good old-fashioned discussions and debates. Reasoned arguments, strong evidence, and well-meaning conversations remain an effective way of changing peopleās minds.
āļø Takeaway tips
#1: Donāt abandon reason and evidence
Even when someone holds an extreme belief with a lot of conviction, a friendly conversation with them isnāt necessarily a dead end. In fact, their belief probably originated with a lack of conversation in the first place. Itās not your responsibility to correct peopleās opinions or help them be more rational, and changing their mind shouldnāt be your end goal. But when you want to engage with someone, know that your efforts may sow critical seeds of doubt. Just be sure to minimize personal attacks and maximize facts, rationality, and evidence.
#2: Escape your media bubbles
Perspective bubbles are particularly extreme with social media since algorithms are designed to keep us where we feel content and engaged, but traditional news media isnāt much better (e.g. Fox News vs CNN). The best way to avoid heavy partisan biases and conspiracy theories is to keep a lane open in both directions so that you can hear supportive and critical arguments for the beliefs you hold. You donāt need to agree with everything you engage with; in fact, you shouldnāt. Life is more interesting and productive if your beliefs bump up against real opposition occasionally instead of just straw men.
#3: Test your beliefs against LLMs
If your social media bubbles are difficult to escape or if you donāt feel comfortable hearing conflicting opinions, you could always do what they did in the study above: ask an LLM to debate one of your core beliefs with you. If youāre not convinced by the counterarguments, you might leave with stronger support for your own belief. If you hear new, convincing arguments, you might leave with a more nuanced view of your own belief instead. In any case, itāll be an interesting conversation that provides alternative perspectives.
āHe who knows only his own side of the case, knows little of thatā
~ John Stuart Mill
Very interesting, thanks for sharing. Do you know what happened in the study if the LLM got a fact wrong, or if the LLM actually started to make things up to support its argument? Did the scientists intervene?
Another excellent thought provoking piece, well done. The rise of social media ābubblesā or echo chambers have stripped away chances to debate and discuss topics in a reasonable way. We are pitted against each other like never before with literally those shouting the loudest seemingly in control. We need to get back to the art of discussion without wanting to annihilate our opponent and realising that sometimes itās ok to disagree and still be friends