🗳 Your Beliefs About Public Opinion Are All Wrong

How mistaken intuitions send democracies astray

Mar 20, 2022

Data is most exciting when it proves you wrong. All of us have intuitions and expectations about how society is structured and how it’s changing. Sometimes, the actual numbers support those perspectives, but occasionally, they upend our world view.

Expectations are particularly influential when it comes to public opinion. Our beliefs and expectations about what other people think determine how we make decisions, how we feel in our communities, and even how we vote in elections. If we have false beliefs about how public opinion is changing on issues like racism, climate change, and gun control, we end up making political choices that are poorly aligned with our priorities.

That’s why polling data is so important. If you believe that atheism and climate change sympathies have been growing, or that gun advocacy and pro-immigration sentiments have been declining, it may be time to check the numbers. One newly published study has taken a long, hard look at how our expectations of public opinion match up with reality.

(By the way, sorry if you were waiting for a newsletter last Sunday. COVID hit me pretty hard and I’m only now resuming normal life again!)

🗣 Unexpected truths about public opinion

In a study published in March 2022, researchers asked ~1000 Americans to estimate public opinion on various issues. Most questions were related to politics such as “Do you favor or oppose the death penalty for persons convicted of murder?” and “Are you for or against a law which would make it illegal to manufacture, sell or possess semi-automatic guns known as assault rifles?”. But questions spanned a range of topics, touching on issues such as abortion, religion, homosexuality, and corporal punishment.

For each question, people estimated how much of the general population would be in favor of a political position at two different time periods: a past year from several decades ago and a recent year after 2010.

The researchers calculated how much people believed public opinion had changed over time, and then compared those beliefs to what really happened according to national polls from organizations such as Gallup and Pew Research Center. They categorized people’s estimates as correct, overestimated, underestimated, or estimated in entirely the wrong direction.

The results showed that people overestimated attitude changes for 57% of all issues, underestimated changes for 20% of issues, and guessed the wrong direction of change for another 20% of issues. They correctly estimated attitude shifts for only two of the 51 total questions. By a clear margin, overestimates were the most common bias in judging the movement of public opinion.

When judging on a 0-100 scale, people overestimated attitude changes by an average of 22 points. In some cases, errors in judgment were fairly extreme. For example, people believed that only 32% of Americans would be willing to vote for a female president in 1972 and that this number would have increased by 38 points to 70% in 2010. In reality, 74% of Americans were willing to vote for a female president in 1972 and 96% were willing in 2010—a change of only 22 points. In other words, people overestimated growth in public support for a female president over a 28-year time period. On top of that, their estimates in some years (e.g. 1972) were so misguided that they mistook a majority opinion for a minority opinion.

Bad estimates of public opinion were linked to an assumption that American society is growing more liberal over time. For example, participants believed that Americans in the present day were more supportive of gun control and extramarital sex and less supportive of the death penalty than Americans in the past—exactly the kinds of opinions that participants linked to liberal politics. In reality, polls showed that public opinion on these issues had moved in the opposite conservative direction across the last few decades.

In a final set of experiments, the researchers wanted to see whether manipulating people’s beliefs on public opinion would shift their political decision-making. They asked people to imagine either increasing or decreasing public support for issues such as immigration limits, gun control, and a death penalty ban. On these specific issues, increasing support was consistent with people’s erroneous beliefs in the previous experiments, while decreasing support was consistent with genuine public opinion.

In all scenarios, people felt it was more justified for Congress to limit immigration or ban assault weapons if they imagined that a larger percentage of the public supported that view. This isn’t entirely surprising since it lines up with basic democratic principles. But interestingly, people felt it was more acceptable to ban the death penalty if public support for it had increased from 33% to 50% since 1972 than if public support had decreased from 67% to 50%. In other words, people don’t just care about what percentage of the public support a particular policy at any given time; they actively care about how attitudes have changed over time too.

This all goes to show that people’s beliefs about public opinion really do matter in shaping their political calculus. If those beliefs deviate from ground truth according to data—as the experiments above prove that they often do—people make misinformed decisions that don’t necessarily reflect their own best interests.

⭐️ Takeaway tips

Check the data: A healthy democracy depends on reliable information and accurate perceptions, but none of us have correct assumptions all of the time. That’s why it’s so important to regularly check in with the data if you care about making optimal decisions. There’s a lot of discussion these days about disinformation (here’s my previous newsletter on fake news), but sometimes we mislead ourselves by relying on our own mistaken intuitions. It’s easy to find polling data from trusted organizations, so check how your current intuitions stand up against the best available numbers.
Question your gut and expand your world view: I’ve had several of my own gut instincts entirely disproven through the course of writing this single newsletter. We’re all wrapped up in information bubbles that give us a biased view of how society is evolving. I’ve lost count of how many times I’ve discussed “shock election outcomes” with friends in the last decade. Perhaps we wouldn’t be so shocked if we had a more representative view of what was going on outside our social circles.
Be wary of social pressure: In any democratic society, it makes sense to care about majority opinions and check how social views are evolving. But it’s also true that some majority opinions in history were moral crimes, and some of today’s majority opinions will also be seen as hideous by future civilizations. Sometimes, the best course of action is the minority view, so check in with your own reasoning and ethics as often as you check in with polling data.