The trouble with polling

Polls are struggling to capture an accurate picture of the American voter. Why? Here's everything you need to know:

Have polls lost credibility?

Yes. In three of the last four national election cycles — 2014, 2016, 2018, and 2020 — polls significantly overestimated the performance of Democratic candidates. The polls' poor performance exposed the reality that polling companies face far more challenges today in getting a representative cross-section of people to respond. In an age of ubiquitous cellphones and caller ID, the Pew Research Center estimates that only 6 percent of the people polling firms call respond, down from nearly 50 percent in the 1970s and '80s. With many Americans abandoning landline phones, pollsters have been forced to call cellphones, but federal law forbids them from making automated calls to these phones, making polling more expensive. People who still do respond to polls are generally older, whiter, more educated, and more likely to be female than the voting public. Pollsters try to correct for that skew, but the industry nonetheless faced a major reckoning after the 2016 presidential election.

Subscribe to The Week

Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.

SUBSCRIBE & SAVE

Sign up for The Week's Free Newsletters

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

Sign up

What happened that year?

Most polls predicted that Hillary Clinton would be elected president. In the final days, aggregate polls predicted she'd win the popular vote by 3 percentage points; that turned out to be only 1 point off, well within the stated margin of error. But on a state level, the polls missed a last-minute surge in support for Donald Trump in Wisconsin, Michigan, and Pennsylvania; his narrow victories there gave him the Electoral College and the presidency. After that black eye, the American Association for Public Opinion Research (AAPOR) released guidelines to reduce polling bias, including more heavily weighing white voters without college degrees. Yet 2020's results were even less accurate.

Why were the polls so far off?

Americans elected Biden with 51.3 percent of the vote vs. Trump's 46.8 percent — a historically substantial margin. But most polls had predicted an even greater victory margin for Biden and a "Blue Wave" of Democrats in the House, which failed to materialize. An AAPOR postmortem published in July found that polls overstated Biden's victory margin by 3.9 percentage points in the final weeks before Election Day — the industry's worst performance in 40 years. Experts aren't sure what skewed the numbers the most, but Vanderbilt University political scientist Joshua Clinton called 2020 "pretty much a perfect storm for pre-election polling." First of all, the pandemic changed campaigning and voting norms. One theory is that left-of-center voters were more likely to follow social distancing guidelines and stay home, making them more reachable by phone and thus overrepresented in polls. Secondly, turnout was the highest in more than 100 years, with new and newly energized voters possibly weakening pollsters' "likely voter" screens. Finally, Trump repeatedly told his supporters not to trust "fake" polls that showed him trailing, perhaps influencing them to ignore calls from pollsters.

Do faulty polls always tilt leftward?

No. In 2012, some polls, including Gallup, predicted a narrow victory for Mitt Romney; instead, Barack Obama was re-elected comfortably. Last September, RealClearPolitics.com's poll aggregation suggested that Californians would vote against recalling Gov. Gavin Newsom by a 14.5-point margin, but he survived by nearly double that. Though polls use "enthusiasm" to gauge how likely it is that a respondent will vote, California made participation extremely easy, mailing all 22 million of the state's registered voters postage-paid mail-in ballots. As Democratic strategist Garry South said, "This doesn't require enthusiasm, folks."

How can polling be fixed?

No one's entirely sure, but polling organizations have begun making major changes. FiveThirtyEight.com has stopped privileging live caller–only polls, finding that they no longer are more accurate than polls that employ a mix of methods to reach voters, including online, text-based, and interactive voice response systems. Polls are changing what they ask people, too. Pew has added screening questions designed to measure a respondent's trust in institutions, such as whether they volunteer, and weight the mistrustful more heavily, since they're more likely to be underrepresented.

Can we trust polls anymore?

On balance, yes — though maybe we should expect a little less of them. All polls have a margin of error, and in close elections that margin can result in "wrong" predictions. If a candidate gets, say, 51 percent of the vote instead of the predicted 49 on Election Day, headlines deem it an upset — even if that 2-point difference is within the estimated margin of error. Because polls are snapshots in time, they can mislead, since events can change how people vote right up to the last minute. In 2016, FBI director James Comey's announcement that he was looking into Clinton's emails 11 days before the election apparently swung many undecided voters to Trump. Still, even though trust in polls has declined, they'll continue to get a lot of attention as a useful if imperfect measure of public opinion. "A polling surprise like we had in 2020 is not going to uproot or destroy the industry," said communications scholar W. Joseph Campbell. "It's just too profoundly attached to American life."

Were there 'shy Trump voters'?

Before the 2016 election, Trump campaign manager Kellyanne Conway was among those who claimed he had hidden support from so-called "shy" voters, who may have told pollsters they were voting for Hillary Clinton out of fear of negative judgment. The gap between the polls and Trump's victory margin appeared to give credence to that theory. Research has shown 45 percent of college degree–holding Republicans said they feared losing their jobs if their support of Trump were known. But the "shy Trump voter" concept remains controversial. In 2020, even online and robocall polls, which offer Trump supporters relative secrecy, overcounted Biden supporters, and the pro-Democratic poll bias was even stronger down ballot. Institutional distrust, encouraged by Trump, instead might have made his supporters reluctant to answer poll questions at all. They were undercounted, in other words, rather than lying to pollsters about their preference. "Trump spent the last four years beating the crap out of polls," said Republican pollster Neil Newhouse, "and a big proportion of his supporters just said, 'I'm not participating.'"

This article was first published in the latest issue of The Week magazine. If you want to read more like it, you can try six risk-free issues of the magazine here.