How accurate are political polls?

With the 2024 presidential election just days away, new data from pollsters about the presidential election is seemingly being released every day. In the early period of the election, these polls depicted a variety of scenarios, but are now mostly focused on the general election matchup: the Republican nominee, former President Donald Trump, and the Democratic nominee, Vice President Kamala Harris.

Harris gained an early momentum in the polls upon replacing President Joe Biden as the Democratic nominee. However, the race remains a closely contested toss-up, with the margin of error for most polls meaning that either candidate could eke out an election victory. The race is particularly close in key battleground states such as Georgia and Pennsylvania.

As recent years have proved, polling is often, sometimes heavily, incorrect. Case in point: Polling generated by HuffPost on Election Day 2016 concluded that Hillary Clinton had a 98% chance of beating Trump. However, while the polls that year were dramatically wrong, most polling throughout the 2020 election cycle correctly predicted that Joe Biden would defeat Trump. So given that polling accuracy has been on both sides of the coin, how much trust should the public place in polls?

Subscribe to The Week

Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.

SUBSCRIBE & SAVE

Sign up for The Week's Free Newsletters

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

Sign up

How are political polls taken?

Polls are used "because posing questions to every member of a community would be costly and logistically difficult," according to a report from CalTech. However, a "sample size of just 1,000 to 1,500 people can be enough to estimate national opinion in the United States with a high level of accuracy."

A common method used in polling is random sampling, in which pollsters "draw a random sample of participants from computer-generated lists of landline and cell phone numbers (increasingly, polls are also being conducted online)," CalTech said. This "gives everyone in the population an equal chance of being included or not." A good poll "needs to be a microcosm of the group" being surveyed, Anthony Salvanto, the director of elections and polling at CBS News, said to CBS. The groups "could be made of voters, Democrats, Republicans, people of different genders, age groups and education levels."

But "different polling organizations conduct their surveys in quite different ways," the Pew Research Center said in 2020. Pew noted that CNN and Fox News conducted polls over the phone, while CBS News, Politico and The Associated Press conducted polls using various online techniques. The "best [polls] actually survey people and get them to join the panel," like online panel polls, political analyst Floyd Ciruli said to Rocky Mountain PBS. The "nice thing about a panel poll is there's maybe a thousand people on the panel in each state who will regularly poll for you, often online." Pollsters can also target specific metrics with online polls, something that is harder to do with phone banking.

Each election cycle sees updates to how polls are conducted, and 2024 is no different. Since 2020, there have been "four basic kinds of changes" in polling, said The New York Times. Many pollsters have begun distributing "surveys taken by text or mail" instead of the phone, and have also made "statistical adjustments to try to better account for underrepresented groups." Pollsters have benefited from new data this cycle, and the makeup of pollsters has also changed as individuals come and go.

How accurate are these polls?

Heavy criticism has been levied at polls in recent years, particularly in the last few rounds of presidential elections. Even in 2020, when most polls correctly predicted Biden's victory, the "national surveys of the 2020 presidential contest were the least accurate in 40 years," Politico said.

Does this mean that polls just aren't accurate? Not always, but they can present a different picture than reality. This is largely because "the real margin of error is often about double the one reported," Pew said. Many polls typically have a margin of error less than 3%, which "leads people to think that polls are more precise than they really are." But this margin "addresses only one source of potential error: the fact that random samples are likely to differ a little from the population just by chance."

There are at least three other identifiable sources of data errors that can come from poll taking, Pew added, but most polls don't calculate these metrics into their margins of error. The differing approaches in how polls are taken can also have "consequences for data quality, as well as accuracy in elections." As a result, the actual margin of error in most historical polls is closer to 6% or 7%, not 3%, said a 2016 study from The New York Times. This represents an error range of 12 to 14 data points, the Times said.

Nonetheless, polls can still be valuable and paint a widespread picture of Americans' feelings — and they are still sometimes on the money. Polling during the 2022 midterms was "historically accurate," FiveThirtyEight reported. This is partially because pollsters began "increasingly weighting surveys based on whom respondents recall voting for in a previous election, in addition to adjusting for standard demographics such as race and age," The New York Times said.

This method has long been used to calculate polling in other countries, but is only recently gaining widespread usage in the United States. After the 2016 election, it was also found that pollsters underrepresented less-educated voters, which heavily skewed poll results. Since then, pollsters have "adopted education as an additional survey weight, and a cycle of accurate polls in 2018 seemed to reflect a return to normalcy," the Times said.

Also notable is the percentage difference between the accuracy of polls versus the outcome of an election, which can often skew in opposite directions. Most polls "report a 95% confidence interval," but the "actual election outcome only lands inside that interval 60% of the time — and that's just a week before the election," said Don Moore, a behavioral professor at the University of California, Berkeley. Going further out from an election, the "hit rates fall even harder." There can also be "bias generated by the method by which pollsters reach respondents." For example, if a poll is only conducted over the phone, then "it will only reach people who have phones and who answer them when pollsters call." This can also lead to skewed results versus the population as a whole.

The accuracy of polls can also be examined when it comes to the battleground states in this year's election: Arizona, Georgia, Michigan, Nevada, North Carolina, Pennsylvania and Wisconsin. If people "want to predict what the polling error will be specifically in the 2024 swing states, we can actually get a little more precise," said Nathaniel Rakich at FiveThirtyEight. Polls in Nevada, Georgia and Arizona have been "even more accurate in the Trump era" than they were in the early 2000s. But polls of the northern swing states, Michigan, Pennsylvania and Wisconsin, haven't "been as impressive, although they're still more accurate than the average state." Since 1998, the seven swing states have weighted-average errors between 3.8 points and 4.9 points But while this sounds high, other states "are even harder" to analyze. At least "18 states have weighted-average errors of six points or more since 1998," said Rakich. This includes safely blue states like Hawaii and safely red states like Wyoming.

So while polling can't determine anything with certainty, it can "provide a nuanced picture of what a country, state or group thinks about both current events and candidates — and how that is changing," Texas A&M University political science professor Kirby Goidel said.