When Nate Silver, the undisputed king of election forecasting, declares that "The World May Have a Polling Problem," it's time to take notice. Why are public opinion polls increasingly proving unreliable?
It seems clear that two things could be happening. Either those who agree to cooperate with the pollsters are unrepresentative of the population, or they are deliberately lying to pollsters about their preferences.
Why on Earth would either of those, or both, be happening?
The Week
Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.
SUBSCRIBE & SAVE
Sign up for The Week's Free Newsletters
Join more than 350,000 subscribers and keep yourself informed with a selection of The Week’s most interesting, enlightening and entertaining stories - plus daily puzzles.
Join more than 350,000 subscribers and keep yourself informed with a selection of The Week’s most interesting, enlightening and entertaining stories - plus daily puzzles.
Latest Videos FromThe Week
Because there are so many polls in the field that few "normal" people are willing to answer questions honestly, leaving responses to oddballs or those eager to have a little fun at the expense of the pollster.
Can you blame them? Growing up in the 1970s and '80s, I never recall our household receiving calls from pollsters. Polls started to become a little more common in the 1990s. But in the past decade or so, they've exploded. I don't just, or even primarily, mean calls from professional political pollsters, though there are certainly more of those, including a lot more "push" polls, in which a person (or, more often, a recording of a person) on the phone asks questions not to gauge your opinions, but to shape them in the direction of a particular candidate or party.
But that's just the tip of the proverbial iceberg. I call Comcast to discuss an issue on my bill, and before I reach a human being I'm asked if I wouldn't mind answering a few questions at the end of the call. I take the car in for an oil change, and I receive an email a day later to ask me a few dozen questions about my customer experience — and then a phone call a few days after that, just in case the email ended up in my spam folder. I visit the doctor for an annual check-up, and the phone rings a week later from a firm wanting to know if I'll answer some questions about the appointment. I check the mailbox outside my home and find a letter from the university where I earned my doctorate, asking if I'd be willing to take part in a study of the career paths of Ph.D.'s.
On it goes, month after month, with the polls piling up.
I know, I know: This is nothing but a first-world problem. And I certainly don't want to portray it as more than it is, which is a minor nuisance. Yet it's a revealing nuisance — one that exposes an aspect of the present historical moment that makes it different from earlier eras.
The human mind is limited in its capacity to memorize, organize, sort, and access information, and in the ability to analyze it dispassionately. That's why for all of human history until the invention of the computer, people (including public figures and business owners) made judgments about the world by eyeballing, guesstimating, and generalizing on the basis of sometimes dubious inferences rooted in inevitably partial individual experiences, memories, and subjective impressions, with all of the distortions that go along with them.
Computers, with their vastly superior memory and capacity to analyze vast quantities of information, give us insights that would be otherwise off-limits. That has very positive consequences for social-scientific research, which can now propose and test hypotheses about the shape and contours of The Aggregate — meaning society (or segments of society) viewed as a whole.
But the computer-facilitated compilation and crunching of data also presents an enormous temptation to public figures and businesses, both of which desperately want every added edge they can get their hands on.
In what might be called the classical model of retail democratic politics, the politician goes out on the campaign trail and stakes out positions based on a mixture of what he thinks the people want to hear and what he thinks is best for the country; if the message resonates with voters, they will turn out for him on Election Day. The same holds for a business-owner or entrepreneur: she offers a product or service, and success is determined by sales and whether customers show their satisfaction by returning for more and spreading good word of mouth.
Surveys go much further — into the minds of voters and customers. Knowledge is power, and survey data and analysis grant politicians and those who own and run businesses the power of knowing precisely what the people want, sometimes even before they're fully aware of it themselves.
What's wrong with that? Viewed one politician or business at a time, nothing much. The problem arises when thousands of public figures and marketing departments launch their own surveys to gain competitive advantages. The result is an over-saturated public-opinion marketplace in which citizens and customers alike begin to feel a mixture of fatigue and irritation that dilutes the accuracy of the polls, leading their trustworthiness to get called into question.
That's where we are now, and it's hard to see how we get out of it. Unless, of course, the politicians and businesspeople who've grown so dependent on data take a step back to the ways of old. But that would require a willingness to live with greater uncertainty than today's pollsters, strategists, and marketing gurus think is tolerable.
If I liked your product or service, I'll be back and tell others about it. If I didn't, I won't. If I really hated it, I'll complain to you and maybe spread the word about how awful my experience was. That's how you'll get your feedback, and it will arrive soon enough. In the meantime, leave me alone. The same holds for politicians and parties. Look out at the country, talk to smart people about it, and propose some plans to make it a little better. I'll vote for you if they sound reasonable, and I won't if they don't. That should be good enough.
And hell, as we've begun to see, the alternative isn't perfect information gleaned from a million accurate surveys. It's distorted, misleading information warped by surveys more and more often completed by resentful voters and customers.
