The only thing revealed by polls on Florida's 'Don't Say Gay' bill is that polling sucks
Why it's harder than ever to get accurate results
Do Americans support or oppose Florida's House Bill 1557, which is officially known as the "Parental Rights in Education" but is almost universally described as the "Don't Say Gay Bill"? You'd think it would be an easy question to answer. Call up a few thousand people, ask their opinions, and tabulate the responses. Problem solved.
Except it's not that simple. A lot of people don't pick up the phone or talk to strangers, so pollsters must call many times the number of respondents they need in order to amass a statistically adequate sample. Those respondents must be asked a battery of other questions to determine their demographic characteristics. Because there's no guarantee that the people who answer calls will be representative, their actual numbers must be weighted to correspond to estimates of the makeup of the broader population based on yet other polls. That's before you even ask the question, which can be formulated in ways that encourage or deter certain responses. And of course you have to hope respondents answer honestly, rather than in ways they consider socially or politically desirable but don't correspond to their real opinions.
This hidden complexity helps explain widely divergent results of polls on the Florida bill. Last month, a statewide poll of registered voters showed that a 49 percent plurality opposed the measure, with just 40 percent expressing support. A couple of weeks ago, a national poll of adults showed overwhelming opposition by nearly two-thirds of the public. That poll acknowledged an oversample of LGBT respondents, which called its results into question. A few days later, another national poll, this time of registered voters, showed a slim majority in favor. Unlike the others, that was a so-called tracking poll, in which the same people are repeatedly surveyed over an extended period of time. Just yesterday, finally, a second Florida survey showed majority support for the law among likely Democratic primary voters.
Subscribe to The Week
Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.
Sign up for The Week's Free Newsletters
From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.
From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.
What's going on? One issue is that the polls didn't include precisely the same cohort. Registered voters in Florida, adults around the country, national registered voters, and Florida Democrats could theoretically have different opinions about a controversial measure. Typically, though, you wouldn't expect the range of responses to be quite so large. Moreover, answers would likely correlate pretty closely to party affiliation, which makes the result among Florida Democrats surprising.
The questions were also subtly different. All but three of the polls paraphrased the most debated provision of the bill, which states that "Classroom instruction by school personnel or third parties on sexual orientation or gender identity may not occur in kindergarten through grade 3 or in a manner that is not age appropriate or developmentally appropriate for students in accordance with state standards." Only the tracking poll added context, noting that critics "say [the bill] will block important conversations about LGBTQ issues" and breaking down opinion about specific measures. Which approach is most accurate? There's no obvious answer.
There are also problems inherent to all issue polls. One is that asking people about tradeoffs and consequences can have significant effects on their responses. In a discussion of recent polling concerning a no-fly zone in Ukraine, Washington Post writer Aaron Blake described how variations in wording and discussions of the likely consequences led to reduced support in different surveys. And it's not uncommon for the public to approve an idea in the abstract and then balk at its implications.
More importantly, merely expressing approval or opposition doesn't tell us how much respondents actually care. In legislative and electoral politics, the opinions of a highly motivated minority can outweigh the preferences of a lukewarm majority. Even when poll results are accurate, then, they don't always tell us the state of play.
Finally, there's a possibility that the numbers are just wrong. It was not even two years ago that many polls predicted an overwhelming victory for Joe Biden — including in states that he went on to lose by a considerable margin. Looking beyond Florida, falling response rates threaten to undermine polls' main source of authority — the claim that they capture the opinions of a broad public rather than a select few. Although political scientists and pollsters employ various techniques to address the problem, that's worth remembering whenever anyone trumpets poll results to vindicate their favored position or candidate.
In sum, polls alone don't reliably tell us what Americans think of the Florida bill. And even if they did, that wouldn't tell us how they'll respond when it's actually implemented, whether in ways consistent with supporters' assurances of its limited scope or that bolster critics' claim that it amounts to a gag rule. As my colleague Jim Antle argued, Democrats have stumbled when they've interpreted majority approval for same-sex marriage and antidiscrimination laws as a mandate for jettisoning traditional understanding of gender altogether (consider Ketanji Brown Jackson's refusal to define the word "woman"). And election results in Virginia, San Francisco, and elsewhere suggest they've underestimated opposition to activist conceptions of public education. Will Republicans also go too far, not only protecting today's norms but challenging sexual freedoms that Americans have long taken for granted?
I honestly don't know.
Create an account with the same email registered to your subscription to unlock access.
Sign up for Today's Best Articles in your inbox
A free daily email with the biggest news stories of the day – and the best features from TheWeek.com
Samuel Goldman is a national correspondent at TheWeek.com. He is also an associate professor of political science at George Washington University, where he is executive director of the John L. Loeb, Jr. Institute for Religious Freedom and director of the Politics & Values Program. He received his Ph.D. from Harvard and was a postdoctoral fellow in Religion, Ethics, & Politics at Princeton University. His books include God's Country: Christian Zionism in America (University of Pennsylvania Press, 2018) and After Nationalism (University of Pennsylvania Press, 2021). In addition to academic research, Goldman's writing has appeared in The New York Times, The Wall Street Journal, and many other publications.
-
Duchess of Gloucester: the hard-working royal you've never heard of
Under The Radar Outer royal 'never expected' to do duties but has stepped up to the plate
By Chas Newkey-Burden, The Week UK Published
-
Are 'judge shopping' rules a blow to Republicans?
Today's Big Question How the abortion pill case got to the Supreme Court
By Joel Mathis, The Week US Published
-
Climate change is driving Indian women to choose sterilization
under the radar Faced with losing their jobs, they are making a life-altering decision
By Theara Coleman, The Week US Published
-
Trump, billions richer, is selling Bibles
Speed Read The former president is hawking a $60 "God Bless the USA Bible"
By Peter Weber, The Week US Published
-
The debate about Biden's age and mental fitness
In Depth Some critics argue Biden is too old to run again. Does the argument have merit?
By Grayson Quay Published
-
How accurate are political polls?
The Explainer And how much should you read into figures ahead of the 2024 election?
By Justin Klawans, The Week US Published
-
How would a second Trump presidency affect Britain?
Today's Big Question Re-election of Republican frontrunner could threaten UK security, warns former head of secret service
By Harriet Marsden, The Week UK Published
-
'Rwanda plan is less a deterrent and more a bluff'
Instant Opinion Opinion, comment and editorials of the day
By The Week UK Published
-
Gen Alpha kids are more informed about the news than you might think, Week Junior survey finds
The Explainer America's youngest generation knows what is going on in the world, and they have something to say
By Theara Coleman, The Week US Published
-
Henry Kissinger dies aged 100: a complicated legacy?
Talking Point Top US diplomat and Nobel Peace Prize winner remembered as both foreign policy genius and war criminal
By Harriet Marsden, The Week UK Last updated
-
Trump’s rhetoric: a shift to 'straight-up Nazi talk'
Why everyone's talking about Would-be president's sinister language is backed by an incendiary policy agenda, say commentators
By The Week UK Published