The only thing revealed by polls on Florida's 'Don't Say Gay' bill is that polling sucks

Do Americans support or oppose Florida's House Bill 1557, which is officially known as the "Parental Rights in Education" but is almost universally described as the "Don't Say Gay Bill"? You'd think it would be an easy question to answer. Call up a few thousand people, ask their opinions, and tabulate the responses. Problem solved.

Except it's not that simple. A lot of people don't pick up the phone or talk to strangers, so pollsters must call many times the number of respondents they need in order to amass a statistically adequate sample. Those respondents must be asked a battery of other questions to determine their demographic characteristics. Because there's no guarantee that the people who answer calls will be representative, their actual numbers must be weighted to correspond to estimates of the makeup of the broader population based on yet other polls. That's before you even ask the question, which can be formulated in ways that encourage or deter certain responses. And of course you have to hope respondents answer honestly, rather than in ways they consider socially or politically desirable but don't correspond to their real opinions.

This hidden complexity helps explain widely divergent results of polls on the Florida bill. Last month, a statewide poll of registered voters showed that a 49 percent plurality opposed the measure, with just 40 percent expressing support. A couple of weeks ago, a national poll of adults showed overwhelming opposition by nearly two-thirds of the public. That poll acknowledged an oversample of LGBT respondents, which called its results into question. A few days later, another national poll, this time of registered voters, showed a slim majority in favor. Unlike the others, that was a so-called tracking poll, in which the same people are repeatedly surveyed over an extended period of time. Just yesterday, finally, a second Florida survey showed majority support for the law among likely Democratic primary voters.

Subscribe to The Week

Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.

SUBSCRIBE & SAVE

Sign up for The Week's Free Newsletters

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.

Sign up

What's going on? One issue is that the polls didn't include precisely the same cohort. Registered voters in Florida, adults around the country, national registered voters, and Florida Democrats could theoretically have different opinions about a controversial measure. Typically, though, you wouldn't expect the range of responses to be quite so large. Moreover, answers would likely correlate pretty closely to party affiliation, which makes the result among Florida Democrats surprising.

The questions were also subtly different. All but three of the polls paraphrased the most debated provision of the bill, which states that "Classroom instruction by school personnel or third parties on sexual orientation or gender identity may not occur in kindergarten through grade 3 or in a manner that is not age appropriate or developmentally appropriate for students in accordance with state standards." Only the tracking poll added context, noting that critics "say [the bill] will block important conversations about LGBTQ issues" and breaking down opinion about specific measures. Which approach is most accurate? There's no obvious answer.

There are also problems inherent to all issue polls. One is that asking people about tradeoffs and consequences can have significant effects on their responses. In a discussion of recent polling concerning a no-fly zone in Ukraine, Washington Post writer Aaron Blake described how variations in wording and discussions of the likely consequences led to reduced support in different surveys. And it's not uncommon for the public to approve an idea in the abstract and then balk at its implications.

More importantly, merely expressing approval or opposition doesn't tell us how much respondents actually care. In legislative and electoral politics, the opinions of a highly motivated minority can outweigh the preferences of a lukewarm majority. Even when poll results are accurate, then, they don't always tell us the state of play.

Finally, there's a possibility that the numbers are just wrong. It was not even two years ago that many polls predicted an overwhelming victory for Joe Biden — including in states that he went on to lose by a considerable margin. Looking beyond Florida, falling response rates threaten to undermine polls' main source of authority — the claim that they capture the opinions of a broad public rather than a select few. Although political scientists and pollsters employ various techniques to address the problem, that's worth remembering whenever anyone trumpets poll results to vindicate their favored position or candidate.

In sum, polls alone don't reliably tell us what Americans think of the Florida bill. And even if they did, that wouldn't tell us how they'll respond when it's actually implemented, whether in ways consistent with supporters' assurances of its limited scope or that bolster critics' claim that it amounts to a gag rule. As my colleague Jim Antle argued, Democrats have stumbled when they've interpreted majority approval for same-sex marriage and antidiscrimination laws as a mandate for jettisoning traditional understanding of gender altogether (consider Ketanji Brown Jackson's refusal to define the word "woman"). And election results in Virginia, San Francisco, and elsewhere suggest they've underestimated opposition to activist conceptions of public education. Will Republicans also go too far, not only protecting today's norms but challenging sexual freedoms that Americans have long taken for granted?

I honestly don't know.