NRL News

So how far off are the polls?

by | Sep 12, 2022

By Dave Andrusko

November 8th—the date of the pivotal mid-term elections which may possibly signal the impact of the Dobbs decision which on June 24th gave Roe v. Wade its just deserves.

Already pollsters are nervous that they will bungle the outcomes as has been the case repeatedly. Take David Leonhardt, of The New York Times, whose column today is entitled “Are the Polls Wrong Again?  Are Democrats again about to be disappointed by overly optimistic polling?” 

There’s a lot to digest. Let me choose some highlights. 

#1. Here’s his lead. [Underlining is mine]:

The final polls in the 2020 presidential election overstated Joe Biden’s strength, especially in a handful of states.

The polls reported that Biden had a small lead in North Carolina, but he lost the state to Donald Trump. The polls also showed Biden running comfortably ahead in Wisconsin, yet he won it by less than a percentage point. In Ohio, the polls pointed to a tight race; instead, Trump won it easily.

In each of these states — and some others — pollsters failed to reach a representative sample of voters. One factor seems to be that Republican voters are more skeptical of mainstream institutions and are less willing to respond to a survey. If that’s true, polls will often understate Republican support, until pollsters figure out how to fix the problem. (I explained the problem in more depth in a 2020 article.)

This possibility offers reason to wonder whether Democrats are really doing as well in the midterm elections as the conventional wisdom holds. …

But the Democrats’ strength in the Senate campaign depends partly on their strength in some of the same states where polls exaggerated Democratic support two years ago, including the three that I mentioned above: North Carolina, Ohio and Wisconsin.

That is “a warning sign” in the words of Nate Cohn, The Times’s chief political analyst, “for both the Democratic Party and for the polls.”

#2. “The unavoidable reality is that polling is both an art and a science, requiring hard judgments about which kinds of people are more or less likely to respond to a survey and more or less likely to vote in the fall,” Leonhardt writes. “There are still some big mysteries about the polls’ recent tendency to underestimate Republican support.” 

You can’t emphasize this enough. For whatever reason, the sample pollsters almost always use includes many more Democrats and many fewer Republicans.

#3. Switching over to Cohn’s article, which also ran today, Cohn notes that in Wisconsin, pollsters have pro-abortion Democrat Mandela Barnes with a comfortable lead over incumbent Sen. Ron Johnson. 

But in this case, good for Wisconsin Democrats might be too good to be true. The state was ground zero for survey error in 2020, when pre-election polls proved to be too good to be true for Mr. Biden. In the end, the polls overestimated Mr. Biden by about eight percentage points. Eerily enough, Mr. Barnes is faring better than expected by a similar margin.

And building on that, #4.

Democratic Senate candidates are outrunning expectations in the same places where the polls overestimated Mr. Biden.

It raises the possibility that the apparent Democratic strength in Wisconsin and elsewhere is a mirage — an artifact of persistent and unaddressed biases in survey research.

If the polls are wrong yet again, it will not be hard to explain. Most pollsters haven’t made significant methodological changes since the last election. The major polling community post-mortem declared that it was “impossible” to definitively ascertain what went wrong in the 2020 election.

Cohn covers his bets at the end: 2022 could really be different. But…

The pattern of Democratic strength isn’t the only sign that the polls might still be off in similar ways. Since the Supreme Court’s Dobbs decision on abortion, some pollsters have said they’re seeing the familiar signs of nonresponse bias — when people who don’t respond to a poll are meaningfully different from those who participate — creeping back into their surveys.

Brian Stryker, a partner at Impact Research (Mr. Biden is a client), told me that his polling firm was getting “a ton of Democratic responses” in recent surveys, especially in “the familiar places” where the polls have erred in recent cycles. …

But the pattern is worth taking seriously after what happened two years ago.

I assume what he is hinting at without saying so specifically is that Republicans once again are not answering pollsters when they call and that as a result the numbers once again—including on abortion—will prove to be well off.

Categories: Polling