Final polls show Democrat Ralph Northam holding a slight edge over Republican Ed Gillespie before Tuesday’s election for governor of Virginia. But a lot of uncertainty remains, especially given the typically low turnout in off-year elections and recent poll misfires in Virginia. Polls significantly underestimated Republican candidates in the 2013 gubernatorial election and 2014 contest for U.S. Senate.
Two recent surveys illustrate the challenge of determining who is actually likely to vote. Based on different methods of identifying likely voters, Northam’s lead could be as large as eight points and as small as two points.
How the two surveys were conducted
The first survey was conducted by The Washington Post and George Mason’s Schar School. Its sample was based on two components: random-digit dialing of landline and cellular phones in Virginia and a random sample from a statewide list of registered voters maintained by Aristotle, Inc. The two samples were combined and weighted together (more details are here).
Registered voters reached through either sample were asked how likely they are to vote in Tuesday’s election. For respondents contacted from the list of registered voters and confirmed to match the intended voters, official records of voting in the 2012, 2013 and 2016 elections were used as indicators of past voting. For respondents contacted via random-digit dialing, self-reported turnout in these three elections was used.
The voter list database was used to estimate the probability of voting in the previous election for governor depending on a person’s turnout in the previous two presidential and previous gubernatorial election. The likely voter model first excluded registered voters who said they were not certain or probable to vote in the governor’s election, and weighted this group by their estimated probability of voting based on whether they voted in 2012, 2013 and 2016.
The poll then re-weighted all voters who said they would at least probably vote by their estimated probability of turnout, and used this weight to produce estimates of Northam and Gillespie’s share of the vote.
The second survey was conducted by SurveyMonkey. SurveyMonkey recruits respondents from the roughly 3 million people who complete surveys on the company’s platform every day. In Virginia, a subsample of respondents to these surveys — which includes polls of community groups, companies, churches and other organizations — was invited to participate in a second survey with the prompt: “Where do you stand on current events? Share your opinion.” Respondents are weighted to match Census estimates of the demographics of registered voters (more details are here).
The SurveyMonkey poll identified likely voters and weighted on their estimated probability of turnout using the same questions that were asked of the Post-Schar School poll’s random-digit dialing respondents.
Different definitions of likely voters = different results
Despite these different methods, both surveys found Northam with a slight lead over Gillespie of five percentage points among likely voters. The Post’s telephone poll, conducted Oct. 26-29, shows Northam leading 49 to 44 percent among likely voters, while the SurveyMonkey online poll, conducted Oct. 20-Nov. 2, shows Northam leading 51 to 46 percent.
But as the graph below shows, the size of Northam’s lead depends on who is defined as likely to vote.
If each poll included only voters who said they had voted or were absolutely certain to vote — excluding those who said they would “probably” vote — Northam’s lead grows to 7-8 points. This suggests that Northam’s supporters are more motivated to vote.
But among voters who say they voted in the last election for governor in 2013, Northam’s margin is just plus-two on the telephone poll, plus-four on the online survey. This suggests that Gillespie has an edge among habitual voters who are likely to vote in a low-turnout contest.
As this shows, different methods of identifying likely voters should help explain the divergence among the recent public polls in Virginia. A lot depends on whether polls rely more on previous voting behavior or current enthusiasm.
Of course, it is impossible to know which method is “right” in advance of any given election. There are valid critiques of asking respondents their vote intention, which is why campaign pollsters prefer to sample from lists of registered voters and draw on records of past voting behavior rather than taking respondents at their word. But past voting behavior can be misleading if a campaign succeeds in getting irregular voters to the polls.
Regardless, the results from both surveys suggest the race may be tightening in its final weeks. Democrats have been more energized than Republicans throughout much of the year, although now Gillespie’s supporters show signs of greater engagement. The more the electorate resembles voters who cast ballots in every general election, the better Gillespie stands to perform Tuesday.
Scott Clement is polling director at The Washington Post. Mark Blumenthal is head of election polling at SurveyMonkey.