Donald Trump’s approval rating is one of the key news stories of 2017. That’s true not just for political junkies. Approval — or the lack of it — can give the president more or less power as he negotiates with Congress.
How do we know a president’s approval rating? From polls. But how the poll data are processed makes a difference in what the results are. That becomes clear when we look at the graph above, prepared by Alexander Agadjanian, which shows Trump’s 2017 support this year according to two different kinds of polls:
- Adjusted for party. The red dots come from YouGov, a survey organization that adjusts its estimates for party identification. That means it estimates approval among Republicans, independents and Democrats, and then averages that over the proportion of these groups in the population. In doing so, it adjusts for any problems that result when a disproportionate number of Democrats or Republicans respond to a particular survey.
- Not adjusted for party. The black dots come from various other pollsters who did not adjust their polls for party ID.
As you can see, the general time trend is the same for both groups, but the YouGov results are much more stable. The variation in the unadjusted polls must be coming from two sources: first, from polling house effects (i.e., differences in how particular pollsters work); and second, variation in who’s responding to the polls.
[interstitial_link url=”https://www.washingtonpost.com/news/monkey-cage/wp/2017/06/05/its-time-to-bust-the-myth-most-trump-voters-were-not-working-class/”]It’s time to bust the myth: Most Trump voters were not working class.[/interstitial_link]
Here’s what’s key: While the poll aggregates appear to be capturing the trend in Trump’s approval, if you’re following individual polls or short groups of polls you’ll be getting jerked around by artificial differences. Agadjanian wrote to me to explain:
Over the last year, I’ve gotten really interested in differential partisan nonresponse bias and considering the idea of weighting by party identification. I wanted to check for partisan nonresponse bias effects in Trump approval rating polls (similar to the way done in this post on your blog), and I came up with the below graphs about a week ago. I explain details here, but interestingly there appears to be a strong relationship (R2 = .45) between the unweighted partisan distribution of a poll and Trump evaluation if you only look at approval rating polls that don’t weight by party or past vote. It feels like a crude way of measuring partisan nonresponse, but it’s still interesting that 1) a fairly strong relationship already exists and 2) it’s occurring outside of just a campaign season context. For the latter point, I’ve been contemplating whether this suggests partisan nonresponse is a systemic issue that polls of any kind must always deal with.
Here’s where you can find a more detailed explanation of how Agadjanian interpreted the results.
We can overreact to each poll. It’s good to keep in mind that we need to adjust for each poll’s biases as much as possible.
[interstitial_link url=”https://www.washingtonpost.com/news/monkey-cage/wp/2017/05/08/why-did-trump-win-more-whites-and-fewer-blacks-than-normal-actually-voted/”]Why did Trump win? More whites — and fewer blacks — actually voted[/interstitial_link]
Disclosures: I have worked with YouGov (although not directly on these popularity polls) and they have supported my group’s research. Also I have an intellectual stake in these issues: My co-authors and I have published papers on adjusting polls for variation in party identification (Post-stratification without population level information on the post-stratifying variable, with application to political polling, with Cavan Reilly and Jonathan Katz in 2001; and The mythical swing voter, with Sharad Goel, Doug Rivers and David Rothschild in 2016). But I don’t see this post as an advertisement for YouGov. For one thing, post-stratifying on party ID is something that any pollster can do. The data are out there and the methods and software are free and open source.