Measuring ideological positions of legislators

Anthony Fowler and Andrew Hall send along a new paper, “Conservative Vote Probabilities: An Easier Method for the Analysis of Roll Call Data”:

We propose a new roll-call scaling method based on OLS which is easier to imple- ment and understand than previous methods and also produces directly interpretable estimates. This measure, Conservative Vote Probability (CVP), indicates the probability that an individual legislator votes “conservatively” relative to the median legislator. CVP is a flexible non-parametric statistical technique that requires no complicated as- sumptions but still produces legislator scalings that correlate with previous roll call methods at extremely high levels. In this paper we introduce the methodology behind CVP and offer several substantive examples to demonstrate its efficacy as an easier, more accessible alternative to previous roll call methods.

The paper has lots of graphs! I don’t know what they think they’re doing, plotting correlations with bar graphs, but other than that the graphs are great.

Here’s my big question. Fowler and Hall talk a lot about how their estimates are close to those obtained by more complicated approaches. But to me the key point of contention nowadays is not the relative positions of legislators, or the increasing gap between the parties—-both of these things can indeed be estimated in any number of simple or complicated ways—-but rather the debate about the absolute positions of the parties. Is it really true, as Poole and Rosenthal’s calculations seem to show, that Democrats in congress have pretty much remained in the same place, ideologically, while the Republican party has moved sharply to the right? My impression is that these shifts are different for economic, social, and foreign policy issues, but Fowler and Hall (and also Poole and Rosenthal) seem pretty committed to the basically one-dimensional model. I’d be curious to see if Fowler and Hall’s approach can address the issue of absolute ideological positions.

10 Responses to Measuring ideological positions of legislators

  1. JC July 23, 2012 at 12:36 pm #

    ” CVP is a flexible non-parametric statistical technique that requires no complicated assumptions but still produces legislator scalings that correlate with previous roll call methods at extremely high levels.”

    Certainly, this method must still make the most problematic assumption (which all of these measure make) — that voting a certain way is clearly the “conservative” vote choice.

    I am unable to read this paper, but does it assume a vote for/against a specific proposals to be a “conservative” or “liberal” vote? This is routinely inaccurate. Take the vote on the budget resolution in the House this March. 10 Republicans voted against their party’s budget plan. Did these 10 Republicans cast a “liberal” vote? Given that many of them are known libertarians, I don’t think it is quite accurate to describe their vote as liberal. Would this measure make that assumption?

  2. Andrew Hall July 23, 2012 at 12:48 pm #

    Thanks for your comment. The paper describes the method in detail and, I believe, addresses your concerns. It can be downloaded here:

    https://dl.dropbox.com/u/21176039/FowlerHall_CVP.pdf

    -Andrew Hall

    • JC July 23, 2012 at 1:44 pm #

      Thanks, Andrew. I appreciate you posting the article. I read through the methodology section briefly. It is very intriguing, but from my reading still makes some precarious assumptions:

      Regarding the procedure for “flipping” the coding of a bill — if Coburn and other Republicans vote for the bill, it may not be that the bill is liberal, but that the bill does not have clear, two-dimensional, ideological content. In “forcing” ideological content onto every bill this measure (like others) is likely over-stating the amount of ideological conflict in Congress.

      This method also still assumes that any vote against a coded conservative bill as a liberal vote. But what if some of those in opposition simply do not consider the bill to be conservative enough? What if you have ends-against-the-middle voting on a bill? What about bills that have both distributive and ideological dimensions, like a Farm Bill re-authorization? In each of these cases the method is either (a) misrepresenting the ideological position a congressperson is taking, or (b) forcing ideological content onto a non-ideological vote.

      Essentially, if I am reading correctly, this measure, rather than identifying the likelihood a member of Congress will take a conservative vote, it is identifying the likelihood a member of Congress will cast a vote that aligns with the votes taken by members of Congress who are typically viewed as conservative. The jump from identifying this likelihood, an assuming the bases for taking that vote are ideological, can be a precarious assumption.

  3. Josh R. July 23, 2012 at 1:06 pm #

    “Is it really true, as Poole and Rosenthal’s calculations seem to show, that Democrats in congress have pretty much remained in the same place, ideologically, while the Republican party has moved sharply to the right?”

    Or moved to the right! To wit: “And over the past few decades, the Democratic Party has shifted strongly in a free-market, anti-regulation, free-trade direction—a shift more consistent with the preferences of the affluent than of the less well off” (Martin Gilens in his Boston Review piece).

  4. Larry Bartels July 23, 2012 at 1:52 pm #

    Andrew, I don’t think your question has an answer. Leaving aside the methodology (though it is always fun and refreshing to rediscover the linear probability model), what would it _mean_ to measure “absolute” ideological positions? With some hard work, it is possible to compare votes in successive Congresses on substantively identical bills (or to scale bills consistently based on a detailed reading of their content, as Josh Clinton has done with minimum wage bills). But even then, voting for or against a $7 per hour minimum wage in the midst of a recession is not the same thing, ideologically, as voting for or against a $7 per hour minimum wage in the midst of an economic boom. If you can’t answer your question even in this very simple case, how can you hope to answer it in general, in different Congresses considering hundreds of very different bills in very different political, social, and economic contexts?

    • Andrew Gelman July 23, 2012 at 2:20 pm #

      Larry:

      Maybe you’re right, but I think Poole and Rosenthal’s method implicitly estimates such absolute shifts.

      • Adam July 23, 2012 at 10:22 pm #

        Suppose Larry is wrong, and there is such a thing as “absolute” ideological positions. My question is whether that absolute measure is meaningful without comparing it to the ideology of the electorate.

  5. steve July 23, 2012 at 7:59 pm #

    can someone please explain to me why political scientists use roll-call votes as indicators of partisan ideology when they, on many occasions, represent anything but a partisan stance on an specific issue? As you all very well know, the success of a bill is often decided well before it gets to the floor and many legislators are released from votes, while others are merely symbolic due to an obvious result (e.g. Obamacare). In fact, most of the partisan wrangling is decided in committees or informally. I understand that these actions or behavior are much more difficult to measure or operationalize, but it seems that some careers are made (and maintained) on these flawed indicators that very rarely measure what they are intended to measure. Just curious. Thank you.

  6. paul gronke July 24, 2012 at 7:26 pm #

    Steve,

    Two answers. First, Poole and Rosenthal originally argued that these were not indicators of ideology but simply of the ability to reduce an apparently large dimensional space to a small number of dimensions. Keith and Howard can speak for themselves, but they have pretty much given up trying to stop people from using Nominate scores as proxies for ideology.

    But as to why it works so well, you should look at the opening chapters of aThe Dance of Ideology. A bill is “decided” because it represents a particular balance of ideological and policy interests. Thus, the vote is revealing of those underlying interests. If the vote is not, then you are left claiming that legislators voted for a bill that was contrary to their votes and/or interests.

    OK, you may say this is a log roll. But keep in mind one log roll implicitly assumes another, balancing log roll–and the system considers ALL votes in a congressional session. You may be able to cherry pick one or another bill where you think the final roll call split is not revealing of “true” ideology, but can you do that for EVERY bill?

    • steve July 24, 2012 at 10:19 pm #

      Dr. Gronke,

      First off, thank you for the response. Yours was an honest answer to an uncomfortable question for some researchers, and I appreciate it.

      Although I agree with you that one log-roll assumes another, each issue represents a different degree of ideological and policy interest for each legislator, assuming that no district (congressional, senate, state, municipality, etc) is identical (an assumption that I believe to be correct). In my experience in legislatures and with legislators, there are relatively few issues that are of interest or importance to them for personal, constituent, and/or ideological reasons. As a result, there is a great deal of deference to their party as well as fellow legislators whom they trust or, as you suggest, they owe a vote to.

      At the same time, even the degree of interest in the issue is not constant over time and is very much impacted by other factors: election year, sporadic events (e.g. gun control following a shooting spree), or economics. Let us also not forget the institutions that are in place that often impact legislation and voting, including agenda setting rules, procedures and constraints on the introduction of bills, germaneness rules, and other voting procedures (yes, these are more applicable in state legislatures where there exists significant variation), as well as levels of legislative professionalism, where some legislatures have full-time staffs that can conduct the research and fact finding that less professional, amateur legislatures cannot.

      I guess my issue was that so much time is spent on using new and different statistical methods to improve the use / interpretation and analysis of roll-call votes, but little work attempting to improve the measure substantively or developing a new measure that more appropriately captures a legislator’s level of partisanship. I know it is difficult and likely that no two legislatures are the same, but should this not be our goal rather than putting a fancier shade of lipstick on the pig or arguing about the which consistency of the mud is better for them to wallow in?

      Again, I appreciate your response and I understand that I am probably preaching to the choir – well at least to some of the choir. But if we are ever going to improve legislative studies and research, we are going to have to commit to doing work that is both meaningful and appropriate. I know that my colleagues and bosses that currently laugh at academic research would appreciate it and maybe those now attempting to de-fund it would as well.