Archive | Academia

A Modest Proposal to Improve the Peer Review Process

The following is a guest post by political scientist Scott Gehlbach (@sgehlbach) of the University of Wisconsin–Madison.

*****

The peer-review process, if not broken, is seriously under strain. Editors are forced to make hasty decisions based on imperfect signals from referees. Referees, in turn, are overburdened with review requests. And authors are at the mercy of referees who are not always qualified to evaluate all parts of a submission.

These three problems have a common cause: as a discipline we are asking referees to do too much. The typical review request takes the form: “Please evaluate this submission as a possible contribution to…” The referee process could be improved by adding a sentence that says, “As an expert in X, your thoughts on Y would be especially valuable,” where Y could be research design, a formal model, country context, framing, or any other element of the paper. In principle, the request could specify Y1, Y2, etc., though the point would be for the list to be less than exhaustive.

As an analogy, think of dissertation-committee members, who typically concentrate on parts of the dissertation where they have particular expertise. We don’t expect the Africanist to offer extensive comments on the model, or the formal theorist to advise on the ethnography, unless these specialists happen to be one and the same person.

Focusing reviewer effort in this way will allow editors to make better decisions, as it will be easier to extract the signal from the noise in referee reports. It will take a bit more work from editors at the front end of the review process, but I expect that it will save time at the back end. And I’m not convinced that it will take that much more time ex ante: editors already choose referees based on expertise (so, in the formulation above, they know X), and the increased use of desk rejection means that editors give manuscripts at least a cursory read on initial submission (so it shouldn’t be too difficult to fill in Y).

There are also benefits to other stakeholders. Referees will appreciate the sanction to direct effort to parts of a manuscript where they have the most expertise. And to the extent that all of this produces better and speedier decisions and more focused referee reports, authors (and ultimately readers) should profit.

A final note: There is a decentralized version of this reform that can provide many of the same benefits. Even if editors choose not to suggest that referees focus on particular elements of a submission, reviewers can still choose to restrict their comments in a way that reflects their substantive or methodological expertise. Indeed, I suspect that some referees do this already, but the key is to make it explicit: “In reviewing this manuscript, I primarily restrict my attention to Y.” Such a statement clarifies to editors and authors what the referee has, and has not, taken responsibility for.

I understand that there might be strategic considerations at play here. I leave those as an exercise for the comments section.

Continue Reading

Top 5 Most Popular Articles from Electoral Studies Available for Free Download Through End of October!

What a great idea! Elsevier Press has decided to make the 5 most popular articles from Electoral Studies during the first half of the year available for free download through the end of October. (Caveat: I am on the Board of Editors of Electoral Studies, but had nothing to do with this decision.) Would love to see more political science journals adopting such a policy, and would be happy to give a similar shout out in The Monkey Cage if any publishers are interested.

Here are the five articles with links:

1) Explaining voter turnout: A review of aggregate-level research

2) Democratic electoral systems around the world, 1946-2000

3) Understanding unequal turnout: Education and voting in comparative perspective

4) The embarrassment of riches? A meta-analysis of individual-level research on voter turnout

5) Civic duty and turnout in the UK referendum on AV: What shapes the duty to vote?


Although I am mainly reporting this as a public service announcement, it is also pretty interesting to note that four of these five article are on turnout (and the fifth is an article accompanying an extremely popular dataset), and two are meta-analyses of previous studies.  Are we witnessing a renaissance of the study of turnout?  A growing popularity of meta-analyses in political science?

Continue Reading

Academic Conferences Are “Lumbering Dinosaurs.” Can Anything Change Them?

The conventional political science conference is a dinosaur, large, lumbering, and increasingly ill-suited for its environment, although extinction does not appear imminent.

So begins a recent article by Georgetown political scientist Mark Rom.  On the eve of the next American Political Science Association meeting, his take is well worth digesting.

For starters, consider this.  In 2008, Rom hired 8 graduate students and offered each student $10 to fill out a questionnaire for every APSA panel they attended.  There were 14 timeslots for panels during the conferences, giving students the chance to make $140.  But on average, the students attended only 2-3 panels, not the 14 possible.  To be sure, most people who attend political science conferences, or academic conferences, or maybe conferences of any sort, don’t attend all the panels available.  People have other things to do at these meetings.  But it does suggest that panels are hardly a big draw, even when there’s a little money in it for the person attending.

Rom then reviews the problems with organizing conferences according to panels: the quality of the papers on the panels varies dramatically, the quality of the presentations also varies, the panels may not align with what attendees want to see (such as when they’d like to see different papers being presented simultaneously on different panels), and the presenters often get poor feedback.

Rom proposes what he calls the “customized conference.”  He would eliminate panels and create two kinds of presentations: “teaching” and “learning.”  Teaching presentations are for more polished projects—where the presenter can teach the audience something.  They would resemble traditional presentations, with a scholar describing research findings and answering questions. Learning presentations are for works in progress—where the presenter still has much to learn and would benefit from feedback.  These would entail smaller-scale interactions, perhaps even one-on-one, and would more resemble traditional poster sessions at political science meetings.

Rom argues that teaching presentations would be selected via a process of on-line voting.  The ones that scholars indicated they most wanted to hear would be the ones formally incorporated into the conference.  Rom further argues that learning presentations needn’t be limited at all—except by whatever space constraints there are at the conference.

With this arrangement, the argument goes, the quality of presentations would improve, people would be more likely to see what they want to see, and people with research-in-progress would get better feedback.  The article has responses to many potential criticisms of the system, particularly to the notion of allowing people to vote on presentations.

Naturally, I have my own opinions on this, but let me turn it over to any commenters.  There is an APSA Task Force on Public Engagement that is, among other things, going to think about political science conferences and how to redesign them.  I’m part of the task force and I’d welcome any thoughts.

[Photo credit: ferswriteshoe.]

Continue Reading

Sex and Networking at Academic Conferences

Large academic conferences are prone to create awkward social situations, especially for those who are not (yet) well integrated into a field. Imagine walking around the Chicago Palmer House lobby among 7,000 political scientists who have gathered for the American Political Science Association annual meetings. You know that many in attendance exercise gate keeping powers over jobs you would like, awards you want to win, or journals you would like to publish in. Yet, you only know these people by name and they have no clue who you are. You could wait around in the lobby for an opportunity to grab the person with the familiar name tag. You might be a bit more strategic and selectively come up to VIPs of your choice after panels. Or, you could send out e-mails in advance in the hope of setting up brief meetings over coffee. Regardless of what option you pick, it will be awkward unless you are a natural at this (which most people are not). It can be doubly stressful for young women given that most “VIPs” are older males.

USC professor Brian Rathburn tried to provide some helpful advice for young academics navigating the scene but packaged this in “the worst metaphor ever,” as Dan Drezner put it. He then pulled the post (a cached copy is here, his explanation is here). I don’t have too much to add that hasn’t already been said by Dan Nexon,  Laura Sjoberg, Steve Saideman, and Dan Drezner. But I do want to reiterate one point that Dan Drezner makes:

 You don’t have to network at all.  It likely helps your professional development a little bit on the margins, but not nearly as much as you would think.  The opportunity costs are small compared to researching and publishing good work.  Pour your manic energy into the latter far more than the former, and don’t fret that you’re missing all the cool parties if you don’t feel like schmoozing.

I think this is both right and potentially useful for mental sanity. Small talk at conferences is not going to get your article accepted in that prestigious journal nor will it land you a job at that university you always wanted to be at. It is important to get to know the people in your field but that is a gradual process much of which takes place after people start inviting you because they like your work. Stay focused on meeting people with whom you share intellectual interests and don’t be too worried if some other grad student manages to line up coffees with all the “big people.” If you have to spend time in lobbies at all, consider playing bingo rather then seeking opportunities to have small talk with “VIPs.”

ps1. I admit it was cheap to put sex in the title. But some of the links really do talk about sex and sexual harassment at conferences.

ps2. I actually did first meet Brian Rathburn at a conference years ago after he e-mailed me. We had a nice chat and I have nothing per se against this strategy. Indeed I often and gladly meet people I don’t know for brief chats at conferences, although I have never set up such networking chats myself. Very often these chats are enjoyable and intellectually engaging. I just don’t think they are very important in the scheme of things. Note that you can make a bad impression as well in such settings, especially if you have nothing of interest to say. So: do it if you are comfortable but don’t fret about it if you would rather not.

Continue Reading

So You Are Starting Your First Year at a Research University…

It’s August—and while that means a dearth of news about American politics, it also means that across the country, recent high school grads are getting ready to start up at a dizzying array of colleges and universities.  I’d be surprised if this blog has much of a readership among people in their late teens.  But having spent some 15 years at research universities, I thought I’d offer a few thoughts about starting undergraduate life at one, and would be curious to get others’ takes in the Comments.  I must have learned something in those eight years living in undergraduate dorms, right?  (Don’t worry—I wasn’t on the eight-year plan.  I was a residential adviser for most of my time in graduate school.  Hm, you still look worried.)

Research universities aren’t just a scaled-up version of a high school, with more students and better sports teams—they are organized differently, and understanding that organization is one of the first tasks of the entering undergraduate.  Case in point: in my high school, any study of literature would have been in the English Department, from Thoreau to Tolstoy.  I knew I liked literature, so when I got to college, I sought out a well-reputed class in the English Department.  But it was only months later, over the winter holidays, that I actually had time to read the course catalog, which was several hundred pages long.  And I realized, belatedly, that literatures written in foreign languages were taught in separate departments—Slavic Languages and Literatures, Germanic Languages and Literatures, etc.  Or else in the Comparative Literature Department.  I also realized that there were whole fields I had never encountered in high school—computer science, sociology, anthropology, to name just a few.

If I were starting again, I’d spend a lot more time reading (or now browsing) the course catalog, to get a better sense of how the fields at a university are organized.  I wouldn’t just read up in the fields I was most interested in.  In fact, I’d read up mostly in the fields I knew nothing about.  The less familiar the Department’s name, the better.  And I’d also spend more time asking people about the different fields, their main tools, their driving questions, their intellectual progress.  High schools are frequently organized by topic area, while universities are organized in part based on different disciplinary toolkits.  You might really like a subject like European history, but also find that the tools you want to use to make sense of that history are actually those of an anthropologist.  Or a computer scientist.  Or an economist.

In that is also a thought about picking classes, to the extent that first-year requirements leave room for choice.  Good classes convey facts, sure.  But they also convey ways of thinking and ways of learning.  More than the specific facts, it is those ways of thinking and learning that you are likely to retain years later.  So if the instructor of a course thinks about problems in a novel or compelling way, give the course a shot—even if you never imagined taking a class on pre-modern Chinese diets.

OK, so I would have spent more time asking people about the academic disciplines—but who, exactly?  Research universities are massive and busy, and some aren’t exactly brimming with people who will stop and explain the intellectual organization of the contemporary university to a wayward first-year.  But that’s where your advisers, teaching assistants, professors, and deans come in.  Harvard professor Richard Light has studied what makes for a successful college experience, and one of his main take-aways is that the students who get to know their instructors have richer college experiences.  His advice: make it a goal to get to know one instructor a semester.  That might mean balancing a few of those big lecture classes with smaller seminars.  It might mean thinking hard about a problem, and then heading to office hours to ask about it.  It might mean asking your teaching assistant why she went into a particular field.  Or it might mean asking a professor about her research, and seeing if you can get involved in it.  They’re called “research universities” for a reason—and yet, many students spend years on university campuses without getting involved in one of their signature activities.

Which brings me to… getting involved in general.  But not getting too involved.  In my high school in the 1990s, and maybe in yours today, a lot of people got involved in a lot of activities: sports teams, student government, religious groups, high school newspapers, weekend jobs, you name it.  But whereas high school life is highly structured, starting at dawn and going well into the evening in some cases, undergraduate life is much less so.  You might find yourself in class for 15 hours a week, leaving a lot of time for other pursuits.  Three weeks into college life, I’ll bet most first-year students couldn’t physically go back to their high school schedules if they tried.  (And having taught classes early on Monday mornings—well, 9:30 am—I have solid evidence of that.)  But the organizational and extra-curricular life at universities is a lot more specialized than that in high school.  It’s not the same people running every activity or doing every on-campus job.  So learn about lots of activities, organizations, and jobs, sure—but plan to devote your time to just a few, and to do those well.

Continue Reading

New issue of Symposium magazine

Symposium magazine (“Where Academia Meets Public Life”) has some fun stuff this month:

Learning to Read All Over Again
Lutz Koepnick
What produces better students – reading in print or reading on-line? The answer is both.

The Elusive Quest for Research Innovation
Claude S. Fischer
Much of what is considered “new research” has actually been around for a while. But that does not mean it lacks value.

Science Journalism and the Art of Expressing Uncertainty
Andrew Gelman
It is all too easy for unsupported claims to get published in scientific publications. How can journalists address this?

A Scientist Goes Rogue
Euny Hong
Can social media and crowdfunding sustain independent researchers?

Still Waiting for Change
Sylvia A. Allegretto
Economists and policymakers alike are ignoring a huge class of workers whose wages have been effectively frozen for decades.

One Professor’s Spirited Enterprise
Bob Benenson
A burgeoning distilling program has successfully combined science and business at Michigan State University.

Slow and Fast Learning in the Digital Age
Linda Essig
The proliferation of online learning tools requires us to take a closer look at how we think, teach and learn.

The authors of these articles include a professor of German and film studies, a sociologist, a reporter/novelist, an economist, a food writer, and a professor of arts management. Enjoy.

Continue Reading

APSA Football (Soccer) Game

Jeffrey Staton and I are organizing another soccer game at the Annual Political Science meetings in Chicago Saturday August 31. It will start at 8:30AM in Grant Park, probably Butler field or near the Lincoln monument. We’ll do an early start to secure a field and to allow people with late morning panels to kick the ball around at least for a bit. You are welcome to join later too. A group will leave from the Palmer House at 8:15 sharp or you can come find us in the park. We’ll be the old geezers with the deft skills. If you would like to be added to the e-mail list send me a note and I will add you to the Evite. Please put APSA Soccer in the subject of your e-mail.

Continue Reading

Journal of Experimental Political Science (JEPS) Now Open For Submissions

JEPS

As of today, The Journal of Experimental Political Science (JEPS), which I am co-editing with NYU political scientist Rebecca Morton, is open for submissions!  JEPS is a double-blind peer-reviewed journal published by Cambridge University Press, and is the official journal of the Experimental Research Section of the American Political Science Association.

We will be using the Editorial Manager system for accepting submissions, so those interested in submitting a paper should login here; you can also register to serve as a reviewer.  Before doing so, please go through the instruction for contributors (available here, and also posted below the break).  Please also note that if you are currently a reviewer for the AJPS, JOP, or a member of the Experimental Political Science section, you should be receiving an email within the next 24 hours with a login and password for the JEPS Editorial Manager website.  If you wait until you have received that email, you won’t have to go through the process of registering for the JEPS site (although you will receive a new password).

If you have research that fits the journal, please do not hesitate to submit it for consideration for publication!

Continue Reading →

Continue Reading