Instead of learning statistics from a robotic professor, learn it from an actual robot!

Steve Kolowich reports:

In experiments at six public universities, students assigned randomly to statistics courses that relied heavily on “machine-guided learning” software—with reduced face time with instructors—did just as well, in less time, as their counterparts in traditional, instructor-centric versions of the courses. . . .

A total of 605 students were randomly assigned to take the course in a “hybrid” format: they met in person with their instructors for one hour a week; otherwise, they worked through lessons and exercises using an artificially intelligent learning platform developed by learning scientists at Carnegie Mellon University’s Open Learning Initiative.
Researchers compared these students against their peers in the traditional-format courses, for which students met with a live instructor for three hours per week, using several measuring sticks: whether they passed the course, their performance on a standardized test (the Comprehensive Assessment of Statistics), and the final exam for the course, which was the same for both sections of the course at each of the universities.

The study was conducted by William Bowen, Matthew Chingos, Kelly Lack, and Thomas Nygren.

I don’t have much to say about this. The results make sense to me: as long as students are willing to do the work, I’d expect that an online system would offer better feedback than the traditional one-size-fits-all homework assignments.

But I think this is a bit silly:

Engaging students, such as professors might by sprinkling their lectures with personal anecdotes and entertaining asides, remains one area where humans have the upper hand.

Sure, I’ve been known to share the occasional personal anecdote. But I think the real value of the professor is his or her depth of understanding of the material.

Also, I don’t understand the claims about the economics. Kolowich writes:

In terms of instructor compensation, the researchers estimated, a machine-guided course featuring weekly face-to-face sessions with part-time instructors would cost between 36 and 57 percent less than a traditional course in which a full professor presides over each 40-student section.

I’d think the savings would be much more. You’re comparing 3 hours a week of a full professor, to one hour a week of a part-time instructor. If the prof gets paid three times what the instructor gets per hour, then shouldn’t the cost savings be 89%? OK, not exactly, someone has to set up the computers etc., so maybe you would get cost savings of only 70% or 80%. But how do you get down to only 36-57% savings??

In any case, I think this sounds like a great idea. My challenge in trying to put together something like this for intro statistics classes (the programmed-learning-and-computer-feedback part, not the get-rid-of-the-instructor part) is that I’m still torn about how to structure the course material itself. I don’t really like the idea of giving students 14 weeks of confidence intervals, paired t-tests, goofy probability problems, and the like. I’m still struggling to organize something that makes sense.

Conditional on the material that’s covered in the course, though (and without tying it to getting rid of the professor) I think programmed learning is great.

5 Responses to Instead of learning statistics from a robotic professor, learn it from an actual robot!

  1. AGH May 23, 2012 at 6:51 pm #

    “But I think the real value of the professor is his or her depth of understanding of the material.”

    In most basic statistics courses, it might be true that a large number of professors have more than adequate ‘depth of understanding.’ Anecdotes, presentation style, and non-verbal communication are all very important when it comes to conveying information effectively, and perhaps even more important than depth, at some levels.

    • Andrew Gelman May 23, 2012 at 7:33 pm #


      I agree that a large number of professors have more than adequate depth of understanding. I was comparing them to a purely automated approach in which that understanding is not accessed by the student.

      I think anecdotes etc are great but they have a positive interaction with depth of understanding.

  2. Joel May 23, 2012 at 8:09 pm #

    More silly is the idea that “engaging students” is tantamount to telling jokes and personal stories.

  3. Manuel Pontes May 23, 2012 at 10:30 pm #

    The big question is the external validity of the study. Are the students sampled at Carnegie Mellon representative of all students in the US who study statistics. I would not be surprised that method of instruction may be contingent on they type of student and I doubt that Carnegie Mellon statistics students are representative of all students studying statistics

  4. Frank in midtown May 26, 2012 at 7:13 pm #

    To answer your request for thoughts about how to structure something like this, I’m struck by the old Army training adage, tell them what you are going to tell them, tell them, tell them what you told them. Class time is cut to a third, so you have to give up two of the three repetitions. Would you rather tell them what the program is going to tell them or tell them what the program told them? While freeing the instructor from repetition, it brings the college learning experience more in line with the Army learning experience.