Skip to main content

How much weight do you give evaluations?

By now, a lot of academics (or at least academic economists) have heard about Scott Carrell and James West's paper on professor quality. They use data from the Air Force Academy (where students are randomly assigned to core courses and take common exams) and find that the 'value-added' of professors in intro courses is both positively correlated with student evaluations and negatively correlated with 'value-added' in follow-on courses (which the authors talk about as evidence of 'deep learning'). Basically, professors who seem to be better at inducing 'deep learning' in their intro students are also more likely to get lower evaluations from those students.

On the one hand, I have to say that this feels kind of validating for people like me - that is, I care a lot about helping my students learn to think critically and I think I put a lot of effort into trying to foster deep learning, rather than allowing my students to just memorize stuff, but I rarely get stellar evaluations (at least in my Principles course) and I've often told myself that my focus on deeper learning is one of the reasons why my evaluations aren't as high as some of my colleagues. Certainly, some of the open-ended comments from students on the anonymous department evaluations could be interpreted as them resenting that I ask them to actually think.

But on the other hand, I can't be quite so cynical as to blame it all on my students. I think about teachers, like those in Ken Bain's What the Best College Teachers Do, and I know that they are able to not only promote deep learning but they are able to do so in a way that students appreciate and respond to. So clearly, I have work to do...

But I have to say, one of the best things about tenure is that I don't really have to care about my teaching evaluations. I know that is exactly the kind of thing that makes a lot of people think tenure is a bad thing, and I do realize that for some people, the 'threat' of bad evaluations is the only thing motivating them to care about their teaching at all. But a) I don't think anyone could seriously argue that me not caring about my evaluations is equivalent to me not caring about teaching and b) since I'm going to keep working on my teaching regardless, the anonymous student evaluations done for my department tend to just stress me out without giving me much useful feedback. I get much more useful information from the end-of-course surveys I have students do that are tailored to the individual courses, which I will be talking more about soon...

Comments

  1. Dr. Sanford AranoffJune 17, 2010 at 5:29 AM

    Words like deep thinking and critical thinking are good, but not enough. What we need to understand is that we must start from basic principles. Teachers must know how students think, and build from there using the principles and logic. See "Teaching and Helping Students Think and Do Better" on amazon.

    ReplyDelete
  2. What struck me most about the Carrell/West study were the unique conditions under which the USAF operates:

    1. Very small classes.

    2. A large number of faculty.

    3. Random assignment of students to sections.

    4. What appears to be close to random assignment of faculty to sections.

    5. Very clear sequencing of courses in many programs.

    I seriously doubt whether their results could be replicated in any system in which a program has a small number of sections, or a small number of faculty, or student choice of faculty in introductory (and follow-on) courses, or....

    So while I'm sympathetic with the effort, I think a lot of the comment on the study overstates the conclusions that can be drawn from it.

    ReplyDelete
  3. @Doc: There was an InsideHigherEd article that made a similar point (i.e., hard to see how to implement a system based on the results). I agree it would be unrealistic to try to evaluate faculty based on how their students do in follow-on classes but what was nice about the Carrell and West study is that since they didn't have any of those issues, one could say their results DO confirm that traditional teaching evaluations may be 'rewarding' the wrong things. So maybe the implication isn't that any other school should try to replicate the results, or do away with evaluations, but rather, should be thinking about how to put student evaluations in the proper context?

    ReplyDelete
  4. Well, I'd agree with that. I've thought for decades that standard CTEs are very limited instruments. One thing I have advocated (unsuccessfully) is, in fact, doing retrospective assessments--asking people about to graduate which courses contributed the most to their ability to learn and work with the material? I don't know of any place that does that...

    ReplyDelete

Post a Comment

Comments that contribute to the discussion are always welcome! Please note that spammy comments whose only purpose seems to be to direct traffic to a commercial site will be deleted.

Popular posts from this blog

What are the costs?

I came across an interesting discussion about a 19-year-old intern who was fired from The Gazette in Colorado Springs for plagiarism. There appears to be some controversy over the fact that the editor publicly named the girl in a letter to readers (explaining and apologizing for the plagiarism), with some people saying that doing so was unduly harsh because this incident will now follow her for the rest of her career. I was intrigued by this discussion for two reasons - one, it seems pretty clear to me that this was not a case of ignorance (as I have often encountered with my own students who have no idea how to paraphrase or cite correctly) and two, putting aside the offense itself, I have often struggled with how to handle situations where there are long-term repercussions for a student, repercussions that lead the overall costs to be far higher than might seem warranted for the specific situation. As an example of the latter issue, I have occasionally taught seniors who need to p

What was your high school economics experience like?

As I mentioned in my last post , I am asking my Econ for Teachers students to reflect on their reading by responding to discussion prompts. It occurred to me that it wouldn't be a bad idea for me to share my thoughts on those issues here and see if anyone wants to chime in. For this week, the students were asked to read the California and national content standards , an article by Mark Schug and others about why social science teachers dread teaching economics and how to overcome the dread, an article by William Walstad on the importance of economics for understanding the world around us and making better personal decisions (with some evidence on the dismal state of economic literacy in this country), and another article by Walstad on the status of economic education in high schools (full citations below). The reflection prompt asks the students to then answer the following questions: What was your high school econ experience like? What do you remember most from that class? How do

When is an exam "too hard"?

By now, you may have heard about the biology professor at Louisiana State (Baton Rouge) who was removed from teaching an intro course where "more than 90 percent of the students... were failing or had dropped the class." The majority of the comments on the Inside Higher Ed story about it are supportive of the professor, particularly given that it seems like the administration did not even talk to her about the situation before acting. I tend to fall in the "there's got to be more to the story so I'll reserve judgment" camp but the story definitely struck a nerve with me, partly because I recently spent 30 minutes "debating" with a student about whether the last midterm was "too hard" and the whole conversation was super-frustrating. To give some background: I give three midterms and a cumulative final, plus have clicker points and Aplia assignments that make up about 20% of the final grade. I do not curve individual exams but will cu