Skip to main content

TBL: Peer evaluations and team grades

[See previous posts for TBL basics, readiness assessments and team applications]

In this post, I'll go over how my students' course grades are calculated and the role of peer evaluations. I think the peer evaluations are probably the part of TBL that students are most worried about at the beginning of the semester, and the aspect that I get asked about the most by faculty. I want to say right off the bat that in the four semesters (eight classes) I've used TBL, I have not seen a single evaluation that appeared to be a student or team trying to 'game the system' (i.e., 'rewarding' a friend or 'punishing' someone unfairly). I've seen a few where students seemed to be not putting in much effort or thought but it has never caused someone's grade to be different than I thought made sense.

Grade weights
Many TBLers have the students themselves determine how much weight will be given to team and individual activities. The way Larry Michaelson and others do it is to have the teams discuss and then send a representative to meet with other team representatives in a fishbowl-type discussion. While I can see how that could be great for getting student buy-in and for building team cohesion, I didn't feel comfortable doing it with my classes so I simply set the weights myself. For the data class, 25% of the final grade is based on team activities (that's 18% from the team RAs and 7% from the team applications). The other 75% of the grade comes from the individual RAs (10%), individual participation based on PollEverywhere responses and pre-class homeworks (10%), two in-class exams (20%), and two writing projects (15% and 20%). Thus, 55% of a student's grade is based on summative assessments (the exams and writing projects), which I think is enough to differentiate the students but not so much that they can ignore/slack off on all the other stuff. I am constantly debating with myself whether I should give more or less weight to things like participation but this mix seems to be working.

Peer evaluations
There are different ways to do the peer evaluations but what I do is have students give a numeric score to each member of their team (not including themselves) and those scores must add up to 100. They must also provide a qualitative explanation of those scores, and those comments are passed on (anonymously) to each student. On the evaluation form, which students complete on Blackboard, the instructions say (borrowed from materials on the TBL website):
"Evaluate the contributions of each person in your group except yourself, by distributing 100 points among them (that is, when you are done, the total points assigned to everyone should sum up to 100). You must provide comments for each person. These comments -- but not who provided them -- will be passed onto your teammates. Your score should reflect your judgment of such things as Preparation (did they come to class prepared?), Contribution (did they contribute productively to group discussion and work?), Respect for others (did they encourage everyone to contribute and listen respectfully to different opinions?), and Flexibility (were they flexible when disagreements occurred?). It is important that you differentiate between people who truly worked hard for the good of the group and those you perceived not to be working as hard on group tasks (NOTE: If you give everyone pretty much the same score when it is not truly deserved, you will be hurting those who did the most and helping those who did the least)."
These evaluations are done twice a semester: the mid-semester evaluations provide students with feedback so they can adjust behavior if necessary, and then the end-of-semester evaluations are the ones that actually 'count'. In order to make sure that students give both numeric and qualitative feedback, I give them individual points for completion.

Incorporating evaluations into grades
Most faculty using TBL tend to use the evaluation scores either as a multiplier applied to the team part of the grade, or as a separate component of the course grade (see the TBL website for a discussion of both). I use the former so I take the peer evaluation scores and convert those to a weight. Since I mostly have teams of 6, the average score for each student is 20 (e.g., if someone wanted to give everyone on the team the same score, that score would be 20) so I start by taking an individual's average score as a percentage of 20. Thus, really good team members will have weights of greater than 1. I played with different ways of calculating the weight (e.g., an individual's average score as a percentage of the lowest score on the team) but finally decided just the score over 20 and I cap the weights at 1.25. So let's say a team has perfect scores on their team RAs and team applications (a situation that has not actually happened in any of my classes), and one member of the team is the clear leader and has high scores from everyone, then that person could actually get 125% for the team portion of their grade. If that person were also to have perfect scores on all their individual assignments, they could actually have more than 100% of the points possible for the semester (again, this has never actually happened).

Do students take it seriously and think it's a fair process?
As mentioned above, I have not yet seen any evidence that students try to game the system. I also have not had any complaints from students about their evaluations being unfair. I think the qualitative feedback helps with this tremendously. Even if one student were to unfairly criticize a teammate, it would be clear from the other team members' comments that the student was out of line. At the same time, when a particular student is not pulling their weight, that generally shows up in comments from multiple teammates, not just one. One thing I find fascinating is how much students will ding a teammate for being absent; comments like "he has missed a lot of classes" or "she doesn't tell us when she's going to miss class" almost always accompany low scores. And those comments matter: I have seen quite a few students start coming to class more regularly after they get their mid-semester evaluations. I've actually considered not having the PollEverywhere responses count for points (and giving more weight to the team portion of the grade instead), since those PE points are often seen by the students as just points for attendance (which, really, they kind of are).

I think the instructions, asking students to think about things like respect for others and flexibility, are also important for getting good qualitative feedback. I have been pleasantly surprised how often I will see comments like, "John always makes sure to get everyone's opinion before we finalize our answer" or "Jane always has a strong opinion, and it's usually right, but she's also good about listening to other people's explanations and admitting when she's wrong" or "I wish Jim would contribute more; when he does, he usually has good points but he's kind of quiet." In general, the majority of comments, both good and bad, are respectful and actually constructive.

In my next post, I'll share some of the feedback I've collected from students about their experience with TBL...

Comments

Popular posts from this blog

What are the costs?

I came across an interesting discussion about a 19-year-old intern who was fired from The Gazette in Colorado Springs for plagiarism. There appears to be some controversy over the fact that the editor publicly named the girl in a letter to readers (explaining and apologizing for the plagiarism), with some people saying that doing so was unduly harsh because this incident will now follow her for the rest of her career. I was intrigued by this discussion for two reasons - one, it seems pretty clear to me that this was not a case of ignorance (as I have often encountered with my own students who have no idea how to paraphrase or cite correctly) and two, putting aside the offense itself, I have often struggled with how to handle situations where there are long-term repercussions for a student, repercussions that lead the overall costs to be far higher than might seem warranted for the specific situation. As an example of the latter issue, I have occasionally taught seniors who need to p

What was your high school economics experience like?

As I mentioned in my last post , I am asking my Econ for Teachers students to reflect on their reading by responding to discussion prompts. It occurred to me that it wouldn't be a bad idea for me to share my thoughts on those issues here and see if anyone wants to chime in. For this week, the students were asked to read the California and national content standards , an article by Mark Schug and others about why social science teachers dread teaching economics and how to overcome the dread, an article by William Walstad on the importance of economics for understanding the world around us and making better personal decisions (with some evidence on the dismal state of economic literacy in this country), and another article by Walstad on the status of economic education in high schools (full citations below). The reflection prompt asks the students to then answer the following questions: What was your high school econ experience like? What do you remember most from that class? How do

When is an exam "too hard"?

By now, you may have heard about the biology professor at Louisiana State (Baton Rouge) who was removed from teaching an intro course where "more than 90 percent of the students... were failing or had dropped the class." The majority of the comments on the Inside Higher Ed story about it are supportive of the professor, particularly given that it seems like the administration did not even talk to her about the situation before acting. I tend to fall in the "there's got to be more to the story so I'll reserve judgment" camp but the story definitely struck a nerve with me, partly because I recently spent 30 minutes "debating" with a student about whether the last midterm was "too hard" and the whole conversation was super-frustrating. To give some background: I give three midterms and a cumulative final, plus have clicker points and Aplia assignments that make up about 20% of the final grade. I do not curve individual exams but will cu