[See previous posts for TBL basics, readiness assessments and team applications]
In this post, I'll go over how my students' course grades are calculated and the role of peer evaluations. I think the peer evaluations are probably the part of TBL that students are most worried about at the beginning of the semester, and the aspect that I get asked about the most by faculty. I want to say right off the bat that in the four semesters (eight classes) I've used TBL, I have not seen a single evaluation that appeared to be a student or team trying to 'game the system' (i.e., 'rewarding' a friend or 'punishing' someone unfairly). I've seen a few where students seemed to be not putting in much effort or thought but it has never caused someone's grade to be different than I thought made sense.
Grade weights
Many TBLers have the students themselves determine how much weight will be given to team and individual activities. The way Larry Michaelson and others do it is to have the teams discuss and then send a representative to meet with other team representatives in a fishbowl-type discussion. While I can see how that could be great for getting student buy-in and for building team cohesion, I didn't feel comfortable doing it with my classes so I simply set the weights myself. For the data class, 25% of the final grade is based on team activities (that's 18% from the team RAs and 7% from the team applications). The other 75% of the grade comes from the individual RAs (10%), individual participation based on PollEverywhere responses and pre-class homeworks (10%), two in-class exams (20%), and two writing projects (15% and 20%). Thus, 55% of a student's grade is based on summative assessments (the exams and writing projects), which I think is enough to differentiate the students but not so much that they can ignore/slack off on all the other stuff. I am constantly debating with myself whether I should give more or less weight to things like participation but this mix seems to be working.
Peer evaluations
There are different ways to do the peer evaluations but what I do is have students give a numeric score to each member of their team (not including themselves) and those scores must add up to 100. They must also provide a qualitative explanation of those scores, and those comments are passed on (anonymously) to each student. On the evaluation form, which students complete on Blackboard, the instructions say (borrowed from materials on the TBL website):
Incorporating evaluations into grades
Most faculty using TBL tend to use the evaluation scores either as a multiplier applied to the team part of the grade, or as a separate component of the course grade (see the TBL website for a discussion of both). I use the former so I take the peer evaluation scores and convert those to a weight. Since I mostly have teams of 6, the average score for each student is 20 (e.g., if someone wanted to give everyone on the team the same score, that score would be 20) so I start by taking an individual's average score as a percentage of 20. Thus, really good team members will have weights of greater than 1. I played with different ways of calculating the weight (e.g., an individual's average score as a percentage of the lowest score on the team) but finally decided just the score over 20 and I cap the weights at 1.25. So let's say a team has perfect scores on their team RAs and team applications (a situation that has not actually happened in any of my classes), and one member of the team is the clear leader and has high scores from everyone, then that person could actually get 125% for the team portion of their grade. If that person were also to have perfect scores on all their individual assignments, they could actually have more than 100% of the points possible for the semester (again, this has never actually happened).
Do students take it seriously and think it's a fair process?
As mentioned above, I have not yet seen any evidence that students try to game the system. I also have not had any complaints from students about their evaluations being unfair. I think the qualitative feedback helps with this tremendously. Even if one student were to unfairly criticize a teammate, it would be clear from the other team members' comments that the student was out of line. At the same time, when a particular student is not pulling their weight, that generally shows up in comments from multiple teammates, not just one. One thing I find fascinating is how much students will ding a teammate for being absent; comments like "he has missed a lot of classes" or "she doesn't tell us when she's going to miss class" almost always accompany low scores. And those comments matter: I have seen quite a few students start coming to class more regularly after they get their mid-semester evaluations. I've actually considered not having the PollEverywhere responses count for points (and giving more weight to the team portion of the grade instead), since those PE points are often seen by the students as just points for attendance (which, really, they kind of are).
I think the instructions, asking students to think about things like respect for others and flexibility, are also important for getting good qualitative feedback. I have been pleasantly surprised how often I will see comments like, "John always makes sure to get everyone's opinion before we finalize our answer" or "Jane always has a strong opinion, and it's usually right, but she's also good about listening to other people's explanations and admitting when she's wrong" or "I wish Jim would contribute more; when he does, he usually has good points but he's kind of quiet." In general, the majority of comments, both good and bad, are respectful and actually constructive.
In my next post, I'll share some of the feedback I've collected from students about their experience with TBL...
In this post, I'll go over how my students' course grades are calculated and the role of peer evaluations. I think the peer evaluations are probably the part of TBL that students are most worried about at the beginning of the semester, and the aspect that I get asked about the most by faculty. I want to say right off the bat that in the four semesters (eight classes) I've used TBL, I have not seen a single evaluation that appeared to be a student or team trying to 'game the system' (i.e., 'rewarding' a friend or 'punishing' someone unfairly). I've seen a few where students seemed to be not putting in much effort or thought but it has never caused someone's grade to be different than I thought made sense.
Grade weights
Many TBLers have the students themselves determine how much weight will be given to team and individual activities. The way Larry Michaelson and others do it is to have the teams discuss and then send a representative to meet with other team representatives in a fishbowl-type discussion. While I can see how that could be great for getting student buy-in and for building team cohesion, I didn't feel comfortable doing it with my classes so I simply set the weights myself. For the data class, 25% of the final grade is based on team activities (that's 18% from the team RAs and 7% from the team applications). The other 75% of the grade comes from the individual RAs (10%), individual participation based on PollEverywhere responses and pre-class homeworks (10%), two in-class exams (20%), and two writing projects (15% and 20%). Thus, 55% of a student's grade is based on summative assessments (the exams and writing projects), which I think is enough to differentiate the students but not so much that they can ignore/slack off on all the other stuff. I am constantly debating with myself whether I should give more or less weight to things like participation but this mix seems to be working.
Peer evaluations
There are different ways to do the peer evaluations but what I do is have students give a numeric score to each member of their team (not including themselves) and those scores must add up to 100. They must also provide a qualitative explanation of those scores, and those comments are passed on (anonymously) to each student. On the evaluation form, which students complete on Blackboard, the instructions say (borrowed from materials on the TBL website):
"Evaluate the contributions of each person in your group except yourself, by distributing 100 points among them (that is, when you are done, the total points assigned to everyone should sum up to 100). You must provide comments for each person. These comments -- but not who provided them -- will be passed onto your teammates. Your score should reflect your judgment of such things as Preparation (did they come to class prepared?), Contribution (did they contribute productively to group discussion and work?), Respect for others (did they encourage everyone to contribute and listen respectfully to different opinions?), and Flexibility (were they flexible when disagreements occurred?). It is important that you differentiate between people who truly worked hard for the good of the group and those you perceived not to be working as hard on group tasks (NOTE: If you give everyone pretty much the same score when it is not truly deserved, you will be hurting those who did the most and helping those who did the least)."These evaluations are done twice a semester: the mid-semester evaluations provide students with feedback so they can adjust behavior if necessary, and then the end-of-semester evaluations are the ones that actually 'count'. In order to make sure that students give both numeric and qualitative feedback, I give them individual points for completion.
Incorporating evaluations into grades
Most faculty using TBL tend to use the evaluation scores either as a multiplier applied to the team part of the grade, or as a separate component of the course grade (see the TBL website for a discussion of both). I use the former so I take the peer evaluation scores and convert those to a weight. Since I mostly have teams of 6, the average score for each student is 20 (e.g., if someone wanted to give everyone on the team the same score, that score would be 20) so I start by taking an individual's average score as a percentage of 20. Thus, really good team members will have weights of greater than 1. I played with different ways of calculating the weight (e.g., an individual's average score as a percentage of the lowest score on the team) but finally decided just the score over 20 and I cap the weights at 1.25. So let's say a team has perfect scores on their team RAs and team applications (a situation that has not actually happened in any of my classes), and one member of the team is the clear leader and has high scores from everyone, then that person could actually get 125% for the team portion of their grade. If that person were also to have perfect scores on all their individual assignments, they could actually have more than 100% of the points possible for the semester (again, this has never actually happened).
Do students take it seriously and think it's a fair process?
As mentioned above, I have not yet seen any evidence that students try to game the system. I also have not had any complaints from students about their evaluations being unfair. I think the qualitative feedback helps with this tremendously. Even if one student were to unfairly criticize a teammate, it would be clear from the other team members' comments that the student was out of line. At the same time, when a particular student is not pulling their weight, that generally shows up in comments from multiple teammates, not just one. One thing I find fascinating is how much students will ding a teammate for being absent; comments like "he has missed a lot of classes" or "she doesn't tell us when she's going to miss class" almost always accompany low scores. And those comments matter: I have seen quite a few students start coming to class more regularly after they get their mid-semester evaluations. I've actually considered not having the PollEverywhere responses count for points (and giving more weight to the team portion of the grade instead), since those PE points are often seen by the students as just points for attendance (which, really, they kind of are).
I think the instructions, asking students to think about things like respect for others and flexibility, are also important for getting good qualitative feedback. I have been pleasantly surprised how often I will see comments like, "John always makes sure to get everyone's opinion before we finalize our answer" or "Jane always has a strong opinion, and it's usually right, but she's also good about listening to other people's explanations and admitting when she's wrong" or "I wish Jim would contribute more; when he does, he usually has good points but he's kind of quiet." In general, the majority of comments, both good and bad, are respectful and actually constructive.
In my next post, I'll share some of the feedback I've collected from students about their experience with TBL...
Comments
Post a Comment
Comments that contribute to the discussion are always welcome! Please note that spammy comments whose only purpose seems to be to direct traffic to a commercial site will be deleted.