Welcome new readers!

The "New to the blog? Start here" page will give you an overview of the blog and point you to some posts you might be interested in. You can also subscribe to receive future posts via RSS, Facebook or Twitter using the links on the right-hand side of the page, or via email by entering your address in the box. Thanks for reading!

Wednesday, August 29, 2012

TBL: Summing up

[See previous posts for TBL basics, readiness assessments, team applications, peer evaluations/team grades, and student responses]

For those who have been following along with this series, I hope it's clear by now that TBL is pretty dramatically different from traditional chalk-and-talk. After teaching this way, it's actually really hard for me to stand up and 'lecture' in any class for more than about ten minutes. For those who are curious to find out more, the absolute best place to start is the Team-Based Learning website. The book by Michaelson, et al, is also a good starting place. To wrap up, I thought I'd address some of the questions that I think folks might have if you're considering adopting TBL...

How do you get student buy-in?
As I mentioned in my last post, I think it's crucial for students to understand why we are using TBL. One thing I do on the first day is ask the two questions in this article in The National Teaching and Learning Forum, "First-Day Questions for the Learner-Centered Classroom," by Gary Smith:
Thinking of what you want to get out of your college education and this course, which of the following is most important to you?
A. Acquiring information (facts, principles, concepts)
B. Learning how to use information and knowledge in new situations
C. Developing lifelong learning skills.
Of these three goals, which do you think you can most easily achieve outside of class with your own reading and studying, and which is best achieved in class, working with your classmates and the Professor?

Typically, answers to the first question include a couple of students who answer A and the rest split between B and C. We talk a little about how A, acquiring information, is a necessary step before you can get to B and C, but when I emphasize that the question asks what they want to get out of their college education, most students agree that knowledge alone isn't that useful if you don't know what to do with it. In response to the second question, students immediately see that acquiring information is easiest on your own and from there, explaining why I use team-based learning is pretty straightforward.

Would this work in really large classes with fixed seats?
My own classes have been maxed out at 75 students and I taught one section, my first semester using TBL, in a classroom with fixed seats. Since then, I've requested rooms with movable seats because I think it's a lot easier. I'd certainly suggest starting with smaller classes if possible but with movable seats, I really think TBL could work for a class of pretty much any size. Of course, hopefully if you have hundreds of students, then you also have at least a few TAs who can help with walking around and keeping the groups under control. With fixed seats, you need to be super-clear about where each team is seated; I think it can work OK if each team is together in two rows so the students in the front row can turn around and talk with the students behind them. It isn't ideal but it's do-able. The TBL website has some videos that show TBL in action in some really large classrooms.

How do you create good teams?
Some TBLers create the teams in class but I always create the teams myself (just seems easier to me). On the first day, students fill out an information sheet and I collect some information from them that I then use in creating the teams. The main things I'm concerned about are having a mix of gender, 'ability' and laptop availability on each team (for the data class, each team needs laptop for a few classes so I try to have at least two people per team who say they are 'willing and able to bring a laptop to class'). I measure 'ability' by asking the students if they took the lower-division stats course more than once and if they have ever tutored for economics or statistics. I also check that the non-native English speakers are distributed somewhat equally across teams, and that there were no teams that might have cliques (e.g., members of the same fraternity or sports team). I now use a spreadsheet called the Group Rumbler, created by a guy at Harvard and available for free; that has made my life a lot easier and I'd highly recommend it for anyone who wants to create groups based on specific characteristics. Although I do think my approach has helped ensure that all the teams are roughly 'equal', some TBLers will tell you that it doesn't (or shouldn't) matter all that much. Doing it totally randomly might mean that one team ends up with four or five slackers while another team ends up with four or five 4.0 students, but if you have well-designed applications, the TBL structure should mean that all students have equal incentive to contribute.

I'm not sure I'm ready to adopt TBL whole-hog but would like to adopt certain parts. How can I get my feet wet?
I think there are big advantages to adopting TBL as a whole-course approach but I also think it's difficult to do because you really need to step back and re-design the entire course. I think a good way to build up to that is to start with 4S applications. If your typical approach is to assign problem sets that students do as homework, think about converting those to 4S applications and having students work through them in teams during class. Of course, you may need to reduce lecturing time in order to make time for that, but you may find that you can condense your lecture and have students discover some of the same information on their own as they work through the applications. For upper-division courses, I also think establishing teams and having them do readiness assessments would be a good way to get students to review material from Principles and to do pre-class reading, even if you still spend a lot of time lecturing.

I hope this series has been useful to folks. If you have other questions or comments about TBL, feel free to leave them here or email me directly!

Sunday, August 26, 2012

First day jitters

Classes start tomorrow and I'm feeling oddly anxious. For some reason, I seem to be having a particularly hard time getting mentally into 'school mode' - I've been saying for the last week that it just doesn't feel like it's time to start classes yet. I think my problem is that we are starting a full week before Labor Day; the last few years, I think we started on Wednesday or Thursday so I had to be on campus for advising and such for a few days at the beginning of the week before starting classes and could get mentally prepared. Although I was on campus a few days last week, it just doesn't feel the same...

I'm always a little bit nervous for my first classes of the semester, though the nerves generally go away as soon as I start talking. I remember feeling like I was almost going to throw up before walking into the 500-seater the first time. It helps when I already know students in the class; last spring, about two-thirds of my writing class were students I'd had in the data class already and I think that was the most relaxed I've ever been in a first class meeting. Deep breaths and chocolate help too... I do wonder if this ever goes away. Maybe it's a good thing - as one friend put it, "You probably wouldn't feel nervous if you didn't care."

Whether you get first-day jitters or not, I hope everyone's fall term gets off to a great start!

Tuesday, August 21, 2012

TBL: Student reactions

[See previous posts for TBL basics, readiness assessments, team applications and peer evaluations/team grades]

At the end of every semester, I survey my students specifically about both PollEverywhere and TBL. The questions are largely adapted from a survey that SDSU's Instructional Technology Services asks all clicker-using faculty to give. I've summarized the responses from the last three semesters (click on the image to see full-size). Response was most positive (highest percentages agreeing or strongly agreeing with most of the statements) in Spring 2011, when I had two sections of 75 students each; the positive responses fall a little in the 2011-12 school year (not sure why) but over 80% still said TBL makes them more likely to attend class and to feel more involved in class, about two-thirds would choose a TBL section over another section of the same course that does not use TBL, and (most important to me) over three-quarters still said that they felt they gained a deeper understanding of the material with TBL compared to traditional lectures.

The open-ended comments had similar percentages of positive responses. A lot of students felt that TBL 'made class more fun' and 'was totally different from any class I've taken, in a good way'. Here are two comments that capture attitudes that seem pretty typical for most students:
"Team based learning was very helpful, it let you discuss things with your group and clear things up. I know many people including myself tend to hold back with questions when confused because of 1) not being able to form a good solid question because of the confusion or 2) being embarrassed to ask a question that may make you look stupid. With team based learning that kind of confusion was easily cleared up."
"At first i was very apprehensive about the team group. When i learned that the whole class is developed around teams i said to myself "oh here we go, others are gonna band wagon on few people's hard work" as it always turns out that way with teams. However the way Professor Imazeki set up the teams really worked out well. Everyone had good input. At some point i started to miss a few classes due to personal reasons and my team members motivated me, check up on me and brought me back to class. I enjoyed working on my own at home and comparing my findings with my teammates in order to reach collaborative answers in class. I have never had such great experience with team work.  I would love to have other classes designed around this kind of team work versus team project where the pressure usually falls on one or two people who care."
Not all students love TBL
In every class, there have been a few comments along the lines of "It would have helped if the professor had explained things a little more", and a handful of students have been downright hostile. In at least two of those cases, I think the students were more frustrated by the material, rather than the method (that is, they hated that there wasn't always a 'right answer' to everything). A few students commented on what they saw as free-riding behavior, noting that not all group members always participate equally but they get credit for the team RAs and applications. To me, those comments indicate the students don't entirely understand how the peer evaluations impact the team part of the grade - if a student really isn't contributing, then the rest of the team should give them a lower evaluation score and they don't get the same credit for team efforts. As I write this, it dawns on me that maybe this fall, I should use those comments as part of my explanation of how the evaluations will work...

I tend to fixate on the few students who give negative feedback but overall, once students understand why I am using TBL, and once they see that this is not like other group work they have experienced in the past, the vast majority enjoy it, if not prefer it to typical lectures. I do think it's important to lay the groundwork on the first day, to explain to students exactly why we will be using TBL and why I believe it is a better learning experience for them than listening to me lecture. In my next post, I'll wrap up this series with some discussion of how to get that buy-in from students and also some thoughts on basic team logistics.

Friday, August 17, 2012

Friday Finds

I'll take a short break from my TBL series to share a couple of resources that you may find helpful if, like me, you're tweaking your fall classes and looking for some new material...
  • Economics memes: I think anyone who is on Facebook, or receives email for that matter, has seen some version of various internet memes, which wikipedia defines as any idea that is propagated through the internet: "The idea may take the form of a hyperlink, video, picture, website, hashtag, or just a word or phrase... The meme may spread from person to person via social networks, blogs, direct email, news sources or other web-based services." The Economics memes site provides a collection of econ-related memes, as well as some information about how to use memes in your class. [Note: the Most Interesting Man in the World picture is actually from the Cengage Facebook page, and Jodi Beggs has a bunch of other variations on the Most Interesting Man meme].
  • Div.E.Q.: Diversifying Economic Quality: A Wiki for Instructors and Departments provides some great teaching resources. As stated on the site:
  • "The purpose of this wiki is to promote best teaching practices in economics, particularly practices that encourage women, students of color, and members of other underrepresented groups to continue their study of economics. Here, economics faculty can both disseminate and discover prescriptions for improving our teaching and the inclusiveness of our discipline."
  • The Peer Instruction Network has lots of information if you want to add some peer instruction elements to your courses. The blog has some good posts if you're new to peer instruction, including this post that answers the basic question, "What is peer instruction?". 

Tuesday, August 14, 2012

TBL: Peer evaluations and team grades

[See previous posts for TBL basics, readiness assessments and team applications]

In this post, I'll go over how my students' course grades are calculated and the role of peer evaluations. I think the peer evaluations are probably the part of TBL that students are most worried about at the beginning of the semester, and the aspect that I get asked about the most by faculty. I want to say right off the bat that in the four semesters (eight classes) I've used TBL, I have not seen a single evaluation that appeared to be a student or team trying to 'game the system' (i.e., 'rewarding' a friend or 'punishing' someone unfairly). I've seen a few where students seemed to be not putting in much effort or thought but it has never caused someone's grade to be different than I thought made sense.

Grade weights
Many TBLers have the students themselves determine how much weight will be given to team and individual activities. The way Larry Michaelson and others do it is to have the teams discuss and then send a representative to meet with other team representatives in a fishbowl-type discussion. While I can see how that could be great for getting student buy-in and for building team cohesion, I didn't feel comfortable doing it with my classes so I simply set the weights myself. For the data class, 25% of the final grade is based on team activities (that's 18% from the team RAs and 7% from the team applications). The other 75% of the grade comes from the individual RAs (10%), individual participation based on PollEverywhere responses and pre-class homeworks (10%), two in-class exams (20%), and two writing projects (15% and 20%). Thus, 55% of a student's grade is based on summative assessments (the exams and writing projects), which I think is enough to differentiate the students but not so much that they can ignore/slack off on all the other stuff. I am constantly debating with myself whether I should give more or less weight to things like participation but this mix seems to be working.

Peer evaluations
There are different ways to do the peer evaluations but what I do is have students give a numeric score to each member of their team (not including themselves) and those scores must add up to 100. They must also provide a qualitative explanation of those scores, and those comments are passed on (anonymously) to each student. On the evaluation form, which students complete on Blackboard, the instructions say (borrowed from materials on the TBL website):
"Evaluate the contributions of each person in your group except yourself, by distributing 100 points among them (that is, when you are done, the total points assigned to everyone should sum up to 100). You must provide comments for each person. These comments -- but not who provided them -- will be passed onto your teammates. Your score should reflect your judgment of such things as Preparation (did they come to class prepared?), Contribution (did they contribute productively to group discussion and work?), Respect for others (did they encourage everyone to contribute and listen respectfully to different opinions?), and Flexibility (were they flexible when disagreements occurred?). It is important that you differentiate between people who truly worked hard for the good of the group and those you perceived not to be working as hard on group tasks (NOTE: If you give everyone pretty much the same score when it is not truly deserved, you will be hurting those who did the most and helping those who did the least)."
These evaluations are done twice a semester: the mid-semester evaluations provide students with feedback so they can adjust behavior if necessary, and then the end-of-semester evaluations are the ones that actually 'count'. In order to make sure that students give both numeric and qualitative feedback, I give them individual points for completion.

Incorporating evaluations into grades
Most faculty using TBL tend to use the evaluation scores either as a multiplier applied to the team part of the grade, or as a separate component of the course grade (see the TBL website for a discussion of both). I use the former so I take the peer evaluation scores and convert those to a weight. Since I mostly have teams of 6, the average score for each student is 20 (e.g., if someone wanted to give everyone on the team the same score, that score would be 20) so I start by taking an individual's average score as a percentage of 20. Thus, really good team members will have weights of greater than 1. I played with different ways of calculating the weight (e.g., an individual's average score as a percentage of the lowest score on the team) but finally decided just the score over 20 and I cap the weights at 1.25. So let's say a team has perfect scores on their team RAs and team applications (a situation that has not actually happened in any of my classes), and one member of the team is the clear leader and has high scores from everyone, then that person could actually get 125% for the team portion of their grade. If that person were also to have perfect scores on all their individual assignments, they could actually have more than 100% of the points possible for the semester (again, this has never actually happened).

Do students take it seriously and think it's a fair process?
As mentioned above, I have not yet seen any evidence that students try to game the system. I also have not had any complaints from students about their evaluations being unfair. I think the qualitative feedback helps with this tremendously. Even if one student were to unfairly criticize a teammate, it would be clear from the other team members' comments that the student was out of line. At the same time, when a particular student is not pulling their weight, that generally shows up in comments from multiple teammates, not just one. One thing I find fascinating is how much students will ding a teammate for being absent; comments like "he has missed a lot of classes" or "she doesn't tell us when she's going to miss class" almost always accompany low scores. And those comments matter: I have seen quite a few students start coming to class more regularly after they get their mid-semester evaluations. I've actually considered not having the PollEverywhere responses count for points (and giving more weight to the team portion of the grade instead), since those PE points are often seen by the students as just points for attendance (which, really, they kind of are).

I think the instructions, asking students to think about things like respect for others and flexibility, are also important for getting good qualitative feedback. I have been pleasantly surprised how often I will see comments like, "John always makes sure to get everyone's opinion before we finalize our answer" or "Jane always has a strong opinion, and it's usually right, but she's also good about listening to other people's explanations and admitting when she's wrong" or "I wish Jim would contribute more; when he does, he usually has good points but he's kind of quiet." In general, the majority of comments, both good and bad, are respectful and actually constructive.

In my next post, I'll share some of the feedback I've collected from students about their experience with TBL...

Tuesday, August 7, 2012

TBL: Team applications

[For an overview of TBL, see this post]

In my last post, I described the Readiness Assessment process; by the time students are done with the team RA, they are ready for the fun stuff: the team applications. For the next several class meetings, class time is almost entirely devoted to teams working on problems that require they think through and discuss the kind of data issues that empirical researchers routinely address. During the all-class discussion that follows the team discussions, my contribution tends to be limited to directing the discussion. I may spend a few minutes at the end of class tying together concepts but that's as close as I get to lecturing. TBL 'guidelines' say that good applications should satisfy the 4 S's: Significant problem, Same problem, Specific choice, Simultaneous report.

Significant problem
This seems sort of obvious - of course you want students to be working on problems that are 'significant', which I take to mean both relevant and complex enough to require application and integration of course concepts. But actually coming up with 'significant' problems isn't always easy. It would be great if I could use some canned examples from textbook supplements but an awful lot of those are neither relevant nor complex, particularly because I want students to think and not plug-and-chug. For most economics course, I think current events are probably the best source for applications. I use a number of applications based on headlines and real-world examples, many involving issues that are not actually resolved among experts (more about this in a minute). For example, for discussion of issues with defining vague concepts, students are asked to decide what variable to use in a comparison of whether Americans are 'better off' today than they were four years ago.

Same problem
I've had groups in other classes do 'jig-saw' problems, where each group (or member of the group) works on a slightly different problem and then the parts have to be put together to form an overall conclusion/product. While I think this can work for some purposes, it is better for discussion to have everyone working on the same problem. Both within and between groups, discussion is more lively as students compare answers and then must defend and explain their approach to the problem.

Specific choice
From my perspective, this is the key factor for ensuring productive team discussions. Teams must agree on ONE choice and, as a team, defend that choice to other teams. I think teachers usually tend to prefer open-ended questions for getting students to think critically. Yet for most of my applications, the way I force students to make a 'specific choice' is by giving them multiple choice options. However, a large number of the applications have more than one answer choice that could be 'right', depending on what assumptions you make. For example, one of my favorite applications is about the CPS definition of 'income'. Before class, students go to the CPS website and get data on median household income and have to read about how the CPS defines income. The team application reiterates that income does not include noncash benefits, including in-kind transfers like food stamps and employer-provided work benefits, and asks:
The exclusions in the CPS definition mean that the measured income gap between rich and poor (assume ‘rich’ and ‘poor’ measured by 10th and 90th percentile of the population), when compared to the "true" income gap, is likely to be:
A. understated
B. overstated
C. unaffected
Almost all students quickly recognize that the exclusion of noncash benefits means that the measured income of poor people is likely less than their 'true' income. Some students see that richer people's incomes are also understated because work benefits are not included either; someone always also points out that income is measured pre-tax and rich people pay more in taxes. The point is that through the team discussion, students have to identify not only how different people's incomes are affected but they have to think about how big those effects are likely to be. And since none of them have any real idea what those magnitudes are, they have to make assumptions, and those assumptions will ultimately drive which answer they choose.

On the worksheets that the teams complete, I ask them specifically to identify any assumptions that they are making but this is always the most difficult part for them - it really takes some work to get them to admit that they are 'assuming' information. But it helps that the answers are always split, both within teams and then among teams, which means there is someone else in the room who is making very different assumptions and coming to a different conclusion.

The biggest challenge for me with these applications is that certain students really hate that there is no one 'right' answer. I've learned that I really have to emphasize to students the 'if-then' nature of the answers; that is, "IF you assume X, THEN you would want to choose answer A; but IF you assume Y, THEN you would want to choose answer B." [I should add that I also point out to them that, usually, both X and Y are assumptions people might have for different reasons, like political beliefs, which is why I'm constantly telling them all to be aware of their own biases...]. I also repeatedly tell students that although some questions may have more than one 'right' answer, there is still a 'right' way to think about the questions, i.e., to identify what assumptions will lead to different answers. I try to point out to them that this is the way the 'real world' works - for example, when they hear politicians saying things that seem to be contradictory, it doesn't necessarily mean that one side is "wrong" but that they are making different assumptions. Of course, then you should next ask whether those assumptions are valid!

Simultaneous report
After the teams have some time to discuss the application and make their choice, every team reports their choice at the same time. This means teams can't change their answer once they see what everyone else chose. The way I do the simultaneous report is to use whiteboards - each team writes the letter corresponding to their choice on their boards and holds it up at the same time. I used clickers the first time I taught the class, which made it easy to see how many teams had chosen each answer but I couldn't see which team had chosen what. I think that's more important because teams need to be accountable for their answer - when I ask Team 6 to explain their choice, they can't just say they chose whatever the most popular answer was. Similarly, I can see immediately if one or two teams choose something different than all the others; that's usually where I start the all-class discussion.

Some logistical issues
    Individual responses: I do have students register their choices individually before the team discussion (using PollEverywhere). I may or may not show them the distribution of those answers. The important thing is that they have to give the question at least a little bit of independent thought, and they have to register their own choice, before discussing the problem with their team. This not only gives the teams an easy starting point (they go around and ask everyone what answer they chose) but I think that once people have registered an answer, even if they don't have a ton of confidence in it, they will either try to defend it or want a coherent explanation of why the team should go with a different choice.
    Team worksheets: For each application, there is a team worksheet where the teams must write down which answer choice they selected and provide a justification for that choice, including identifying any assumptions they are making. I do grade these worksheets, though on mostly a "plus, check, minus" kind of scale. One of the issues I'm still trying to figure out is how to make those worksheets more productive for everyone. What typically happens is that one person (either the 'smartest' person or the one with the best writing) ends up always writing up the explanation and everyone else just sort of sits around while he or she does that. I'm trying to decide if I want to 'force' teams to rotate this responsibility, since I think whoever does the writing probably learns the most.
    Timing: Another thing I am constantly re-evaluating is how much time to give teams for the applications. This has largely been trial and error: the first few times I taught the class, I gave the teams a set amount of time (usually 5 to 7 minutes) and then would monitor the teams and give them additional time as necessary. But I got a lot of complaints from students that they needed more time. This past spring, I decided to make the time open-ended and gave each team a green piece of paper, folded in half. When they were done, they were supposed to put the green 'tent' on someone's desk so I could see they were done. When about half the teams were done, I'd tell everyone else they have 1 minute to finish up. That seemed to work well and although the teams took a lot longer, we never ran out of time either (I'm not entirely sure how that happened but I think it may be that because they had more time to discuss without feeling rushed, I had to do less explaining afterwards).
   All-class discussion: When there are one or two teams that select a different answer than everyone else, I will ask someone from that team to explain their answer, and then ask someone from a different team to explain why they made a different choice. I try to learn all the students' names so I can call on individuals - if I don't, there is a tendency for the same people to always answer for a given team. I'm still playing around with this; sometimes I will call on teams randomly by using a deck of cards, or a random student-picker app. If there is a lot of agreement about one particular answer, I will sometimes extend discussion by asking students to identify why someone might choose one of the other options, or I ask someone who chose a different option with PollEverywhere originally to explain their thinking.

In my next post, I'll discuss the final step of the TBL process, team evaluations.