Skip to main content

TBL: Readiness Assessments

[Today (July 31) is the last day to complete the blog reader survey if you want to be included in the drawing for a free copy of The Intl Handlbook on Teaching and Learning Economics. Thank you!]

My last post outlined the basic structure of TBL. That structure can be summarized as a) students acquire basic concepts on their own, b) students are assessed to make sure they actually have acquired those concepts, c) class time is spent on application exercises that require they use that basic knowledge in more complex ways, and d) students assess their teammates and those evaluations are used to 'weight' the team portion of each individual's final grade. In this post, I'll talk about the first stages of the cycle, where students acquire basic concepts and are assessed on those.

Although there has been a lot of discussion recently about 'flipped classrooms', I'm wary of pushing too much content acquisition outside the classroom in classes like micro principles; I do think it can be done but honestly, it exhausts me just to think about how much work it would be (both to set it up and to get students on board). But I was willing to do it with the data class because all of the content that students need to "acquire" on their own is really stuff that they should already know from the prerequisite classes. I do not make exceptions to the prereq requirements and I make it super-clear to them on Day One that since they have already passed lower-division stats and MIS (info systems), I expect they already have acquired the basic statistical and Excel tools we will be using throughout the semester. I tell them that we will be spending minimal class time reviewing those tools so that class time can be spent on using them.

Assessment at the beginning of each unit
The data course is divided into four modules and at the beginning of each module (i.e., the first day of the unit), students come in and take what I call a Readiness Assessment (RA) [NOTE: In the TBL community, these are more often called 'RATs', short for Readiness Assurance Test, but I really didn't want to call them that, both because I try to avoid calling them 'tests' and because RAT just sounds kind of negative to me]. I provide a study guide with questions covering the necessary content and I tell students that if they can answer all those questions, they will be fine for the RA (I should note that many of those questions are also on a knowledge survey that students take the first week of the semester; that gives me and them some indication of how much review they will need to do). The RA questions are multiple choice and since the point is to make sure they have sufficient understanding of basic concepts, the questions are largely definitions and recall. For example, the third module is when we talk about correlation versus causation. In this class, we talk a lot about why two variables might be correlated without one causing the other, and what data we would need in order to actually identify real causation, but I don't spend any time on the mechanics of computing the correlation or regression coefficients; they should have already gotten that in their previous stats class. But obviously, they need to know what a correlation coefficient is and what a regression tells us, so  the study guide has questions like, "What does it mean for the correlation between two variables to be positive or negative?" and the RA has questions like:
The correlation between X and Y is -0.75. We can conclude:
A. when X increases in value, Y also increases
B. when X decreases in value, Y increases
C. when X decreases in value, Y also decreases
D. X causes Y
E. X does not cause Y
If the students can answer the study guide question about what it means for the correlation to be positive or negative, they should have no problem answering this RA question, but it does usually require that they go back and review their notes from (or re-learn) their intro stats class (there is no stats textbook for the class but I provide links on Blackboard to some online texts and other resources for review since the majority of students have not kept their stats books from previous classes). In a more traditional class, this really shouldn't be all that different from what most professors usually do, in terms of assigning readings from a textbook, but with TBL, students actually have to DO the reading, rather than having the professor repeat it all via lecture in class.

Team assessment ensures content knowledge
IF-AT formA key thing here is that the RA is NOT intended to test higher-level skills; it is to make sure students are ready to move on and start applying the concepts. So even if students don't do the review they should do, and do poorly when they take the RA on their own, the TBL process ensures that they will still be ready to move on because immediately after students finish the RA individually, they re-take the same assessment as a team. To do this most effectively, most TBLers use a special form, called an "IF-AT" form. IF-ATs are multiple-choice scratch-off forms (see picture) - they sort of look like scantrons but instead of bubbles, there's that gray scratch-off material, like on a lottery card. For each question, the teams decide on their answer and scratch off the corresponding letter; if they are correct, they will see a star. The cool part is that if they are wrong, they can try again and still receive partial credit. I give 3 points for one scratch, 2 points for two scratches and 1 point for three scratches; there are five answer options for every question and if they can't get it right in three tries, they get zero points. This process means that not only do students get instant feedback but if a team is wrong on their first attempt, they still have incentive to keep discussing the question so they can try again. The scratch-off form also creates a sort of 'game' atmosphere - it isn't unusual for teams to cheer when they see the star.

The way most teams do the team RA is they go around and everyone will say which answer they chose; if there is any disagreement, students then explain their choice and they try to convince others who might want to scratch off a different letter. Sometimes I'll hear weaker students say things like, "Well, I put A but I really was just guessing and have no idea," but even then, they will usually want someone else to explain why a different answer is right. I have never seen a team just give the IF-AT form to one person and have that person complete it without input from everyone else. So by the time the teams are done, students not only know the answers to all of the RA questions but they have discussed both the right and wrong answers, so they generally understand the concepts well enough to dive into the applications.

Student resistance to assessment process
It's probably not surprising that the individual RAs are generally the students' least-favorite aspects of TBL. In a future post, I will talk about the many things I do to get students to 'buy into' TBL but even when they 'get it', I still get lots of comments along the lines of "I don't like that we get tested before covering the material in class" or "I think Professor Imazeki needs to lecture more before the RAs". On the other hand, students tend to really like the team RAs. I'm still trying to figure out if there are ways to get student to 'feel better' about the RAs but I tend to think that this is just one of those things where I have to trust that I am doing the right thing, even if students don't see it.

In my next post, I'll explain how the team applications work...

Comments

  1. Brand new to your blog. Eagerly awaiting more on TBL. Can you give your opinion about a) the largest size of class for which you think TBL is doable, and b) whether it is doable in a lecture hall with the chairs bolted to the floor in rows.

    ReplyDelete
    Replies
    1. Hi Dan, thanks for reading! These are good questions, and closely related - I think the classroom set-up matters a lot and if the seats are movable, I don't see any reason why TBL wouldn't work for a class of any size. But I've never been seen a classroom for more than about 100 that had movable seats. The first time I used TBL, I had one class in fixed seats and one in movable seats and I felt a bit that the teams in the fixed-seat class weren't all as cohesive as in the other section (but it was also my first shot at TBL so there were definitely other things going on too). If the seats aren't movable, I think it's definitely harder, but not impossible. The 'Intro to TBL' video on the TBL website (http://www.teambasedlearning.org/vid) shows it in use in huge lecture halls (presumably, if you've got 100s, you also have TAs and they can help keep order).

      Delete
  2. Thanks so much for this detailed and thoughtful description. I'm curious -- where do you get those cool scratch-off forms?

    ReplyDelete
    Replies
    1. The IF-AT forms are from Epstein: http://www.epsteineducation.com/home/.

      Delete

Post a Comment

Comments that contribute to the discussion are always welcome! Please note that spammy comments whose only purpose seems to be to direct traffic to a commercial site will be deleted.

Popular posts from this blog

What are the costs?

I came across an interesting discussion about a 19-year-old intern who was fired from The Gazette in Colorado Springs for plagiarism. There appears to be some controversy over the fact that the editor publicly named the girl in a letter to readers (explaining and apologizing for the plagiarism), with some people saying that doing so was unduly harsh because this incident will now follow her for the rest of her career. I was intrigued by this discussion for two reasons - one, it seems pretty clear to me that this was not a case of ignorance (as I have often encountered with my own students who have no idea how to paraphrase or cite correctly) and two, putting aside the offense itself, I have often struggled with how to handle situations where there are long-term repercussions for a student, repercussions that lead the overall costs to be far higher than might seem warranted for the specific situation. As an example of the latter issue, I have occasionally taught seniors who need to p

What was your high school economics experience like?

As I mentioned in my last post , I am asking my Econ for Teachers students to reflect on their reading by responding to discussion prompts. It occurred to me that it wouldn't be a bad idea for me to share my thoughts on those issues here and see if anyone wants to chime in. For this week, the students were asked to read the California and national content standards , an article by Mark Schug and others about why social science teachers dread teaching economics and how to overcome the dread, an article by William Walstad on the importance of economics for understanding the world around us and making better personal decisions (with some evidence on the dismal state of economic literacy in this country), and another article by Walstad on the status of economic education in high schools (full citations below). The reflection prompt asks the students to then answer the following questions: What was your high school econ experience like? What do you remember most from that class? How do

When is an exam "too hard"?

By now, you may have heard about the biology professor at Louisiana State (Baton Rouge) who was removed from teaching an intro course where "more than 90 percent of the students... were failing or had dropped the class." The majority of the comments on the Inside Higher Ed story about it are supportive of the professor, particularly given that it seems like the administration did not even talk to her about the situation before acting. I tend to fall in the "there's got to be more to the story so I'll reserve judgment" camp but the story definitely struck a nerve with me, partly because I recently spent 30 minutes "debating" with a student about whether the last midterm was "too hard" and the whole conversation was super-frustrating. To give some background: I give three midterms and a cumulative final, plus have clicker points and Aplia assignments that make up about 20% of the final grade. I do not curve individual exams but will cu