TBL: Readiness Assessments

[Today (July 31) is the last day to complete the blog reader survey if you want to be included in the drawing for a free copy of The Intl Handlbook on Teaching and Learning Economics. Thank you!]

My last post outlined the basic structure of TBL. That structure can be summarized as a) students acquire basic concepts on their own, b) students are assessed to make sure they actually have acquired those concepts, c) class time is spent on application exercises that require they use that basic knowledge in more complex ways, and d) students assess their teammates and those evaluations are used to 'weight' the team portion of each individual's final grade. In this post, I'll talk about the first stages of the cycle, where students acquire basic concepts and are assessed on those.

Although there has been a lot of discussion recently about 'flipped classrooms', I'm wary of pushing too much content acquisition outside the classroom in classes like micro principles; I do think it can be done but honestly, it exhausts me just to think about how much work it would be (both to set it up and to get students on board). But I was willing to do it with the data class because all of the content that students need to "acquire" on their own is really stuff that they should already know from the prerequisite classes. I do not make exceptions to the prereq requirements and I make it super-clear to them on Day One that since they have already passed lower-division stats and MIS (info systems), I expect they already have acquired the basic statistical and Excel tools we will be using throughout the semester. I tell them that we will be spending minimal class time reviewing those tools so that class time can be spent on using them.

Assessment at the beginning of each unit
The data course is divided into four modules and at the beginning of each module (i.e., the first day of the unit), students come in and take what I call a Readiness Assessment (RA) [NOTE: In the TBL community, these are more often called 'RATs', short for Readiness Assurance Test, but I really didn't want to call them that, both because I try to avoid calling them 'tests' and because RAT just sounds kind of negative to me]. I provide a study guide with questions covering the necessary content and I tell students that if they can answer all those questions, they will be fine for the RA (I should note that many of those questions are also on a knowledge survey that students take the first week of the semester; that gives me and them some indication of how much review they will need to do). The RA questions are multiple choice and since the point is to make sure they have sufficient understanding of basic concepts, the questions are largely definitions and recall. For example, the third module is when we talk about correlation versus causation. In this class, we talk a lot about why two variables might be correlated without one causing the other, and what data we would need in order to actually identify real causation, but I don't spend any time on the mechanics of computing the correlation or regression coefficients; they should have already gotten that in their previous stats class. But obviously, they need to know what a correlation coefficient is and what a regression tells us, so  the study guide has questions like, "What does it mean for the correlation between two variables to be positive or negative?" and the RA has questions like:

The correlation between X and Y is -0.75. We can conclude:
A. when X increases in value, Y also increases
B. when X decreases in value, Y increases
C. when X decreases in value, Y also decreases
D. X causes Y
E. X does not cause Y

If the students can answer the study guide question about what it means for the correlation to be positive or negative, they should have no problem answering this RA question, but it does usually require that they go back and review their notes from (or re-learn) their intro stats class (there is no stats textbook for the class but I provide links on Blackboard to some online texts and other resources for review since the majority of students have not kept their stats books from previous classes). In a more traditional class, this really shouldn't be all that different from what most professors usually do, in terms of assigning readings from a textbook, but with TBL, students actually have to DO the reading, rather than having the professor repeat it all via lecture in class.

Team assessment ensures content knowledge
A key thing here is that the RA is NOT intended to test higher-level skills; it is to make sure students are ready to move on and start applying the concepts. So even if students don't do the review they should do, and do poorly when they take the RA on their own, the TBL process ensures that they will still be ready to move on because immediately after students finish the RA individually, they re-take the same assessment as a team. To do this most effectively, most TBLers use a special form, called an "IF-AT" form. IF-ATs are multiple-choice scratch-off forms (see picture) - they sort of look like scantrons but instead of bubbles, there's that gray scratch-off material, like on a lottery card. For each question, the teams decide on their answer and scratch off the corresponding letter; if they are correct, they will see a star. The cool part is that if they are wrong, they can try again and still receive partial credit. I give 3 points for one scratch, 2 points for two scratches and 1 point for three scratches; there are five answer options for every question and if they can't get it right in three tries, they get zero points. This process means that not only do students get instant feedback but if a team is wrong on their first attempt, they still have incentive to keep discussing the question so they can try again. The scratch-off form also creates a sort of 'game' atmosphere - it isn't unusual for teams to cheer when they see the star.

The way most teams do the team RA is they go around and everyone will say which answer they chose; if there is any disagreement, students then explain their choice and they try to convince others who might want to scratch off a different letter. Sometimes I'll hear weaker students say things like, "Well, I put A but I really was just guessing and have no idea," but even then, they will usually want someone else to explain why a different answer is right. I have never seen a team just give the IF-AT form to one person and have that person complete it without input from everyone else. So by the time the teams are done, students not only know the answers to all of the RA questions but they have discussed both the right and wrong answers, so they generally understand the concepts well enough to dive into the applications.

Student resistance to assessment process
It's probably not surprising that the individual RAs are generally the students' least-favorite aspects of TBL. In a future post, I will talk about the many things I do to get students to 'buy into' TBL but even when they 'get it', I still get lots of comments along the lines of "I don't like that we get tested before covering the material in class" or "I think Professor Imazeki needs to lecture more before the RAs". On the other hand, students tend to really like the team RAs. I'm still trying to figure out if there are ways to get student to 'feel better' about the RAs but I tend to think that this is just one of those things where I have to trust that I am doing the right thing, even if students don't see it.

In my next post, I'll explain how the team applications work...