PISA And TIMSS: A Distinction Without A Difference?

Our guest author today is William Schmidt, a University Distinguished Professor and co-director of the Education Policy Center at Michigan State University. He is also a member of the Shanker Institute board of directors.

Every year or two, the mass media is full of stories on the latest iterations of one of the two major international large scale assessments, the Trends in International Mathematics and Science Study (TIMSS) and the Program for International Student Assessment (PISA). What perplexes many is that the results of these two tests -- both well-established and run by respectable, experienced organizations -- suggest different conclusions about the state of U.S. mathematics education. Generally speaking, U.S. students do better on the TIMSS and poorly on the PISA, relative to their peers in other nations. Depending on their personal preferences, policy advocates can simply choose whichever test result is convenient to press their argument, leaving the general public without clear guidance.

Now, in one sense, the differences between the tests are more apparent than real. One reason why the U.S. ranks better on the TIMSS than the PISA is that the two tests sample students from different sets of countries. The PISA has many more wealthy countries, whose students tend to do better – hence, the U.S.’s lower ranking. It turns out that when looking at only the countries that participated in both the TIMSS and the PISA we find similar country rankings. There are also some differences in statistical sampling, but these are fairly minor.

The Global Relationship Between Classroom Content And Unequal Educational Outcomes

Our guest author today is William Schmidt, a University Distinguished Professor and co-director of the Education Policy Center at Michigan State University. He is also a member of the Shanker Institute board of directors.

It is no secret that disadvantaged students are more likely to struggle in school. For decades now, public policy has focused on how to reduce the achievement gap between poorer and more affluent students. Despite numerous reform efforts, these gaps remain virtually unchanged – a fact that is deeply frustrating, and also a little confusing. It would be reasonable to assume that background inequalities would shrink over the years of schooling, but that’s not what we find. At age eighteen, rather, we find differences that are roughly the same size as we see at age six.

Does this mean that schools can’t effectively address inequality? Certainly not. I devoted a whole book to the subject, Inequality for All, in which I argued that one of the key factors driving inequality in schools is unequal opportunity to learn, or OTL.

It is very unlikely that students will learn material they are not exposed to, and there is considerable evidence that disadvantaged students are systematically tracked into classrooms with weaker content. Rather than mitigating the effects of poverty, many American schools are exacerbating them.

PISA For Our Time: A Balanced Look

Press coverage of the latest PISA results over the past two months has almost been enough to make one want to crawl under the bed and hide. Over and over, we’ve been told that this is a “Sputnik moment," that the U.S. among the lowest performing nations in the world, and that we’re getting worse.

Thankfully, these claims are largely misleading. Insofar as we’re sure to hear them repeated often over the next few years—at least until the next set of international results come in — it makes sense to try to correct the record (also see here and here).

But, first, I want to make it very clear that U.S. PISA results are not good enough by any stretch of the imagination, and we can and should do a whole lot better. Nevertheless, international comparisons of any kind are very difficult, and if we don’t pay careful attention to what the data are really telling us, it will be more difficult to figure out how to respond appropriately.

This brings me to three basic points about the 2009 PISA results that we need to bear in mind.

Subscribe to Pisa