Teacher Quality On The Red Carpet; Accuracy Swept Under The Rug

The media campaign surrounding “Waiting for Superman," which has already drawn considerable coverage, only promises to get bigger. While I would argue – at least in theory – that increased coverage of education is a good thing, it also means that this is a critically important (and, in some respects, dangerous) time. Millions of new people will be tuning in, believing that they are hearing serious discussions about the state of public education in America and “what the research says” about how it can be improved.

It’s therefore a sure bet that what I’ve called the “teacher effects talking point” will be making regular appearances. It goes something like this: Teachers are the most important schooling factor influencing student achievement. This argument provides much of the empirical backbone for the current push toward changes in teacher personnel policies. It is an important finding based on high-quality research, one with clear policy implications. It is also potentially very misleading.

The same body of evidence that shows that teachers are the most important within-school factor influencing test score gains also demonstrates that non-school factors matter a great deal more. The first wave of high-profile articles in our newly-energized education debate not only seem to be failing to provide this context, but are ignoring it completely. Deliberately or not, they are publishing incorrect information dressed up as empirical fact, spreading it throughout a mass audience new to the topic, to the detriment of us all.

In a previous post, I reviewed the literature on teacher effects, which shows that teachers explain roughly 10-15 percent of the variation in student achievement outcomes (out of the 20 or so percent explained by schools overall). About three times as much variation (60 percent) is explained by factors (mostly unobserved) outside of schools, such as poverty and family background. The rest of the variation (roughly 20 percent) is unexplained error (see the post for all my citations).

In other words, the majority of what accounts for student performance is located “outside the control” of schools. Even though the 10-15 percent explained by teachers still represents a great deal of power (and is among the only factors “within the jurisdiction” of education policy), it is nevertheless important to bear in mind that poor educational outcomes are a result of a complicated web of social and economic forces.  People have to understand that, or they will maintain unrealistic expectations about the extent to which teacher-related policies alone can solve our problems, and how quickly they will work.

So, as I argued in the post, those who comment on education, especially to non-technical audiences, need to put the teacher effect argument in this larger context. Unfortunately, at least in the earliest stages of this deluge of mass media education coverage, it seems that reporters are not only failing to place the teacher effects talking point into its larger “schooling factor” context, they aren’t mentioning it at all. 

For instance, Jonathan Chait, in a recently-published New Republic article (9/2/10), notes:

Copious research has shown that teacher effectiveness has the single most dramatic impact on student performance.

Years of research has shown that Guggenheim is right, that no variable is more critical to the success of students than terrific teachers.

This is a crucial time for the public discourse on education policy – there are four new education documentaries, an education campaign by NBC, and even Oprah has joined the fray. Hundreds of reporters, many of whom are relatively uninformed about the topic, will be shaping public views on the state of education, and these opinions will have dramatic implications for years to come. 

As education policy moves from the relative obscurity of blogs and newspapers’ back sections to the spotlight of national television and glossy magazines, those of us with public platforms (modest though some of them may be) must do what we can to correct these types of statements. The research on teacher effects will play a huge role in driving this debate, and we should not let the important findings of this literature be reduced to misleading – or simply incorrect – talking points. 

Allowing that to happen only threatens the degree to which this research will guide meaningful, productive teacher-related policy changes. Reputable researchers must also help – making sure that those who use and cite their work actually understand it (the researchers who came out publicly during the recent LA Times affair to urge caution, including William Sanders and Dan Goldhaber, provide a good example). 

We all saw what misinformation can do to the public discourse (and the policy that arises from it) during the health care debate. There are dozens of previous examples as well. Let’s make this education debate the exception.