A Big Fish In A Small Causal Pond

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post

In three previous posts, I discussed what I’ve begun to call the “trifecta” of teacher-focused education reform talking points:

• Teachers are the most important (in-school) factor affecting achievement;
• Firing the bottom 5-10 percent of teachers will increase our scores to the level of high-performing nations; and
• Providing children with three, four, or five consecutive “top” teachers in a row would close the achievement gap.

Indeed, they are guiding principles behind much of the Obama Administration’s education agenda, as well as the philosophies of high-profile market-based reformers, such as Joel Klein and Michelle Rhee. The talking points have undeniable appeal. They imply, deliberately or otherwise, that policies focused on improving teacher quality in and of themselves can take us a very long way - not all the way, but perhaps most of the way - towards solving all of our education problems.

This is a fantasy.

All of the research that undergirds the “trifecta” only shows that, in any given year, teachers’ effects on students’ test score gains vary widely. This line of work has served a meaningful policy purpose insofar as it has helped to convince people that teacher quality matters greatly, and efforts to improve it are worthwhile. But no one, on any "side" of the education reform debates, is arguing anything different (if they ever did).

Moreover, at least in my opinion, many of the proposals out there for improving teacher quality are perfectly reasonable - including new teacher evaluation systems (with growth model estimates, if used responsibly [not the case in most places]), alternative compensation systems, and new layoff policies. My only requirements are that teachers be involved in the design of these programs (and subsequently approve them) and that their cost and efficacy be objectively assessed.

But, as I and others have pointed out repeatedly, teachers are only one factor amongst all of the many factors that make for academically successful students. In terms of in-school effects, they are very significant – the most influential measurable variable. But the same body of research that has shown us that teachers matter - the evidence from which the “trifecta” was born - has also shown that non-school factors matter much more (also see here, here, or this non-technical summary).

So, while we can and should debate the choice of specific teacher-focused interventions and their details, I also think we need to start being openly realistic about these ideas. There’s an uncomfortable truth here, one which is too infrequently acknowledged, and is completely obscured by the siren song of the “trifecta." Even if we’re wildly successful in improving teacher quality - and that’s far from certain – this will not, by itself, get us anywhere near where we need to be. In fact, without a more holistic approach to improving student outcomes – one which addresses the circumstances in which parents and students live – it’s unrealistic to believe that teacher-focused policies will make anything more than a minor dent.

And even this will be an uphill battle. For starters, we cannot yet reliably measure teacher “quality." Even if you support the use of growth model estimates in personnel decisions, you cannot deny that the results are highly unstable and subject to considerable error (among other problems). Nor would anyone argue that they are anything but one component of a high-quality teacher performance measure. Efforts to design more useful and reliable evaluation systems are still very much works in progress, and it will be years before we have a solid grasp of how good they are and why.

Even if we’re successful, however, the big question isn’t whether we can measure the “quality” of the teaching force, but rather whether we can affect it. And there is virtually no evidence, at least not yet, that we can use policy to spur major shifts in the “quality distribution” of teachers. As discussed above, there are plenty of related policy ideas out there, and many are spreading rapidly. But, as of yet, few have been proven to do much.

Given the enormity of trying to bring meaningful change to a high-attrition workforce comprised of millions of individual teachers working in thousands of districts, the realistic, best-case outcome would be to see minor, incremental progress over time. In other words, if we play all of our teacher policy cards correctly – better evaluations, etc. - with a little luck and over a period of years, if not decades, we will be able to generate modest overall improvements in teacher “quality." Test scores should also show slow, incremental improvements over this period - gains that we would hope will be shared widely by most students, regardless of race, income, or other characteristics.

But even under this scenario, we would look around and still be nowhere near to achieving equal educational opportunity for all children. There will still be sizeable gaps by income and race, with unacceptably high rates of failure and too many dropouts. Many disadvantaged students with higher scores will still lack the critical non-cognitive abilities – such as social skills and motivation – that are so critical for their success.

For many years, we have known that poor, urban students tend to enter the public education system a grade or more behind their more prosperous peers in the suburbs. It will take more than a few tweaks to teacher compensation and evaluation policies to help them catch up (or, preferably, help prevent the gaps in the first place). The kind of genuine, meaningful change that could truly put the nation’s poor, minority students on an equal footing with the children of the educated and the affluent  (and hopefully lift all boats as well) would require us to tackle the effects of poverty and the awful conditions under which some children are forced to live.

This includes programs focused on jobs, healthcare, and housing. It would require high-quality early childhood intervention systems - not just pre-K, but from birth to age 3. It would mean ensuring the proper content is being taught, providing intensive assistance for struggling students, and attending to kids’ medical, psychological, and social needs as soon as they arise. Saying we can’t do all this because of the political environment, or because it’s too expensive, or too slow, is as much of an “excuse” and a “defense of the status quo” as anything being said in education debates.

No doubt, some may fear that emphasizing the limits of teacher-focused policies will create complacency or, worse, deepen the destructive battle trenches that have been dug over the past decade. I understand these concerns, but I believe that both supporters and skeptics of market-based reform - including teachers - would react very positively to hearing a more balanced, comprehensive debate about which policies – including those outside the “traditional scope” of our education system - have the best chance of helping schools improve, even incrementally. We might reconsider our obsession with cross-sectional, short-term testing gains. There would be less pressure, less blame, and less public delusion about “firing our way” to a just society or finding those “four consecutive” magic teachers who will heroically solve the problems of each and every child in the country. These are talking points, not realistic policy arguments. 

I wish I were wrong. I wish that we could change a few personnel policies, turn around our education system, and redeem the nation. We can’t, and we all know it. If we just agreed that teacher quality is important, and should be improved, but were just as persistent in acknowledging that it’s not enough, this might go a long way towards helping us overcome the awful stalemate in which our education policy debate now finds itself mired.