Teacher Quality Is Not A Policy

I often hear the following argument: Improving teacher quality is more cost-effective than other options, such as reducing class size (see here, for example). I am all for evaluating policy alternatives based on their costs relative to their benefits, even though we tend to define the benefits side of the equation very narrowly - in terms of test score gains.

But “improving teacher quality” cannot yet be included in a concrete costs/benefits comparison with class size or anything else. It is not an actual policy. At best, it is a category of policy options, all of which are focused on recruitment, preparation, retention, improvement, and dismissal of teachers. When people invoke it, they are presumably referring to the fact that teachers vary widely in their test-based effectiveness. Yes, teachers matter, but altering the quality distribution is whole different ballgame from measuring it overall. It’s actually a whole different sport.

I think it is reasonable to speculate that we might get more bang for our buck if we could somehow get substantially better teachers, rather than more of them, as would be necessary to reduce class sizes. But the sad, often unstated truth about teacher quality is that there is very little evidence, at least as yet, that public policy can be used to improve it, whether cost-effectively or otherwise.

Positing teacher quality as a concrete policy intervention represents circular reasoning. It’s saying that, if we had more teachers who increase test scores, this would increase test scores. Well, yes. But that’s more of an effect than a means. The relevant policy question is: How do we do so?

Two of the most popular policies that fall under the “teacher quality” banner are merit pay and modified teacher evaluations (including those using growth model estimates). The major benefit of merit pay, according to proponents, is that it could attract a “different type” of candidate to the profession, one who is less risk-averse and more drawn to a job that rewards them according to results. Thus, even if we implemented performance-based pay programs on a widespread basis starting tomorrow, it would still be several years before the aggregate results were discernible, as additional cohorts of “good candidates” entered teaching.

More importantly, given how tough it is to identify and isolate the reasons why people choose careers, it would be a tough haul to demonstrate that a differentiated pay system was actually having the desired effect (here’s one of the first attempts I’ve seen). As I have said before, merit pay is a leap of faith. Now, don’t get me wrong – almost all research on public policy impacts is approximate, even under the best of circumstances. But this is particularly true in this case - we cannot now (and may never be able to) contrast the costs and benefits of merit pay with an alternative like class size reduction.

The outlook is better for gauging the benefits of teacher evaluations. These “teacher quality” interventions are, of course, supposed to identify the highest- and lowest-performing teachers, allowing us to reward the former, and either help or dismiss the latter. There is every reason to believe that better evaluation policies can have a positive effect on teacher quality (and there is already some evidence on this). But this is all in the fairly early stages in most places, and it will be several years before we get a handle on the various effects of all the different types of evaluation changes, as well as on their relative costs (remember that designing, implementing, and maintaining these systems is not cheap).

Obviously, performance pay and evaluations are only two options among several, including increasing salaries, more rigorous hiring practices, teacher induction programs, and the latest idea, increasing class sizes among more “effective” teachers. Among those that have been tested at all, none has yet been proved consistently effective, to say nothing of cost-effective.

One major snag in all this is that we still have a lot of trouble measuring teacher “quality," even when we predefine it in easily-quantified terms, such as testing gains. We know that teachers vary widely in their test-based effectiveness, but, as I have argued before, we still cannot reliably identify the persistent high- and low-performers – or the characteristics that maintain a systematic relationship with performance - at the level of individual teachers. Developing these abilities, which will also have to be based on more than just testing outcomes, is still a work in progress, and its outcome is far from certain.

Personally, I am optimistic that we will know a lot more in the near future. But we are not there yet. In the meantime, any unqualified argument that “improving teacher quality” is a better investment than class size – or anything else – is really just speculation.