Three Questions For Those Who Dismiss The Nashville Merit Pay Study

The reaction from many performance pay advocates to the Nashville evaluation released last week has been that the study is relatively meaningless (see here and here for examples).  The general interpretation: The results show that the pay bonuses do not improve student achievement, but short-term test score gains are not the "true purpose" of these incentive programs. What they are really supposed to improve, so the line goes, is the quality of people who pursue teaching as a career, as well as their retention rates.

While I disagree that the findings are not important (they are, if for no other reason than they discredit the idea that teachers are holding their effort hostage to more money), I am sympathetic towards the view that the study didn’t tackle the big issues. Attracting the best possible people into the profession – and keeping them there – are much more meaningful goals than short-term test score gains, and they are not addressed in this study (though some results for retention are reported).

But this argument also begs a few important questions that I hope we can answer before the Nashville study fades into evaluation oblivion.  I have three of them.

How will we ever know if merit pay has the desired effect?  It seems unlikely that we will ever get far beyond highly speculative results on performance pay programs’ effect on improving the quality of the labor supply. There would really be no way even partially to isolate merit pay’s effect from the swirling mass of factors, observed and unobserved, that influence career choices. We could assess whether retention improved, but only for all teachers, not specifically for those who opted to teach because of the bonuses. So, not only do we not know if merit pay “works," but I see no way that we could ever know with any confidence, at least on the “attracting better people” front.

There is nothing necessarily wrong with not knowing – indeed, it is usually the case in public policy that we don’t (and "knowledge" is approximate even when we do). But let’s be clear about our standards and prospects for empirical evidence, both now and in the future. The assumption on the merit pay side is that the most talented young people will be more likely to opt for compensation systems where they might make a lot more money if their students' scores improve more quickly. Is this assumption so safe that we can be comfortable accepting it on pure faith?

What is the evidence that current compensation systems – increased pay for years of experience and more education – haven’t improved the quality and retention of entrants into the profession?  Normally, the argument against teachers’ current experience/education-weighted pay scale is based on these factors’ weak association with short-term test score gains. But dismissing the Nashville study means that this argument is no longer particularly credible. If compensation systems are not really supposed to improve or motivate current teachers, as assessed by their effect on student test scores – only to attract “better” ones and keep them around – then we also need to reassess the utility of education and experience according to this new standard. 

Extra salary for more education (e.g., a master’s degree) would seem to promote self-selection among talented, motivated applicants who either have higher education degrees or would be willing to pursue them. It might also keep these people in the classroom after they got their degrees (by paying them more). As for experience, are we confident that the “best” applicants are more attracted to big, possibly occasional raises based on test scores, rather than to smaller, guaranteed raises over the long haul? Perhaps they are, but what reasons/evidence do we have?

Isn’t the best idea to simply increase teachers’ salaries? I’m not sure that performance incentives will attract better people, but I am certain that higher salaries would. Competition for open positions would increase, and the best applicants would get the jobs. At least that’s the way it seems to work among most of the word’s highest performing nations.

Of course, there are arguments against it. For one thing, it is expensive. It is also not necessarily the most efficient way to spend money – ideally, we would want to target spending directly at attracting and keeping talent (that would get pretty expensive as the talent accumulated and stayed). At the same time, though, incentive programs are also rather expensive (both the bonuses themselves and designing/maintaining the systems), and they (I would argue) carry some risk of not having the desired effect. So, if the true goal is to attract and keep talent, isn't it the safest path to invest in raising teachers’ base salaries?  We could at least be confident that this would produce results.

I would add one final semantic point to these three questions. Given the new, ostensible goals of attracting/retaining talent, the term “performance incentives” is not very appropriate. If, according to the argument, they are not supposed to incentivize performance per se, but rather career decisions (which to choose and how long to stay), perhaps they should be called “career incentives." It has a different ring, doesn’t it?