Persistently Low-Performing Incentives

Today, the National Center on Performance Incentives (NCPI) and the RAND Corporation released a long-awaited experimental evaluation of teacher performance pay in Nashville, Tenn. It finds that performance bonuses have virtually no effect on student math test scores (there were small but significant gains by fifth graders, but only in two of the three years examined, and the gains did not last into sixth grade).

Since this is such a politically contentious issue, these findings are likely to spark a lot of posturing and debate. So it’s worth trying to put them in context. As I discussed in a prior post, we now have at least preliminary results from three randomized experimental evaluations of merit pay in the U.S., the first contemporary, high-quality evidence of its kind.  This Nashville report and the two previously-released studies – one from Chicago and one from New York City's schoolwide bonus program – reached the same conclusion: Performance bonuses for teachers have little or no discernable effect on student test scores. 

Although the NYC and Chicago findings are preliminary (the evaluations are still in progress), the NYC program provides schoolwide and not individual bonuses, and one additional study (Round Rock, Tex.) is yet to be released, the three already-released reports do represent a fairly impressive, though still very tentative, body of evidence on merit pay’s utility as a means to improve test scores.

And at this point, it’s a good bet that, when all the evaluations are final and the smoke has cleared, we will have to conclude that performance bonuses are, at the very least, a very unpromising policy for producing short-term test score gains. 

So, what does all this mean?  There will be the predictable denials and overreactions at the extremes.  Merit pay supporters will likely offer methodological criticisms or say that the results are meaningless because they miss the “real issues.”  Opponents will use the results to argue that the very idea of performance pay is anathema to improving public education.

And both arguments have at least some merit. Test scores aren’t the only thing that matters (though you wouldn’t know it from listening to the way people talk). Other outcomes, both desired (more capable people pursuing teaching, higher retention) and undesired (narrowing the curriculum, more turnover), will most likely require more than a few years to manifest themselves. Moreover, it’s possible that these other outcomes, once they pick up steam, might generate test score changes over the longer-term.

On the other hand, those who wage war on the accountability battlefield must also take their lumps on it.  The best experimental evaluation in U.S. history, which took five years and cost millions of dollars, shows that teacher bonuses – as high as $15,000 – did not produce even minimal improvements in students’ scores, and these findings are complemented by preliminary results from two other performance pay programs (one schoolwide) in major cities.

Let’s be honest: Those who criticize current teacher compensation systems argue that the primary factors in these systems (experience and education) are not related to teachers’ ability to boost test scores.  For these people, performance incentives should, at least in part, be judged by the same standard – whether they improve scores.  They don’t.

More broadly, it may soon be time for us to acknowledge that the massive policy-changing scramble created by Race to the Top was too rushed and prescriptive.  Since the time that the RTTT regulations were finalized, with their emphasis on performance incentives and charter schools, red flags have been raised by the best analysis to date on performance incentives (this Nashville study) and one of the best on charter schools.

I’m not sure this evidence would have swayed many minds when the (rather short and limited) debate on RTTT was occurring.  But I can say that it would have benefited everyone – from both sides – to have had more evidence in hand before the policies were written into law.