The Cost Of Success In Education

Many are skeptical of the current push to improve our education system by means of test-based “accountability” - hiring, firing, and paying teachers and administrators, as well as closing and retaining schools, based largely on test scores. They say it won’t work. I share their skepticism, because I think it will.

There is a simple logic to this approach: when you control the supply of teachers, leaders, and schools based on their ability to increase test scores, then this attribute will become increasingly common among these individuals and institutions. It is called “selecting on the dependent variable," and it is, given the talent of the people overseeing this process and the money behind it, a decent bet to work in the long run.

Now, we all know the arguments about the limitations of test scores. We all know they’re largely true. Some people take them too far, others are too casual in their disregard. The question is not whether test scores provide a comprehensive measure of learning or subject mastery (of course they don’t). The better question is the extent to which teachers (and schools) who increase test scores a great deal are imparting and/or reinforcing the skills and traits that students will need after their K-12 education, relative to teachers who produce smaller gains. And this question remains largely unanswered.

This is dangerous, because if there is an unreliable relationship between teaching essential skills and the boosting of test scores, then success is no longer success. And by selecting teachers and schools based on those scores, we will have deliberately engineered our public education system to fail in spite of success.

It may be only then that we truly realize what we have done.

If they can afford it, many students with higher scores still will not be well-prepared to make it in college (even SAT and ACT scores are unreliable predictors of college readiness and success). Others will enter the labor force, and find that employers are still hiring based on skills that they have always needed – like the ability to think creatively, communicate effectively, and problem-solve. These employers will slowly realize that the information they used to rely on as signals of these skills is no longer transmitting them faithfully. And they will, of course, act accordingly.

Very ironically, our obsession with standardized tests may be the very thing that threatens the validity that they do have, and I suspect that the (already murky) connection between scores and future success in college and career will slowly but surely weaken even further (if it hasn’t already).

So, if we “succeed” in boosting test scores, as I believe we eventually will, the honest among us may have to acknowledge that we didn’t get what we bargained for, and that the system we created was as much about “decision-driven data” as “data-driven decisions." We might regret having so aggressively selected and deselected our teacher and administrator workforce in the service of test scores for the sake of test scores, and having succumbed to the siren song of quantifiable measures without understanding what they are measuring and how this is affected by the way we use them.

Don’t get me wrong: I believe deeply in using quantitative research to guide policy. I also believe that test scores do provide meaningful information, and that they have an important role to play in education. The advances that researchers of all stripes are making in using testing data to understand teacher effects, student learning, and school effectiveness are a credit to these people and an asset to our education system.

Using test scores as one yardstick for success and failure is a good idea, but we have granted tests a judge/jury/executioner role that largely ignores the expert consensus on their limitations. Making things even worse, our strategy for increasing test scores seems to be little more than finding more teachers, administrators, and schools that increase test scores, and firing/closing those that do not. The ends don’t justify the means; the ends are the means. This tautological approach to improving education may not only lead to failure dressed up as success, but it might also end up seriously blunting one of our best tools: the more we blindly rely on testing data for making high-stakes decisions in our education system, the more unreliable those data become.

Without a primary emphasis on why test scores rise, and whether the reasons reflect our priorities, we may be dooming ourselves to a future of uncertainty and false positives, one in which success and failure co-exist peacefully, right under our noses, where nobody can see them.