In Education Policy, Good Things Come In Small Packages

A recent report from the U.S. Department of Education presented a summary of three recent studies of the differences in the effectiveness of teaching provided advantaged and disadvantaged students (with the former defined in terms of value-added scores, and the latter in terms of subsidized lunch eligibility). The brief characterizes the results of these reports in an accessible manner - that the difference in estimated teaching effectiveness between advantaged and disadvantaged students varied quite widely between districts, but overall is about four percent of the achievement gap in reading and 2-3 percent in math.

Some observers were not impressed. They wondered why so-called reformers are alienating teachers and hurting students in order to address a mere 2-4 percent improvement in the achievement gap.

Just to be clear, the 2-4 percent figures describe the gap (and remember that it varies). Whether it can be narrowed or closed – e.g., by improving working conditions or offering incentives or some other means – is a separate issue. Nevertheless, let’s put aside all the substantive aspects surrounding these studies, and the issue of the distribution of teacher quality, and discuss this 2-4 percent thing, as it illustrates what I believe is the among the most important tensions underlying education policy today: Our collective failure to have a reasonable debate about expectations and the power of education policy.

On one hand, when reading the tepid reaction to the 2-4 percent comments, the policy researcher in me immediately wants to say: 2-4 percent is a lot.

Let’s say we found a cost-effective, widely-supported (including by teachers) way to equalize teacher effectiveness between poor and more affluent schools. And then let’s say that doing so improved the achievement gap by a few percentage points over a given period of time. That would be a major accomplishment, especially at the state or national level. It may not sound like much, but even small improvements can make a big difference for thousands of kids, and this is just one type of policy, which, along with other improvements, could make a sizeable dent (though it’s worth noting that such redistribution, without an increase in the quality distribution overall, would benefit some students at the expense of others).

On the other hand, I can understand that some people are less than impressed, as the interpretation of any effect size must rely on a frame of reference. And we live in a nation where much of rhetoric from advocates and high-level leaders implies, and sometimes states directly, that education policy changes can and will generate massive aggregate improvements over relatively short periods of time.

We are bombarded constantly with catchphrases such as “all students must be college and career ready," told that we can “close the achievement gap” relatively quickly, and promised that the lowest-performing districts can become the highest-performing within five years. Administrators in many states and districts are expected to make such promises. And, of course, NCLB dictated that the vast majority of students had to be proficient within 10 years.

Anyone who dares point out that these goals are, in reality, multi-generational endeavors risks being accused of believing that “poverty is destiny," or that “some children cannot learn." And, making things worse, the primary policy agenda among many of those leveling such accusations consists largely of interventions, such as personnel policies, that are either totally untested or have never been shown to have anything beyond what by these grandiose standards can only be called a modest impact in the short-term.

In that context, it’s hardly surprising that 2-4 percent improvement would seem to some like a drop in the bucket.

I can fully understand the need to avoid complacency, but acknowledging that even the most effective educational interventions require patience and realism is not tantamount to maintaining low expectations, or admitting students can’t learn. It is time-tested, sound policymaking. It is saying that we should test and make changes that improve outcomes, but that we should avoid at all costs ending policies that are working because they’re not working quickly enough, and refusing to implement policies that seem like small potatoes simply because we don’t know how large potatoes should be.

So, no, the U.S. will not rise to the level of the highest-performing nations within a few years, or even within 10-20 years. We will not eradicate achievement gaps, or bring all students up to proficiency, within this time frame. And the reason is because real progress is slow and sustained. It takes decades, and much of the progress will be intergenerational - children who receive a better education today will pass that on to their children, and so on.

In the meantime, let’s remember that real improvement often occurs in 2-4 percent increments.

- Matt Di Carlo