Among Charter Schools, Inconsistency Begets Opportunity

Andrew Rotherham – who writes the blog "Eduwonk" – has also recently started writing a weekly column for Time Magazine. Most of his articles have been interesting and relatively fair, even on the controversial issues. He has a point of view, just like the rest of us, but usually makes a good-faith effort to present alternate viewpoints and the relevant research.

His most recent piece was a partial disappointment. In it, Rotherham takes up the issue of charter schools. His overarching argument is that too many people focus on whether or not charter schools are “better” or “worse” than regular public schools, rather than why – which policies and practices are associated with success or failure.

As I stated in my very first post on this blog (and others), I completely agree. Given the overt politicization of the charter school discussion, the public desperately needs a move away from the pro/anti-charter framework, towards a more useful conversation about how and why particular schools do or don’t work. Their inconsistent performance has caused controversy, but it also an opportunity.

But, when Rotherham lays out the characteristics (“ethos and operations”) that these successful charters supposedly share, the factors he specifies are vague and unsubstantiated – it’s hard to figure what they mean, to say nothing of whether they actually have the stated effect.

Before quickly reviewing Rotherham’s arguments, let’s just establish what we’re looking for here. We know that a minority of charters perform marginally better than comparable regular publics, and a very few achieve stellar results (the same goes for regular public schools). This is the conclusion of the over-cited CREDO study, as well as others.

What we need to see, then, is if there are specific policies or practices – preferably easy to observe and replicable – that are common among high-performing charters, but are typically not found among low- (or average-) performers. The association, of course, doesn’t have to be perfect – nor necessarily demonstrated empirically – but there has to be some reasonable standard by which we can say that a given feature helps to distinguish the more successful charters from their less successful counterparts.

Rotherham identifies three such features. He provides virtually no empirical support for their selection, but fair enough – he knows a lot about the topic and strong research in this area is quite rare. So long as his features are concrete and plausible, he has a lot of leeway to make his case.

The first feature is “tight controls over who teaches in [the schools]." We have to assume he means that these schools are able to hire and fire teachers at will. It sounds plausible until we consider an important fact: The vast majority of charter schools – good, bad, and indifferent – can also hire and fire as they see fit. If most charters share this feature, how can it help us to differentiate among them? Maybe “tight” control can be distinguished from “loose” controls in practice, but most do have the control, so it remains unclear what this actually refers to, how it varies, and whether that matters.

The second feature that supposedly typifies high-performing charters is “a relentless focus on results." This one is so vague as to defy even rough identification. I’m not sure what this means, nor how one could possibly demonstrate to any reasonable standard that it varies by school, to say nothing of whether or not it works to improve performance. Moreover, the phrase has become so ubiquitous in education lingo that I suspect the vast majority of all schools – both charters and regular public – would claim that they are also doing this. Without elaboration – such as specific practices that typify a “relentless focus” – it’s really just a talking point, something you might find in a charter school brochure.

The third feature Rotherham offers is “an intense use of data to inform decisions." Again – this is so common among charters (and regular public schools) as to be unhelpful. The majority of schools – both public and charter – would be only too happy to explain to you how they are pursuing this policy. In this case, Rotherham has also used the qualifier “intensely” – so maybe there is some unstated difference in what successful schools do, that it can be shown that other schools do not. As with the other two features, we are forced to trust him here, but it’s less than satisfying.

By contrast, Rotherham’s subsequent discussion of the reasons for low-performing charter schools is a bit more concrete. He cites a specific study, which he co-authored with Sara Mead - a qualitative review of charter laws and systems laid out in 12 previous case studies (eight states and four cities). Based on this review, Rotherham argues that, for example, charters in states with less oversight and inadequate financial backing tend to be of low “quality."

Oversight mechanisms and funding are concrete policies that are, at least according to this report, associated with a particular outcome. They are also, however, state-level (or district-level) factors. They only show that governments may have a role to play in improving the quality of the charter schools in their states, not why, all else being equal, these schools seem to vary so widely in their results at the school-level.

I would like to add my own, admittedly crude attempt. Rotherham argues that several high-profile charter chains have demonstrated that their schools can be scaled up. He names KIPP, Uncommon Schools, Achievement First, and Aspire as successful chains, as well as two particularly successful individual schools – the Preuss School in California and the MATCH Charter School in Boston.

I don’t know if these schools are unique in how they tightly control their teaching force, relentlessly focus on results, or make intense use of data. I do, however, know that they all share at least one feature in common, which I have discussed before – vast amounts of additional time compared with regular public schools.

Most regular public schools have a 180-day year and a school day of 6-6.5 hours. There is some variation in all these charter chains, but KIPP schools have three weeks of summer school, mandatory Saturdays, and a nine-hour day (roughly 50-60 percent more school time). Achievement First has an 8.5 hour day and summer school. Uncommon Schools also has an 8.5 hour day and a longer school year, as do Aspire schools, which run for 190 days with an extended day. The MATCH school has three weeks of “summer academy” in July, and an 8.5 hour day. The Preuss School in La Jolla, California has a 198-day year and a seven-hour day. Overall, these schools offer between 25-60 percent more time, the equivalent of about 2-4 additional regular public school months. If these schools have something in common that may help explain their larger test score growth, this seems like a stronger possibility (see here for my take on whether extended time is a good idea for regular public schools).

For the record, many of these schools also tout smaller classes and intensive tutoring programs.

Now, this is not to say that Rotherham is wrong. But I think two things would have made his case stronger. First, he might have elaborated a bit as to what his three features actually entail; as stated, they sound more like slogans than explanations. Second, he could have noted that the characteristics he puts forth are far from proven (or even testable) policy options. Isolating the effect of specific charter policies is terribly difficult, and while I find the time factor anecdotally compelling, there is still very little good evidence as to why charters’ results vary (see here, here and here for recent studies; two find tentative support for school time).

It may be some time before we get a good handle on why charter results vary. And it’s certainly possible that the key to success is in fact an “ethos," such as a “relentless focus” on results, as opposed to a more relenting focus. But I’m holding out for more than just an adjective.