The Real Charter School Experiment

The New York Times reports that there is a pilot program in Houston, called the "Apollo 20 Program" in which some of the district’s regular public schools are "mimicking" the practices of high-performing charter schools. According to the Times article, the group of pilot schools seek to replicate five of the practices commonly used by high-flying charters: extended school time; extensive tutoring; more selective hiring of principals and teachers; “data-driven” instruction, including frequent diagnostic quizzing; and a “no excuses” culture of high expectations.

In theory, this pilot program is a good idea, since a primary mission of charter schools should be as a testing ground for new policies and practices that could help to improve all schools. More than a decade of evidence has made it very clear that there’s nothing about "charterness" that makes a school successful – and indeed, only a handful get excellent results. So instead of arguing along the tired old pro-/anti-charter lines, we should, like Houston, be asking why these schools excel and working to see if we can use this information productively.

I’ll be watching to see how the pilot schools end up doing. I’m also hoping that the analysis (the program is being overseen by Harvard’s EdLabs) includes some effort to separate out the effects of each of the five replicated practices. If so, I’m guessing that we will find that the difference between high- and low-performing urban schools depends more than anything else on two factors: time and money.

If we quickly review the five “best practices," they’re a mixed bag in terms of specificity and evidence, and, as is often the case in policymaking, most present challenges in terms of replication.

One of them – more selective hiring of teachers and principals – sounds like a no-brainer. In practice, though, how districts can improve their selection processes is very much an open question, since the available evidence suggests that it’s very tough to predict systematically which prospective teachers will be most effective. The same goes for principals.

I couldn’t find many details about hiring from the program’s materials, other than the fact that they’re using performance bonuses to “attract talent." I’d be interested in seeing more about this particular policy, and which selection criteria the schools will be using (see here, here and here for ideas). I would also point out that we sometimes fail to consider an important, if unfortunate reality: In the poorest schools and districts, there is often a shortage of people willing to take and keep teaching/administrative jobs, and this limits the degree to which the benefits of selectivity hold up under wider implementation. Similarly, the program is selecting principals using a national search process, which becomes far more difficult on a larger scale.

The second key practice – data-driven practices, including diagnostic quizzing – strikes me as a decent idea, and there is research showing that it can yield modest benefits (also here and here). Again, the details are important: How frequently will the quizzes be administered? Who designs them? How will the results of these assessments and other data be used? And so on.

The third policy – adopting a “no excuses” approach – is extremely vague. In education conversations, the term “no excuses” usually means that people should not use poverty as an excuse for low-performance. The program’s fact sheet suggests that the “no excuses” component consists mostly of high expectations for student performance – 100 percent proficiency, high attendance, etc. But all of these are attitudes and outcomes, not concrete interventions.

In its policy manifestation, “no excuses” might include parental accountability (including parent contracts, which the Houston program is using), strict discipline policies and an emphasis on regulating student behavior and comportment (see discussion here).

In “no excuses” charter schools, students who fail to live up to these behavioral expectations risk suspension (and perhaps expulsion), while many who don’t do well end up back in regular public schools, sometimes per the terms of the parental contracts. I’m hoping that, unlike some “no excuses” charters, the Houston pilot schools will not have the ability to lose students without having to replace them (which generates peer effects that can influence outcomes).

Rather, I’m hoping that they will operate as per usual – taking all comers and figuring out how to deal with the students they have. If this is the case, then the pilot schools will represent an opportunity to see how these “no excuses” policies, such as parental contracts, work in a typical public school context. For example, what will happen to students who are discipline problems? If they are expelled or encouraged to leave the school, then this severely limits the degree to which the Houston program will be testing the scalability of this type of intervention.

The two remaining “best practices” – extensive tutoring and extended time – are, in my view, among the most compelling possible factors explaining the performance of the high-flying charters. Actually, I have been unable to find a single high-profile charter chain that doesn’t provide at least 15-20 percent more time (through extended days and years) than nearby district schools. Many of them provide a great deal more than that – most KIPP schools, for example, offer around 50 percent more time than do comparable public schools, the equivalent of roughly four regular public school months (see here, here and here for some evidence on the effect of additional time in charters).

And there’s no question that, when done correctly, in-school tutoring programs – whether one-on-one or small groups – are an effective intervention for struggling students (see here and here for examples). Some well-known charter chains, such as Achievement First and the MATCH school in Boston offer extensive one-on-one instruction, while in other nations, private tutoring plays a huge role.

Now, don’t get me wrong – these are good schools for many reasons. They are carefully planned and thoughtfully managed (as are many regular public schools). But let’s face it, an effective tutoring program and/or a well-planned school day that is 3-4 hours longer can provide a real boost to student performance – especially when the students who can’t handle it sometimes leave.

These two policies also share other things in common. They are not especially innovative or new. And, as policy interventions, they don’t necessarily rely on doing things better (though both tutoring and time must be carefully-designed), but rather on doing them more – more direct, personalized instruction for longer periods of time. In some respects, it’s a “blunt force” approach.

Finally, and perhaps most importantly, both policies are expensive. Hiring tutors is obviously cost-intensive, as is extended time, for which buildings much be kept open longer and school employees compensated. It also bears mentioning that other aspects of the Apollo 20 program, such as performance bonuses, frequent assessment and setting up data collection and dissemination systems, can also be costly.

Charter schools such as KIPP rely on huge private donations to meet these and other costs. The Houston pilot costs $19 million per year, for a relatively small group of schools (and the schools will not be offering as much additional time as many high-profile charters). At least some of that funding is also coming from private donations, including grants from the Gates Foundation and Wells Fargo.

So, I hope the Houston program ends up yielding good results and I hope that researchers are able to determine which policies made the biggest contributions to this success. I believe, based on the available evidence, that extended time and tutoring will be determined to be the primary (but not sole) drivers of any improvement. If I’m correct, then I can’t help but wonder how the education policy community will react. Will people put politics aside and at least consider the kind of massive investment that these policies require? I hope that too.

- Matt Di Carlo