Charter Schools

Peer Effects And Attrition In High-Profile Charter Schools

An article in last week’s New York Times tells the story of child who was accepted (via lottery) into the highly-acclaimed Harlem Success Academy (HSA), a charter school in New York City. The boy’s mother was thrilled, saying she felt like she had just gotten her son a tuition-free spot in an elite private school. From the very first day of kindergarten, however, her child was in trouble. Sometimes he was sent home early; other times he was forced to stay late and “practice walking the hallways” as punishment for acting out. During his third week, he was suspended for three days.

Shortly thereafter, the mother, who had been corresponding with the principal and others about these incidents, received an e-mail message from HSA founder Eva Moskowitz. Moskowitz told her that, at this school, it is “extremely important that children feel successful," and that HSA, with its nine-hour days, during which children are “being constantly asked to focus and concentrate," can sometimes “overwhelm children and be a bad environment." The mother understood this to be a veiled threat of sorts, but was not upset at the time. Indeed, she credits HSA staff with helping her to find a regular public school for her child to attend. Happily, her son eventually ended up doing very well at his new school.

It’s very important to remember that this is really only one side of the story. It’s also an anecdote, and there is no way to tell how widespread this practice might be at HSA, or at charter schools in general. I retell it here because it helps to illustrate a difficult-to-measure “advantage” that some charter schools have when compared with regular neighborhood schools – the peer effects of attrition without replacement.

The Implications Of An Extreme "No Excuses" Perspective

In an article in this week’s New York Times Magazine, author Paul Tough notifies supporters of market-based reform that they cannot simply dismiss the "no excuses" maxim when it is convenient. He cites two recent examples of charter schools (the Bruce Randolph School in Denver, CO, and the Urban Prep Academy in Chicago) that were criticized for their low overall performance. Both schools have been defended publicly by "pro-reform" types (the former by Jonathan Alter; the latter by the school’s founder, Tim King), arguing that comparisons of school performance must be valid – that is, the schools’ test scores must be compared with those of similar neighborhood schools.

For example, Tim King notes that, while his school does have a very low proficiency rate – 17 percent – his students are mostly poor African-Americans, whose scores should be compared with those of peers in nearby schools. Paul Tough’s rejoinder is to proclaim that statements like these represent the "very same excuses for failure that the education reform movement was founded to oppose." His basic argument is that a 17 percent pass rate is not good enough, regardless of where a school is located or how disadvantaged are its students, and that pointing to the low performance of comparable schools is really just shedding the "no excuses" mantra when it serves one’s purposes.

Without a doubt, the sentiment behind this argument is noble, not only because it calls out hypocrisy, but because it epitomizes the mantra that "all children can achieve." In this extreme form, however, it also carries a problematic implication: Virtually every piece of high-quality education research, so often cited by market-based reformers to advance the cause, is also built around such "excuses."

A 'Summary Opinion' Of The Hoxby NYC Charter School Study

Almost two years ago, a report on New York City charter schools rocked the education policy world. It was written by Hoover Institution scholar Caroline Hoxby with co-authors Sonali Murarka and Jenny Kang. Their primary finding was that:

On average, a student who attended a charter school for all of grades kindergarten through eight would close about 86 percent of the “Scarsdale-Harlem achievement gap” [the difference in scores between students in Harlem and those in the affluent NYC suburb] in math, and 66 percent of the achievement gap in English.

The headline-grabbing conclusion was uncritically repeated by most major news outlets, including the New York Post, which called the charter effects “off the charts," and the NY Daily News, which announced that, from that day forward, anyone who opposed charter schools was “fighting to block thousands of children from getting superior educations." A week or two later, Mayor Michael Bloomberg specifically cited the study in announcing that he was moving to expand the number of NYC charter schools. Even today, the report is often mentioned as primary evidence favoring the efficacy of charter schools.

I would like to revisit this study, but not as a means to relitigate the “do charters work?" debate. Indeed, I have argued previously that we spend too much time debating whether charter schools “work," and too little time asking why some few are successful. Instead, my purpose is to illustrate an important research maxim: Even well-designed, sophisticated analyses with important conclusions can be compromised by a misleading presentation of results.

In Census Finance Data, Most Charters Are Not Quite Public Schools

Last month, the U.S. Census Bureau released its annual public K-12 school finance report (and accompanying datasets). The data, which are for FY 2009 (there’s always a lag in finance data), show that spending increased roughly two percent from the previous year. This represents much slower growth than usual.

These data are a valuable resource that has rightfully gotten a lot of attention. But there’s a serious problem within them, which, while slightly technical, hasn’t received any attention at all: The vast majority of public charter schools are not included in the data.

To gather its data, the Census Bureau relies on reporting from “government entities." Some charter schools fit this description neatly, such as those operated by governments or government-affiliated bodies, including states, districts, counties, and public universities. But most charter schools are operated by private organizations (mostly non-profits), and finance figures for these schools are not included in the report (the Census classifies them as "private charter schools").

What does this mean? Well, for one thing, it means that the overall spending figures (total dollar amounts) are a bit understated. Charters only account for a relatively small proportion of all public school enrollments (around 5-6 percent); still, given the huge amounts of money we’re dealing with here (the U.S. spends roughly $600 billion a year), we’re talking about quite a bit in absolute terms. Perhaps more important is the potential effect on per-pupil spending figures – the way that education financing is usually expressed.

A Response To Joel Klein

Our guest author today is Edith (Eadie) Shanker, Albert Shanker’s widow and a retired New York City teacher.

A few months ago, in the Wall Street Journal (WSJ), Joel Klein invoked Al Shanker’s name as an educator in support of today’s charter school “reform” efforts. Klein wanted the public to believe that Al was the originator of the charter school concept (he wasn’t) and that he would today be supportive of the charter school ”reform” ideology now being spread around New York City and the country as a panacea for low student achievement. Conveniently, Klein did not indicate that Al denounced the idea of charters when it became clear that the concept had changed and was being hijacked by corporate and business interests. In Al’s view, such hijacking would result in the privatization of public education and, ultimately, its destruction - all without improving student outcomes.

Now, in his recent Atlantic magazine article, Klein trots out a quotation attributed to Al (said in jest if at all) to support the stereotype that, as a union leader, Al cared only about “protecting” the union’s members, including “bad” teachers. Using this alleged quotation – “when school children start paying union dues, that’s when I’ll start representing the interests of children” - Klein not only plays fast and loose with Al’s reputation as a union leader but also as a sterling educator. (To be a true expert on Al’s views on how to improve education for children - and how to be a union leader - Klein could check out 27 years’ worth of his “Where We Stand” columns in the New York Times.)

Revisiting The CREDO Charter School Analysis

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post

Most people involved in education policy know exactly what you mean when you refer to “the CREDO study." I can’t prove this, but suspect it may be the most frequently mentioned research report over the past two years (it was released in 2009).

For those who haven’t heard of it (or have forgotten), this report, done by the Center for Research on Education Outcomes (CREDO), which is based at Stanford University, was a comparison of charter schools and regular public schools in 15 states and the District of Columbia. Put simply, the researchers matched up real charter school students with fictional amalgamations of statistically similar students in the same area (the CREDO team called them “virtual twins”), and compared charter school students’ performance (in terms of test score gains) to that of their “twins." The “take home” finding – the one that everybody talks about – was that, among the charter schools included in the analysis, 17 percent had students who did better on the whole than their public school twins, in 37 percent they did worse, and in 46 percent there was no statistical difference. Results varied a bit by student subgroup and over time.

There are good reasons why this analysis is mentioned so often. For one thing, it remains the largest study of charter school performance to date, and despite some criticism that the "matching" technique biased charter effects downward, it was also a well done large-scale study (for a few other good multi-state charter studies, see here, here, and here). Nevertheless, as is so often the case, the manner in which its findings are discussed and understood sometimes reflect a few key errors of interpretation. Given that it still gets attention in major media outlets, as well as the fact that the CREDO team continues to release new state-specific reports (the latest one is from Pennsylvania), it makes sense to quickly clear up three of the most common misinterpretations.

Among Charter Schools, Inconsistency Begets Opportunity

Andrew Rotherham – who writes the blog "Eduwonk" – has also recently started writing a weekly column for Time Magazine. Most of his articles have been interesting and relatively fair, even on the controversial issues. He has a point of view, just like the rest of us, but usually makes a good-faith effort to present alternate viewpoints and the relevant research.

His most recent piece was a partial disappointment. In it, Rotherham takes up the issue of charter schools. His overarching argument is that too many people focus on whether or not charter schools are “better” or “worse” than regular public schools, rather than why – which policies and practices are associated with success or failure.

As I stated in my very first post on this blog (and others), I completely agree. Given the overt politicization of the charter school discussion, the public desperately needs a move away from the pro/anti-charter framework, towards a more useful conversation about how and why particular schools do or don’t work. Their inconsistent performance has caused controversy, but it also an opportunity.

But, when Rotherham lays out the characteristics (“ethos and operations”) that these successful charters supposedly share, the factors he specifies are vague and unsubstantiated – it’s hard to figure what they mean, to say nothing of whether they actually have the stated effect.

Digging For Data In The Garden State

In January, the New Jersey Department of Education released a report titled, "Living Up to Expectations: Charter Schools in New Jersey Outperforming District Schools." It consisted of a list of charter schools and their students’ aggregate proficiency rates by grade, along with comparisons with the rates of the regular public school districts in which they are located. The state then tallied the number of charters with higher rates (79 percent in language arts, and 69 percent in math), and concluded - in a press release - that this represented evidence of superior performance. The conclusion was reported without scrutiny. Later that same day, NJ Governor Chris Christie formally announced his plan to expand the state’s charter school sector.

In a short post that evening, I pointed out the obvious fact that the state’s analysis was wholly inadequate to demonstrate charter performance – good, bad or indifferent – relative to comparable regular public schools. Rutgers Professor Bruce Baker did the same, and also presented a school-level analysis showing that there was no difference.

Christopher Cerf, the state’s acting education commissioner, decided to stand by the suspect results, basically saying that they were imperfect but good enough to draw the conclusions from.

It was an astonishing position.

Policy And Research: A Shotgun Wedding In New Jersey

Earlier today, New Jersey Governor Chris Christie announced his plan to open 23 new charter schools in his state.  Just hours before this announcement, the NJ education department issued an analysis of new data on the performance of charter schools in the state (during the 2009-10 school year).   In an accompanying press release, the department claims that “the data affirms [sic] the need for Governor Christie’s reform proposals to grow the number of high-quality charter schools…” 

The release also contains several other extremely bold assertions that the results support expanding the state’s charter sector.  The title of the actual report, which contains only tables, is: "Living Up to Expectations: Charter Schools in New Jersey Outperforming District Schools."

Unfortunately, however, the analysis could barely pass muster if submitted by a student in one of the state’s high school math classes (charter or regular public).

"No Comment" Would Have Been Better

Bruce Baker is a professor at Rutgers University who writes an informative blog called School Finance 101.  He presented some descriptive analysis of New Jersey charter schools in a post, and ended up being asked to comment on the data by a reporter.  The same reporter dutifully asked the New Jersey Charter Schools Association (NJCSA) to comment on the analysis. 

The NJCSA describes itself as “the leading statewide advocate for charter public schools in New Jersey and a principal source of public information about charter schools in the state.”  The organization issued the following response to Baker’s analysis:

The New Jersey Charter Schools Association seriously questions the credibility of this biased data. Rutgers University Professor Bruce Baker is closely aligned with teachers unions, which have been vocal opponents of charter schools and have a vested financial interest in their ultimate failure.

Baker is a member of the Think Tank Review Panel, which is bankrolled by the Great Lakes Center for Education Research and Practice. Great Lakes Center members include the National Education Association and the State Education Affiliate Associations in Illinois, Indiana, Michigan, Minnesota, Ohio and Wisconsin. Its chairman is Lu Battaglieri, the executive director of the Michigan Education Association.

There are now thousands of children on waiting lists for charters schools in New Jersey. This demand shows parents want the option of sending their children to these innovative schools and are satisfied with the results.

Note the stretch that they have to make to allege that Baker is “closely aligned” with teachers unions—he occasionally reviews papers for an organization that is partly funded by unions. There is no formal connection beyond that. Note also that the NJCSA statement “questions the credibility of [sic] this biased data”—meaning they doubt the credibility of data from the State of New Jersey, which Baker merely recasts as graphs and maps. There is not a shred of substance in this statement that addresses the data or Baker’s description of them. It’s pure guilt by association (and there’s not really even an association).