Shanker Blog

The Uncertain Future Of Charter School Proliferation

This is the third in a series of three posts about charter schools. Here are the first and second parts.

As discussed in prior posts, high-quality analyses of charter school effects show that there is wide variation in the test-based effects of these schools but that, overall, charter students do no better than their comparable regular public school counterparts. The existing evidence, though very tentative, suggests that the few schools achieving large gains tend to be well-funded, offer massive amounts of additional time, provide extensive tutoring services and maintain strict, often high-stakes discipline policies.

There will always be a few high-flying chains dispersed throughout the nation that get results, and we should learn from them. But there’s also the issue of whether a bunch of charters schools with different operators using diverse approaches can expand within a single location and produce consistent results.

Charter supporters typically argue that state and local policies can be leveraged to “close the bad charters and replicate the good ones." Opponents, on the other hand, contend that successful charters can’t expand beyond a certain point because they rely on selection bias of the best students into these schools (so-called “cream skimming”), as well as the exclusion of high-needs students.

Given the current push to increase the number of charter schools, these are critical issues, and there is, once again, some very tentative evidence that might provide insights.

Why Do Most Americans Support "Assistance To The Poor" But Oppose "Welfare"?

Politicians and other public figures spend a great deal of resources – time and money – on crafting their messages so as to elicit a desired response. A famous example is the effort to relabel the estate tax as the death tax – the former conjures images of very wealthy people paying their fair share, whereas the latter obscures this limited applicability, and invokes outrage at being “taxed just for dying."

As everyone knows, words matter, and these efforts pay off. You don’t need to look at the results of too many surveys or polls to realize that people respond very differently depending on what you call something or how you describe it (e.g., see this post on attitudes toward teacher tenure).

One other particularly interesting – and important – example of this description-based divergence of attitudes toward social programs for the poor.

Explaining The Consistently Inconsistent Results of Charter Schools

This is the second in a series of three posts about charter schools. Here is the first part, and here is the third.

As discussed in a previous post, there is a fairly well-developed body of evidence showing that charter and regular public schools vary widely in their impacts on achievement growth. This research finds that, on the whole, there is usually not much of a difference between them, and when there are differences, they tend to be very modest. In other words, there is nothing about "charterness" that leads to strong results.

It is, however, the exceptions that are often most instructive to policy. By taking a look at the handful of schools that are successful, we might finally start moving past the “horse race” incarnation of the charter debate, and start figuring out which specific policies and conditions are associated with success, at least in terms of test score improvement (which is the focus of this post).

Unfortunately, this question is also extremely difficult to answer – policies and conditions are not randomly assigned to schools, and it’s very tough to disentangle all the factors (many unmeasurable) that might affect achievement. But the available evidence at this point is sufficient to start draw a few highly tentative conclusions about “what works."

The Evidence On Charter Schools

** Also posted here on "Valerie Strauss' Answer Sheet" in the Washington Post and here on the Huffington Post

This is the first in a series of three posts about charter schools. Here are the second and third parts.

In our fruitless, deadlocked debate over whether charter schools “work," charter opponents frequently cite the so-called CREDO study (discussed here), a 2009 analysis of charter school performance in 16 states. The results indicated that overall charter effects on student achievement were negative and statistically significant in both math and reading, but both effects sizes were tiny. Given the scope of the study, it’s perhaps more appropriate to say that it found wide variation in charter performance within and between states – some charters did better, others did worse and most were no different. On the whole, the size of the aggregate effects, both positive and negative, tended to be rather small.

Recently, charter opponents’ tendency to cite this paper has been called “cherrypicking." Steve Brill sometimes levels this accusation, as do others. It is supposed to imply that CREDO is an exception – that most of the evidence out there finds positive effects of charter schools relative to comparable regular public schools.

CREDO, while generally well-done given its unprecedented scope, is a bit overused in our public debate – one analysis, no matter how large or good, cannot prove or disprove anything. But anyone who makes the “cherrypicking” claim is clearly unfamiliar with the research. CREDO is only one among a number of well-done, multi- and single-state studies that have reached similar conclusions about overall test-based impacts.

This is important because the endless back-and-forth about whether charter schools “work” – whether there is something about "charterness" that usually leads to fantastic results – has become a massive distraction in our education debates. The evidence makes it abundantly clear that that is not the case, and the goal at this point should be to look at the schools of both types that do well, figure out why, and use that information to improve all schools.

When The Legend Becomes Fact, Print The Fact Sheet

The New Teacher Project (TNTP) just released a "fact sheet" on value-added (VA) analysis. I’m all for efforts to clarify complex topics such as VA, and, without question, there is a great deal of misinformation floating around on this subject, both "pro-" and "anti-."

The fact sheet presents five sets of “myths and facts." Three of the “myths” seem somewhat unnecessary: that there’s no research behind VA; that teachers will be evaluated based solely on test scores; and that VA is useless because it’s not perfect. Almost nobody believes or makes these arguments (at least in my experience). But I guess it never hurts to clarify.

In contrast, the other two are very common arguments, but they are not myths. They are serious issues with concrete policy implications. If there are any myths, they're in the "facts" column.

Americans Do NOT Want To Cut Government Programs

Conservatives sometimes assert and often imply that Americans want to cut government spending on social assistance and other programs. This is a myth.

In fact, when it comes to the types of programs that get most of the attention in our national debate, almost nobody supports spending reductions and, in many cases, there is strong support for increases.

Take a look at the figure below, which presents General Social Survey data for 2010. Each bar presents the distribution of responses to questions of whether the U.S. spends too much (red), about the right amount (yellow) or too little (green) on several different types of programs and public resources.

The False Conflict Between Unionism and Professionalism

Some people have the unfortunate idea that unionism is somehow antithetical to or incompatible with being a professional. This notion is particularly salient within education circles, where phrases like “treat teachers like professionals” are often used as implicit arguments against policies associated with unions, such as salary schedules and tenure (examples here, here, here and here).

Let’s take a quick look at this "conflict," first by examining union membership rates among professionals versus workers in other types of occupations. As shown in the graph below, if union membership and professionalism don’t mix, we have a little problem: Almost one in five professionals is a union member. Actually, union membership is higher among professionals than among any other major occupational category except construction workers.

Schedule Conflicts

As most people know, the majority of public school teachers are paid based on salary schedules. Most (but not all) contain a number of “steps” (years of experience) and “lanes” (education levels). Teachers are placed in one lane (based on their degree) and proceed up the steps as they accrue years on the job. Within most districts, these two factors determine the raises that teachers receive.

Salary schedules receive a great deal of attention in our education debates. One argument that has been making the rounds for some time is that we should attract and retain "talent" in the teaching profession by increasing starting salaries and/or the size of raises teachers receive during their first few years (when test-based productivity gains are largest). One common proposal (see here and here) for doing so is reallocating salary from the “top” of salary schedules (the salaries paid to more experienced teachers) down to the “bottom” (novice teachers’ salaries). As a highly simplified example, instead of paying starting teachers $40,000 and teachers with 15 years of experience $80,000, we could pay first-year teachers $50,000 and their experienced counterparts $70,000. This general idea is sometimes called “frontloading," as it concentrates salary expenditures at the “front” of schedules.

Now, there is a case for changes to salary schedules in many places – bargained and approved by teachers – including, perhaps, some degree of gradual frontloading (though the research in this area is underdeveloped at best). But there is a vocal group of advocates who assume an all-too-casual attitude about these changes. They seem to be operating on the mistaken assumption that salary schedules can be easily overhauled – just like that. We can drastically restructure them or just “move the money around” without problem or risk, if only unions and "bureaucrats" would get out of the way.**

The Impact Of The Principal In The Classroom

Direct observation is way of gathering data by watching behavior or events as they occur; for example, a teacher teaching a lesson. This methodology is important to teacher induction and professional development, as well as teacher evaluation. Yet, direct observation has a major shortcoming: it is a rather obtrusive data gathering technique. In other words, we know the observer can influence the situation and the behavior of those being observed. We also know people do not behave the same way when they know they are being watched. In psychology, these forms of reactivity are known as the Hawthorne effect, and the observer- or experimenter- expectancy effect (also here).

Social scientists and medical researchers are well aware of these issues and the fact that research findings don’t mean a whole lot when the researcher and/or the study participants know the purpose of the research and/or are aware that they are being observed or tested. To circumvent these obstacles, techniques like “mild deception” and “covert observation” are frequently used in social science research.

For example, experimenters often take advantage of “cover stories” which give subjects a sensible rationale for the research while preventing them from knowing (or guessing) the true goals of the study, which would threaten the experiment’s internal validity – see here. Also, researchers use double-blind designs, which, in the medical field, mean that neither the research participant nor the researcher know when the treatment or the placebo are being administered.

In Research, What Does A "Significant Effect" Mean?

If you follow education research – or quantitative work in any field – you’ll often hear the term “significant effect." For example, you will frequently read research papers saying that a given intervention, such as charter school attendance or participation in a tutoring program, had “significant effects," positive or negative, on achievement outcomes.

This term by itself is usually sufficient to get people who support the policy in question extremely excited, and to compel them to announce boldly that their policy “works." They’re often overinterpreting the results, but there’s a good reason for this. The problem is that “significant effect” is a statistical term, and it doesn’t always mean what it appears to mean. As most people understand the words, “significant effects” are often neither significant nor necessarily effects.

Let’s very quickly clear this up, one word at a time, working backwards.