Education

What Are "Middle Class Schools"?

An organization called “The Third Way” released a report last week, in which they present descriptive data on what they call “middle class schools." The primary conclusion of their analysis is that “middle class schools” aren’t “making the grade," and that they are “falling short on their most basic 21st century mission: To prepare kids to get a college degree." They also argue that “middle class schools” are largely ignored in our debate and policymaking, and we need a “second phase of school reform” in order to address this deficit.

The Wall Street Journal swallowed the report whole, running a story presenting Third Way’s findings under the headline “Middle class schools fail to make the grade."

To be clear, I think that our education policy debates do focus on lower-income schools to a degree that sometimes ignores those closer to the middle of the distribution. So, it’s definitely worthwhile to take a look at “middle class schools’” performance and how it can be improved. In other words, I’m very receptive to the underlying purpose of the report.

That said, this analysis consists mostly of arbitrary measurement and flawed, vague interpretations. As a result, it actually offers little meaningful insight.

Be Careful What You Think The Public Thinks About Tenure

Monmouth University polling director Patrick Murray offered this response to my criticism of how he described tenure in a recent poll of New Jersey public opinion (see my brief reply and Bruce Baker's as well).

I’m not particularly keen on arguing about the wording of poll questions. As I stated in my original post, wordings are never perfect, and one must always take this into account when trying to interpret polling results. I took issue with Monmouth’s phrasing because it is demonstrably inaccurate, nothing more.

But I do want to quickly add that, as is often the case, responses to poll questions about tenure are very sensitive to the nature of the description offered. A 2009 PDK/Gallup poll provides an illustration.

Serious Misconduct

A recent Monmouth University poll of New Jersey residents is being widely touted by Governor Chris Christie and his supporters as evidence that people support his education reform plans. It’s hardly unusual for politicians to ignore the limitations of polling, but I’d urge caution in interpreting these results as a mandate.

Others have commented on how some of the questions are worded in a manner that could skew responses. These wording issues are inherent to polling, and that’s one of the major reasons why they must be interpreted carefully. But one of the questions caught my eye – the question about teacher tenure – and it’s worth quickly discussing.

Revisiting The Merits Of Merit Pay

Al Shanker was very concerned about the need to identify and replace incompetent teachers. The first time he wrote a column about it, his wife was one of the many people who warned him that the union’s teachers would be up in arms (see here). Shanker wasn’t worried, replying that "All of my members will read that, and they’ll all agree, because not one of them will think that they are one of the bad teachers that I’m talking about."

He was right. Most of the members were very supportive, probably for a variety of reasons. First, most teachers take their responsibilities as teachers very seriously, thus favoring the establishment and enforcement of high standards of professional practice. Second, teachers who don’t believe themselves to be effective are more likely to leave the profession – see here. And third, we know from research that most of us just believe that we are simply better than most other people. Psychologists describe this "illusory superiority" or "above average" effect as the tendency to make self-serving comparisons between oneself and others, with the consequence that an overwhelming majority of people judge themselves to be "better than average" on a variety of traits, skills, and socially desirable dimensions ( here and here).

Nevertheless, there are many teachers who support the idea of performance pay, even if they're wary of the details of how "merit" is defined (specifically, whether or not it includes test scores).

Now, it’s no secret that I think merit pay for teachers is of limited practical utility. Nevertheless, it’s important to understand why, evidence aside, some people (including teachers) might find the policy to be attractive. These are my thoughts on the issue:

The Real Charter School Experiment

The New York Times reports that there is a pilot program in Houston, called the "Apollo 20 Program" in which some of the district’s regular public schools are "mimicking" the practices of high-performing charter schools. According to the Times article, the group of pilot schools seek to replicate five of the practices commonly used by high-flying charters: extended school time; extensive tutoring; more selective hiring of principals and teachers; “data-driven” instruction, including frequent diagnostic quizzing; and a “no excuses” culture of high expectations.

In theory, this pilot program is a good idea, since a primary mission of charter schools should be as a testing ground for new policies and practices that could help to improve all schools. More than a decade of evidence has made it very clear that there’s nothing about "charterness" that makes a school successful – and indeed, only a handful get excellent results. So instead of arguing along the tired old pro-/anti-charter lines, we should, like Houston, be asking why these schools excel and working to see if we can use this information productively.

I’ll be watching to see how the pilot schools end up doing. I’m also hoping that the analysis (the program is being overseen by Harvard’s EdLabs) includes some effort to separate out the effects of each of the five replicated practices. If so, I’m guessing that we will find that the difference between high- and low-performing urban schools depends more than anything else on two factors: time and money.

How Cross-Sectional Are Cross-Sectional Testing Data?

In several posts, I’ve complained about how, in our public discourse, we misinterpret changes in proficiency rates (or actual test scores) as “gains” or “progress," when they actually represent cohort changes—that is, they are performance snapshots for different groups of students who are potentially quite dissimilar.

For example, the most common way testing results are presented in news coverage and press releases is to present year-to-year testing results across entire schools or districts – e.g., the overall proficiency rate across all grades in one year compared with the next. One reason why the two groups of students being compared (the first versus the second year) are different is obvious. In most districts, tests are only administered to students in grades 3-8. As a result, the eighth graders who take the test in Year 1 will not take it in Year 2, as they will have moved on to the ninth grade (unless they are retained). At the same time, a new cohort of third graders will take the test in Year 2 despite not having been tested in Year 1 (because they were in second grade). That’s a large amount of inherent “turnover” between years (this same situation applies when results are averaged for elementary and secondary grades). Variations in cohort performance can generate the illusion of "real" change in performance, positive or negative.

But there’s another big cause of incomparability between years: Student mobility. Students move in and out of districts every year. In urban areas, mobility is particularly high. And, in many places, this mobility includes students who move to charter schools, which are often run as separate school districts.

I think we all know intuitively about these issues, but I’m not sure many people realize just how different the group of tested students across an entire district can be in one year compared with the next. In order to give an idea of this magnitude, we might do a rough calculation for the District of Columbia Public Schools (DCPS).

Predicaments Of Reform

Our guest author today is David K. Cohen, John Dewey Collegiate Professor of Education and professor of public policy at the University of Michigan, and a member of the Shanker Institute’s board of directors. This is a response to Michael Petrilli, who recently published a post on the Fordham Institute’s blog that referred to Cohen’s new book.

Dear Mike:

Thank you for considering my book Teaching And Its Predicaments (Harvard University Press, 2011), and for your intelligent discussion of the issues. I write to continue the conversation. 

You are right to say that I see the incoherence of U.S. public education as a barrier to more quality and less inequality, but I do not "look longingly" at Asia or Finland, let alone take them as models for what Americans should do to improve schools. 

In my 2009 book (The Ordeal Of Equality: Did Federal Regulation Fix The Schools?), Susan L. Moffitt and I recounted the great difficulties that the "top-down" approach to coherence, with which you associate my work, encountered as Title I of the 1965 ESEA was refashioned to leverage much greater central influence on schooling. Susan and I concluded that increased federal regulation had not fixed the schools, and had caused some real damage along with some important constructive effects. We did not see central coherence as The Answer.

Quality Control, When You Don't Know The Product

Last week, New York State’s Supreme Court issued an important ruling on the state’s teacher evaluations. The aspect of the ruling that got the most attention was the proportion of evaluations – or “weight” – that could be assigned to measures based on state assessments (in the form of estimates from value-added models). Specifically, the Court ruled that these measures can only comprise 20 percent of a teacher’s evaluation, compared with the option of up to 40 percent for which Governor Cuomo and others were pushing. Under the decision, the other 20 percent must consist entirely of alternative test-based measures (e.g., local assessments).

Joe Williams, head of Democrats for Education Reform, one of the flagship organizations of the market-based reform movement, called the ruling “a slap in the face” and “a huge win for the teachers unions." He characterized the policy impact as follows: “A mediocre teacher evaluation just got even weaker."

This statement illustrates perfectly the strange reasoning that seems to be driving our debate about evaluations.

Charter And Regular Public School Performance In "Ohio 8" Districts, 2010-11

Every year, the state of Ohio releases an enormous amount of district- and school-level performance data. Since Ohio has among the largest charter school populations in the nation, the data provide an opportunity to examine performance differences between charters and regular public schools in the state.

Ohio’s charters are concentrated largely in the urban “Ohio 8” districts (sometimes called the “Big 8”): Akron; Canton; Cincinnati; Cleveland; Columbus; Dayton; Toledo; and Youngstown. Charter coverage varies considerably between the “Ohio 8” districts, but it is, on average, about 20 percent, compared with roughly five percent across the whole state. I will therefore limit my quick analysis to these districts.

Let’s start with the measure that gets the most attention in the state: Overall “report card grades." Schools (and districts) can receive one of six possible ratings: Academic emergency; academic watch; continuous improvement; effective; excellent; and excellent with distinction.

These ratings represent a weighted combination of four measures. Two of them measure performance “growth," while the other two measure “absolute” performance levels. The growth measures are AYP (yes or no), and value-added (whether schools meet, exceed, or come in below the growth expectations set by the state’s value-added model). The first “absolute” performance measure is the state’s “performance index," which is calculated based on the percentage of a school’s students who fall into the four NCLB categories of advanced, proficient, basic and below basic. The second is the number of “state standards” that schools meet as a percentage of the number of standards for which they are “eligible." For example, the state requires 75 percent proficiency in all the grade/subject tests that a given school administers, and schools are “awarded” a “standard met” for each grade/subject in which three-quarters of their students score above the proficiency cutoff (state standards also include targets for attendance and a couple of other non-test outcomes).

The graph below presents the raw breakdown in report card ratings for charter and regular public schools.

What Americans Think About Teachers Versus What They're Hearing

The results from the recent Gallup/PDK education survey found that 71 percent of surveyed Americans “have trust and confidence in the men and women who are teaching children in public schools." Although this finding received a fair amount of media attention, it is not at all surprising. Polls have long indicated that teachers are among the most trusted professions in the U.S., up there with doctors, nurses and firefighters.

(Side note: The teaching profession also ranks among the most prestigious U.S. occupations – in both analyses of survey data as well as in polls [though see here for an argument that occupational prestige scores are obsolete].)

What was rather surprising, on the other hand, was the Gallup/PDK results for the question about what people are hearing about teachers in the news media. Respondents were asked, “Generally speaking, do you hear more good stories or bad stories about teachers in the news media?"

Over two-thirds (68 percent) said they heard more bad stories than good ones. A little over a quarter (28 percent) said the opposite.