Education

Digging For Data In The Garden State

In January, the New Jersey Department of Education released a report titled, "Living Up to Expectations: Charter Schools in New Jersey Outperforming District Schools." It consisted of a list of charter schools and their students’ aggregate proficiency rates by grade, along with comparisons with the rates of the regular public school districts in which they are located. The state then tallied the number of charters with higher rates (79 percent in language arts, and 69 percent in math), and concluded - in a press release - that this represented evidence of superior performance. The conclusion was reported without scrutiny. Later that same day, NJ Governor Chris Christie formally announced his plan to expand the state’s charter school sector.

In a short post that evening, I pointed out the obvious fact that the state’s analysis was wholly inadequate to demonstrate charter performance – good, bad or indifferent – relative to comparable regular public schools. Rutgers Professor Bruce Baker did the same, and also presented a school-level analysis showing that there was no difference.

Christopher Cerf, the state’s acting education commissioner, decided to stand by the suspect results, basically saying that they were imperfect but good enough to draw the conclusions from.

It was an astonishing position.

Choosing A Superintendent - Or A Chancellor

Our guest author today is Sol Hurwitz, president emeritus of the Committee for Economic Development and a member of the Albert Shanker Institute’s Board of Directors.

Early in January, less than two weeks into her tenure as chancellor of the New York City Public Schools, Cathleen P. Black found herself mired in controversy over a remark she made to parents distraught over their children’s overcrowded schools. “Couldn’t we just have some birth control for a while?" she joked.

The media pounced, and Ms. Black squandered an opportunity to address one of the school system’s most acute problems. The chancellor’s subsequent public appearances have provoked boos and jeering, to which her responses have veered from silence to mocking sarcasm. Her challenge now is to dispel the widely-held notion that she is unfit to hold her job.

An experienced educator facing a group of worried parents probably would not have made such a gaffe. But Ms. Black, the former president and chairwoman of Hearst Magazines, is not an educator; nor has she or her children ever attended a public school. Clearly, her experience as a publishing executive did not prepare her for the rough-and- tumble, media-driven politics of New York City’s schools.

A Call For Common Content

A diverse group of influential education and other leaders today announced support for clear curricular guidance to complement the new Common Core State Standards that have been adopted by most states. 

Today’s statement released by the nonpartisan Albert Shanker Institute and signed by dozens of educators, advocates, policymakers, researchers and scholars from across the educational and political spectrum, highlights one largely ignored factor needed to enable  American students to achieve to high levels and become internationally competitive—the creation of voluntary model curricula that can be taught in the nation’s classrooms.

Randi Weingarten, president of the American Federation of Teachers, urged broad support and dissemination for the statement, "A Call for Common Content". “We are arguing for the tools and materials that teachers need," she said. “With rich, sequential common curricula, amplified by state and local content—and with teacher preparation, classroom materials, student assessments,  teacher development, and teacher evaluation all aimed at  the mastery of that content—we can finally build the kind of coherent system that supports the achievement of all learners; the kind of system enjoyed by the world’s highest performing nations."

The release of “A Call for Common Content” comes at a special time. After decades of debate, the nation is finally on its way to having common, voluntary standards in mathematics and English language arts. Although this recent state-led effort is an important and positive first step, notes the statement, it is not sufficient to achieve a well-functioning education system that offers both excellence and opportunity.

Would The New York City Layoffs Hurt Poor Schools More?

As first reported by the New York Times, the New York City Department of Education released a dataset this past Sunday, which lists the number of potential teacher layoffs that would occur in each school absent a budget infusion.

Layoffs are a terrible thing for schools and students, and this list is sobering. But the primary impetus for releasing for this dataset appears to be the city’s ongoing push to end so-called seniority-based layoffs, and its support for seniority-ending legislation that is now making its way through the state legislature. One of the big talking points on this issue has always been that layoffs that take experience into account would hurt high-poverty schools the most, because these schools tend to have the least experienced teachers. As I discussed in a prior post, Michelle Rhee is making this argument everywhere she goes, and it was one of the primary themes in a new report by the New Teacher Project (released last week). Although I have not heard city officials use the argument since the database was released over the weekend, similar assertions have very recently been made by Mayor Bloomberg, former Chancellor Joel Klein, and current Chancellor Cathie Black.

I find all this a bit curious, given that the best research on the topic finds that the argument is untrue (including a study of New York City, and a statewide analysis of Washington [also here]). Now, it is at least possible that, if layoffs were conducted strictly on the basis of seniority, higher-poverty schools could end up bearing the brunt of dismissals. This is almost never the case, however – layoffs in almost every district proceed based on a variety of criteria, among which seniority is only one (albeit often the most important).

It is fortuitous, then, that the city’s dataset provides an opportunity to test the claim that the “worst-case scenario” – over 4,500 layoffs using current New York City procedures – would hurt high-poverty schools the most. Let’s take a look.

A List Of Education And Related Data Resources

We frequently present quick analyses of data on this blog (and look at those done by others). As a close follower of the education debate, I often get the sense that people are hungry for high-quality information on a variety of different topics, but searching for these data can be daunting, which probably deters many people from trying.

So, while I’m sure that many others have compiled lists of data resources relevant to education, I figured I would do the same, with a focus on more user-friendly sources.

But first, I would be remiss if I didn’t caution you to use these data carefully. Almost all of the resources below have instructions or FAQ’s, most non-technical. Read them. Remember that improper or misleading presentation of data is one of the most counterproductive features of today’s education debates, and it occurs to the detriment of all.

That said, here are a few key resources for education and other related quantitative data. It is far from exhaustive, so feel free to leave comments and suggestions if you think I missed anything important.

Students First, Facts Later

On Wednesday, Michelle Rhee’s new organization, Students First, rolled out its first big policy campaign: It’s called “Save Great Teachers," and it is focused on ending so-called “seniority-based layoffs."

Rhee made several assertions at the initial press conference and in an accompanying op-ed in the Atlanta Constitution Journal (and one on CNN.com). At least three of these claims address the empirical research on teacher layoffs and quality. Two are false; the other is misleading. If history is any guide, she is certain to repeat these “findings” many times in the coming months.

As discussed in a previous post, I actually support the development of a better alternative to seniority-based layoffs, but I am concerned that the debate is proceeding as if we already have one (most places don't), and that there's quite a bit of outrage-inspiring misinformation flying around on this topic. So, in the interest of keeping the discussion honest, as well as highlighting a few issues that bear on the layoff debate generally, I do want to try and correct Rhee preemptively.

A Wisconsin Moment For Our Education Policy Debate

There is an obvious, albeit somewhat uncomfortable connection between what’s happening in Wisconsin and what’s been happening in education policy discussions.

A remarkably high proportion of the discussion is focused – implicitly or explicitly – on the presumed role of teachers’ unions. The public is told that our school systems are failing, and that teachers’ unions are at least partially to blame because they protect bad teachers and block “needed” reforms such as merit pay. In this storyline, unions are faceless villains that put the interests of adults above those of children.

Wisconsin represents a threat to this perspective in at least three important manners.

Revisiting The Effect Of Teachers' Unions On Student Test Scores

The Wisconsin protests have predictably spurred a great deal of information-seeking, with union supporters and opponents alike searching for evidence that supports their cases. One of the most prevalent topics over the past week or so is the effect of teacher collective bargaining on student test scores. As a result, a couple of our previous posts have been shared widely. The first (also republished here) compares NAEP scores in states that allow binding teacher contracts with those in states that do not (or have only one or two); the second, follow-up post offers some additional, multivariate analysis.

Although it is true that the first post (which was at least partially satirical - see the last few sentences) shows that states without binding contracts are among the lowest-performing in the nation, I want to clear something up: As I noted in both posts, neither the data nor my argument offer any conclusive proof that teacher contracts act to increase student test scores. The intention of those posts was to address the age-old counter claim – that teacher contracts are somehow injurious to student achievement – and to provide very tentative evidence that the contracts appear to have little discernible impact either way (which is what the follow-up post, using state-level models that controlled for basic student characteristics, indicated, along with the requisite caveats).

This speaks directly to those who seek to blame unions for poor achievement in the U.S. - if union contracts were in fact a major contributing cause of low test performance, it might be reasonable to expect to find at least some clear differences between states that did and did not allow them. Although my analysis was extremely limited, I found no such evidence.

But this also applies to those who have been claiming recently – many in the Wisconsin context – that teacher bargaining clearly improves these outcomes.

The Legend Of Last Fall

The subject of Michelle Rhee’s teaching record has recently received a lot of attention. While the controversy has been interesting, it could also be argued that it’s relatively unimportant. The evidence that she exaggerated her teaching prowess is, after all, inconclusive (though highly suggestive). A little resume inflation from a job 20 years ago might be overlooked, so long as Rhee’s current claims about her more recent record are accurate. But are they?

On Rhee’s new website, her official bio - in effect, her resume today (or at least her cover letter) - contains a few sentences about her record as chancellor of D.C Public Schools (DCPS), under the header "Driving Unprecedented Growth in the D.C. Public Schools." There, her test-based accomplishments are characterized as follows:

Under her leadership, the worst performing school district in the country became the only major city system to see double-digit growth in both their state reading and state math scores in seventh, eighth and tenth grades over three years.

This time, we can presume that the statement has been vetted thoroughly, using all the tools of data collection and analysis available to Rhee during her tenure at the helm of DCPS.

But the statement is false.

A Quality-Based Look At Seniority-Based Layoffs

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post

Eliminating seniority-based layoffs is a policy idea that is making the rounds these days, with proponents making special appeals to cash-strapped states and districts desperately looking for ways to save money while minimizing decreases in the quality of services.  Mayors, editorial boards, and others have joined in the chorus.

There’s a few existing high-quality simulations that compare seniority-based layoffs with one alternative – laying off based on teachers’ value-added scores (most recently, one analysis of Washington State and another using data from New York City; both are worth reading).  Unsurprisingly, the simulations show that the two policies would not lay off the same teachers, and that the seniority-based layoffs would save less money for the same number of dismissals (since the least experienced teachers are paid less).  In addition, the teachers laid off based on seniority have lower average value-added scores than those laid off based on those value-added scores (as would inevitably be the case).

Based in part on these and other analyses, critics have a pretty solid argument on the surface: Seniority makes us “fire good teachers” simply because they don’t have enough experience, and we can fire fewer teachers if we use “quality” instead of seniority. 

To be clear: I think that there is a sound case for exploring alternatives to seniority-based layoffs, but many of the recent arguments for so-called “quality-based” layoffs have been so simplistic and reactionary that they may actually serve to deter serious conversations about how to change these practices.