Education Policy

Teachers And Professional Collaboration: How Sweden Has Become The ABBA Of Educational Change

Our guest author today is Andy Hargreaves, the Brennan Chair in Education at Boston College. He is the coauthor of Professional Capital: Transforming Teaching in Every School, which won the 2015 Grawemeyer Award for the idea in education most likely to have the most effect on practice worldwide. He is also the 2016 recipient of the Horace Mann League’s Outstanding Friend of Public Education Award. An extended version of this column originally appeared in Pedagogiska Magasinet, the Swedish teachers’ magazine in February 2016.

In the 1960s and 70s, Sweden’s economic productivity and social engineering were the envy of democrats all over the world. The nation’s comprehensive schools were an inspiration for public education reformers in the United Kingdom and many other nations too. In Sweden, market prosperity and the collective good went side by side. It was a country where, like the nations’ classic pop group, Abba, people banded and bonded together really well.

In the 90s, however, Sweden entered an age of what political scientists call free-market neo-liberalism, and educational reform was at the leading edge of it. In some ways moving ahead of the US trend, Sweden introduced large numbers of competitive “free schools”, funded with public money but no longer regulated by their school districts. Hedge fund companies were the largest single group of owners of these schools. Sweden’s society and its schools were, in the titles of two of Abba’s songs, now driven by a “Winner Takes it All” culture of “Money, Money, Money!” Between 2003 and 2012, Sweden experienced the greatest deterioration in PISA scores out of all OECD countries who were performing above average in 2003. Despite the country's proud and internationally admired egalitarian tradition, its achievement gaps have been widening faster than in any other country.

Student Sorting And Teacher Classroom Observations

Although value added and other growth models tend to be the focus of debates surrounding new teacher evaluation systems, the widely known but frequently unacknowledged reality is that most teachers don’t teach in the tested grades and subjects, and won’t even receive these test-based scores. The quality and impact of the new systems therefore will depend heavily upon the quality and impact of other measures, primarily classroom observations.

These systems have been in use for decades, and yet, until recently, relatively little is known about their properties, such as their association with student and teacher characteristics, and there are, as yet, only a handful of studies of their impact on teachers’ performance (e.g., Taylor and Tyler 2012). The Measures of Effective Teaching (MET) Project, conducted a few years ago, was a huge step forward in this area, though at the time it was perhaps underappreciated the degree to which MET’s contribution was not just in the (very important) reports it produced, but also in its having collected an extensive dataset for researchers to use going forward. A new paper, just published in Educational Evaluation and Policy Analysis, is among the many analyses that have and will use MET data to address important questions surrounding teacher evaluation.

The authors, Rachel Garrett and Matthew Steinberg, look at classroom observation scores, specifically those from Charlotte Danielson’s widely employed Framework for Teaching (FFT) protocol. These results are yet another example of how observation scores share most of the widely-cited (statistical) criticisms of value added scores, most notably their sensitivity to which students are assigned to teachers.

The IMPACT Of Teacher Turnover In DCPS

Teacher turnover has long been a flashpoint in education policy, yet these debates are rife with complications. For example, it is often implied that turnover is a “bad thing,” even though some turnover, as when low-performing teachers leave, can be beneficial, whereas some retention, as when low-performing teachers stay, can be harmful. The impact of turnover also depends heavily on other factors, such as the pool of candidates available to serve as replacements, and how disruptive turnover is to the teachers who are retained.

The recent widespread reform of teacher evaluation systems has made the turnover issue, never far below the surface, even more salient in recent years. Critics contend that the new evaluations, particularly their use of test-based productivity measures, will cause teachers to flee the profession. Supporters, on the other hand, are in a sense hoping for this outcome, as they anticipate that, under the new systems, voluntary and involuntary separations will serve to improve the quality of the teacher workforce.

A new working paper takes a close look the impact of teacher turnover under what is perhaps the most controversial teacher evaluation system in the nation – that used in the District of Columbia Public Schools (DCPS). It's a very strong analysis that speaks directly to policy in a manner that does not fit well into the tribal structure of education debates today.

New Report: Does Money Matter in Education? Second Edition

In 2012, we released a report entitled “Does Money Matter in Education?,” written by Rutgers Professor Bruce Baker. The report presented a thorough, balanced review of the rather sizable body of research on the relationship between K-12 education spending and outcomes. The motivation for this report was to address the highly contentious yet often painfully oversimplified tribal arguments regarding the impact of education spending and finance reforms, as well as provide an evidence-based guide for policymakers during a time of severe budgetary hardship. It remains our most viewed resource ever, by far.

Now, almost four years later, education spending in most states and localities is still in trouble. For example, state funding of education is lower in 2016 than it was in 2008 (prior to the recession) in 31 states (Leachman et al. 2016). Moreover, during this time, there has been a continuing effort to convince the public that how much we spend on schools doesn’t matter for outcomes, and that these spending cuts will do no harm.

As is almost always the case, the evidence on spending in education is far more nuanced and complex than our debates about it (on both “sides” of the issue). And this evidence has been building for decades, with significant advances since the release of our first “Does Money Matter?” report. For this reason, we have today released the second edition, updated by the author. The report is available here.

Evidence From A Teacher Evaluation Pilot Program In Chicago

The majority of U.S. states have adopted new teacher evaluation systems over the past 5-10 years. Although these new systems remain among the most contentious issues in education policy today, there is still only minimal evidence on their impact on student performance or other outcomes. This is largely because good research takes time.

A new article, published in the journal Education Finance and Policy, is among the handful of analyses examining the preliminary impact of teacher evaluation systems. The researchers, Matthew Steinberg and Lauren Sartain, take a look at the Excellence in Teaching Project (EITP), a pilot program carried out in Chicago Public Schools starting in the 2008-09 school year. A total of 44 elementary schools participated in EITP in the first year (cohort 1), while an additional 49 schools (cohort 2) implemented the new evaluation systems the following year (2009-10). Participating schools were randomly selected, which permits researchers to gauge the impact of the evaluations experimentally.

The results of this study are important in themselves, and they also suggest some more general points about new teacher evaluations and the building body of evidence surrounding them.

The Role Of Teacher Diversity In Improving The Academic Performance Of Students Of Color

Last month, the Albert Shanker Institute released a report on the state of teacher diversity, which garnered fair amount of press attention – see here, here, here, and here. (For a copy of the full report, see here.) This is the second of three posts, which are all drawn from a research review published in the report. The first post can be found here. Together, they help to explain why diversity in the teaching force—or lack thereof—should be  a major concern.

It has long been argued that there is a particular social and emotional benefit to children of color, and especially those children from high-poverty neighborhoods, from knowing—and being known and recognized by—people who look like themselves who are successful and in positions of authority. But there is also a growing body of evidence to suggest that students derive concrete academic benefits from having access to demographically similar teachers.

For example, in one important study, Stanford professor Thomas Dee reanalyzed test score data from Tennessee’s Project STAR class size experiment, still one of the largest U.S. studies to employ the random assignment of students and teachers. Dee found that a one-year same-race pairing of students and teachers significantly increased the math and reading test scores of both Black and White students by roughly 3 to 4 percentile points. These effects were even stronger for poor Black students in racially segregated schools (Dee, 2004).

Recent Evidence On The New Orleans School Reforms

A new study of New Orleans (NOLA) schools since Katrina, published by the Education Research Alliance (ERA), has caused a predictable stir in education circles (the results are discussed in broader strokes in this EdNext article, while the full paper is forthcoming). The study’s authors, Doug Harris and Matthew Larsen, compare testing outcomes before and after the hurricanes that hit the Gulf Coast in 2005, in districts that were affected by those storms. The basic idea, put simply, is to compare NOLA schools to those in other storm-affected districts, in order to assess the general impact of the drastic educational change undertaken in NOLA, using the other schools/districts as a kind of control group.

The results, in brief, indicate that: 1) aggregate testing results after the storms rose more quickly in NOLA vis-à-vis the comparison districts, with the difference in 2012 being equivalent to roughly 15 percentile points ; 2) there was, however, little discernible difference in the trajectories of NOLA students who returned after the storm and their peers in other storm-affected districts (though this latter group could only be followed for a short period, all of which occurred during these cohorts' middle school years). Harris and Larsen also address potential confounding factors, including population change and trauma, finding little or no evidence that these factors generate bias in their results.

The response to this study included the typical of mix of thoughtful, measured commentary and reactionary advocacy (from both “sides”). And, at this point, so much has been said and written about the study, and about New Orleans schools in general, that I am hesitant to join the chorus (I would recommend in particular this op-ed by Doug Harris, as well as his presentation at our recent event on New Orleans).

Where Al Shanker Stood: Union-District Partnerships

In this New York Times piece, which was published on March 9, 1986, Al Shanker discusses a study suggesting that union-district partnership, not confrontation, is the best way to enact and implement reforms that will improve schools.

In the last 25 years, teachers' unions have grown in size and influence. In the minds of many they represent an establishment just as much as the local board of education and the superintendent of schools. Many critics of our schools have been eager to portray teacher unions as supporters of educationally undesirable rules and procedures, such as seniority, which were borrowed from the industrial sector. They view teacher unions as fighting for these rules at any cost and using their bargaining powers to shoot down constructive change whenever it threatens to infringe on teachers' vested interests.

But an interesting new study gives us quite a different picture of the impact that teacher unions and collective bargaining have on the reform process. In preparing Teacher Unions, School Staffing and Reform, a Harvard Graduate School of Education research team led by Susan Moore Johnson analyzed 155 contracts chosen at random from a variety of school districts around the country. And, from June of 1984 to February, 1985, they did extensive, in-depth field work in 5 of the districts, where they examined documents, sat in on meetings and interviewed 187 teachers, principals, union leaders and central office administrators.

What emerges is a valuable insight into the dynamics and complexity of the reform process, why some proposals work and why others fall flat. Though new programs tend to be formulated in legislative chambers or in governors' mansions, the key to success, the authors conclude, is what happens on the district level, within the individual collective bargaining unit. And some interesting patterns emerge.

The Story Behind The Story: Social Capital And The Vista Unified School District

Our guest author today is Devin Vodicka, superintendent of Vista Unified, a California school district serving over 22,000 students that was recently accepted into the League of Innovative Schools. Dr. Vodicka participates in numerous state and national leadership groups, including the Superintendents Technical Working Group of the U.S. Education Department .

Transforming a school district is challenging and complex work, often requiring shifts in paradigms, historical perspective, and maintaining or improving performance. Here, I’d like to share how we approached change at Vista Unified School District (VUSD) and to describe the significant transformation we’ve been undergoing, driven by data, focused on relationships, and based in deep partnerships. Although Vista has been hard at work over many years, this particular chapter starts in July of 2012 when I was hired.  

When I became superintendent, the district was facing numerous challenges: Declining enrollment, financial difficulties, strained labor relations, significant turnover in the management ranks, and unresolved lawsuits were all areas in need of attention. The school board charged me and my team with transforming the district, which serves large numbers of linguistically, culturally, and economically diverse students. While there is still significant room for improvement, much has changed in the past three years, generally trending in a positive direction. Below is the story of how we did it.

The Magic Of Multiple Measures

Our guest author today is Cara Jackson, Assistant Director of Research and Evaluation at the Urban Teacher Center.

Teacher evaluation has become a contentious issue in U.S.  Some observers see the primary purpose of these reforms as the identification and removal of ineffective teachers; the popular media as well as politicians and education reform advocates have all played a role in the framing of teacher evaluation as such.  But, while removal of ineffective teachers was a criterion under Race to the Top, so too was the creation of evaluation systems to be used for teacher development and support.

I think most people would agree that teacher development and improvement should be the primary purpose, as argued here.  Some empirical evidence supports the efficacy of evaluation for this purpose (see here).  And given the sheer number of teachers we need, declining enrollment in teacher preparation programs, and the difficulty disadvantaged schools have retaining teachers, school principals are probably none too enthusiastic about dismissing teachers, as discussed here.

Of course, to achieve the ambitious goal of improving teaching practice, an evaluation system must be implemented well.  Fans of Harry Potter might remember when Dolores Umbridge from the Ministry of Magic takes over as High Inquisitor at Hogwarts and conducted “inspections” of Hogwart’s teachers in Book 5 of J.K. Rowling’s series.  These inspections pretty much demonstrate how not to approach classroom observations: she dictates the timing, fails to provide any of indication of what aspects of teaching practice she will be evaluating, interrupts lessons with pointed questions and comments, and evidently does no pre- or post-conferencing with the teachers.