Shanker Blog

Getting Serious About Measuring Collaborative Teacher Practice

Our guest author today is Nathan D. Jones, an assistant professor of special education at Boston University. His research focuses on teacher quality, teacher development, and school improvement. Dr. Jones previously worked as a middle school special education teacher in the Mississippi Delta. In this column, he introduces a new Albert Shanker Institute publication, which was written with colleagues Elizabeth Bettini and Mary Brownell.

The current policy landscape presents a dilemma. Teacher evaluation has dominated recent state and local reform efforts, resulting in broad changes in teacher evaluation systems nationwide. The reforms have spawned countless research studies on whether emerging evaluation systems use measures that are reliable and valid, whether they result in changes in how teachers are rated, what happens to teachers who receive particularly high or low ratings, and whether the net results of these changes have had an effect on student learning.

At the same time,  there has been increasing enthusiasm about the promise of teacher collaboration (see here and here), spurred in part by new empirical evidence linking teacher collaboration to student outcomes (see Goddard et al., 2007; Ronfeldt, 2015; Sun, Grissom, & Loeb, 2016). When teachers work together, such as when they jointly analyze student achievement data (Gallimore et al., 2009; Saunders, Gollenberg, & Gallimore, 2009) or when high-performing teachers are matched with low-performing peers (Papay, Taylor, Tyler, & Laski, 2016), students have shown substantially better growth on standardized tests.

This new work adds to a long line of descriptive research on the importance of colleagues and other social aspects of the school organization.  Research has documented that informal relationships with colleagues play an important role in promoting positive teacher outcomes, such as planned and actual retention decisions (e.g., Bryk & Schneider, 2002; Pogodzisnki, Youngs, & Frank, 2013; Youngs, Pogodzinski, Grogan, & Perrone, 2015). Further, a number of initiatives aimed at improving teacher learning – e.g., professional learning communities (Giles & Hargreaves, 2006) and lesson study (Lewis, Perry, & Murrata, 2006) – rely on teachers planning instruction collaboratively.

The Hidden Power Of Our Social Worlds

In July 2014 the Albert Shanker Institute began a blog series on the “social side” of education reform. The collection, which includes contributions from established and emerging scholars, attempts to shine a light on new research arguing for the centrality of the social dimension in educational improvement. This blog post serves as the preface of a new ASI publication featuring six of the most important blog posts from this series. The publication is now available for download here. ASI is holding a research and policy conference on this theme Friday April 8th.

Whatever level of teacher human capital schools acquire through hiring can subsequently be developed through formal and informal professional interactions. As teachers join together to solve problems and learn from one another, the school’s instructional capacity becomes greater than the sum of its parts.

This quote from Harvard professor Susan Moore Johnson (p. 20 of this volume) may make perfect sense to you. Our systems and organizations, however, are largely structured around individualistic values. As such, a primary goal is to optimize and reward performance at the individual level. So, while some of us (perhaps many of us) might agree that a team’s capacity can exceed the sum of individual members’ capacity, we generally have a difficult time translating that knowledge into action – e.g., rewarding individual behaviors that enhance team dynamics. Part of the problem is that there’s still a lot to learn about how teamwork and collaboration are properly measured.

No matter how challenging, understanding the social dynamics that underpin our work organizations seems particularly timely given the interdependent nature of the modern workplace. According to a recent Harvard Business Review article, “time spent by managers and employees in collaborative activities has ballooned by 50 percent or more” over the past two decades. At many companies, the article notes, “more than three quarters of an employee’s day is spent communicating with colleagues.”

Caring School Leadership

Our guest authors today are Mark A. Smylie, professor emeritus at the University of Illinois-Chicago, Joseph Murphy, professor at Peabody College, Vanderbilt University, and Karen Seashore Louis, professor at the University of Minnesota.  Their research concerns school organization, leadership, and improvement.  This blog post is based on an article titled “Caring School Leadership: A Multi-Disciplinary, Cross-Occupational Model” which will be published later this year in the American Journal of Education.

From our years of studying school leadership and reform, working with practicing educators, and participating in education policy development, we have come to the conclusion that caring lies at the heart of effective schooling and good school leadership.  In this time of intense academic pressures, accountability policies, and top-down approaches to reform, however, the concept of caring has been neglected, overshadowed by attention to more “objective”, task-oriented aspects of school organization and leadership (Cassidy & Bates, 2005; Richert, 1994 (pp.109-118); Rooney, 2015).  This, we contend, is a serious problem for both students and teachers.

In this blog, we share some of our recent thinking about what caring school leadership is and why it is important. We draw on empirical and theoretical literatures from education and from disciplines outside education, particularly research on human service occupations such as health care, social services, and the ministry. And we present a model of caring school leadership. Our ideas were developed with principals in mind, but they apply to any educator engaged in school leadership work. We focus on students as the primary beneficiaries of caring. It should be noted that, as we argue for the importance of caring in schools, we do not mean to diminish the importance of academic achievement nor the need to care for staff and the community. We consider managing mutually-reinforcing combinations of caring support and academic press a central function of school leadership.

Collaboration Is The Way We Work, Not An "Activity"

Our guest author today is Joseph Vincente, 10th Grade Chemistry Science Team Leader at the East Side Community High School in New York City. East Side is one of a growing network of 38 NY public high schools (mostly in NYC) with waivers that replace standardized state tests with performance based assessment. Vincente is interested in educational technology, sustainability education, and empowering young women and students of color to pursue STEM careers.

So, 300 homework assignments checked, 200 email replied to, 100 quizzes graded, 50 more lab reports left from Monday still to read, 30 lessons executed, 10 revised notebook entries re-graded, 5 phone calls home and texts made to check-in with parents, 4 curriculum maps revised, 3 extra help sessions held before school, during lunch, and after school, 2 college bound pep-talks made, and 1 mediation between quarreling best friends conducted.

Phew. 

I take a deep breath and do a bit of mindless silent cleaning and organizing in my classroom to decompress. Another exhausting week in the life of a high school teacher comes to a close. Must be time for the weekend, right?  Well, almost... Friday afternoon at my school is when we do some of our most demanding but essential work as teachers.  You may be thinking it’s time for the dreaded weekly PD meetings or for some “collaboration”. Yes, that’s right; but, at East Side collaboration isn’t just an activity or behaving in a friendly, respectful, or cooperative way toward colleagues. Rather, collaboration underpins how we structure and conduct most of our work, how we serve students, and how we learn and grow as professionals. In the next few paragraphs, I describe some of East Side’s collaborative structures as well as the norms and conditions that support them.

Evaluating The Results Of New Teacher Evaluation Systems

A new working paper by researchers Matthew Kraft and Allison Gilmour presents a useful summary of teacher evaluation results in 19 states, all of which designed and implemented new evaluation systems at some point over the past five years. As with previous evaluation results, the headline result of this paper is that only a small proportion of teachers (2-5 percent) were given the low, “below proficiency” ratings under the new systems, and the vast majority of teachers continue to be rated as satisfactory or better.

Kraft and Gilmour present their results in the context of the “Widget Effect,” a well-known 2009 report by the New Teacher Project showing that the overwhelming majority of teachers in the 12 districts for which they had data received “satisfactory” ratings. The more recent results from Kraft and Gilmour indicate that this hasn’t changed much due to the adoption of new evaluation systems, or, at least, not enough to satisfy some policymakers and commentators who read the paper.

The paper also presents a set of findings from surveys of and interviews with observers (e.g., principals). These are in many respects more interesting and important results from a research and policy perspective, but let’s nevertheless focus a bit on the findings on the distribution of teachers across rating categories, as they caused a bit of a stir. I have several comments to make about them, but will concentrate on three in particular (all of which, by the way, pertain not to the paper’s discussion, which is cautious and thorough, but rather to some of the reaction to it in our education policy discourse).

Where Al Shanker Stood: The Problem of Student Mobility

In this New York Times piece, published on January 13, 1991, Al Shanker discusses the persistent problem of student mobility, how it disrupts children's lives and educational prospects, and what schools and school systems might do to help.

Once upon a time, people talked about student achievement in terms of the kids' responsibility for what they leaned. Some youngsters were smart, and others were dummies. Some worked hard; some were lazy. Nowadays, we've discarded these crude yardsticks because we understand that many things can influence a child's success in school. But we've substituted something just as crude. I mean the notion of accountability that makes schools totally responsible for student learning.

Of course, people have the right—the responsibility—to find out whether schools are doing a good job. And they have the right to call for the changes that are needed. But people should also understand that schools face some big problems over which they have no control.

Take the problem of student mobility, especially among poor children in urban school systems. Every year between September and June, an enormous number of students transfer in and out of these schools, often because their families are in a state of collapse or because they've lost their current housing and have to find somewhere else to live. A recent Wall Street Journal article (November 14, 1990) about the Rochester, NY, schools says that in 1987 annual student mobility—that is, the number of student transfers in relation to the entire student population—reached 64 percent. In one elementary school, it was 100 percent. And if this is true in Rochester, there's no question that something like it goes on in other urban school systems. What does it mean for teaching and learning in these schools?

Improving Teaching Through Collaboration

Our guest author today is Matthew Ronfeldt, Assistant Professor at the University of Michigan School of Education. Ronfeldt seeks to understand how to improve teaching quality, particularly in schools and districts that serve historically marginalized student populations. His research sits at the intersection of educational practice and policy and focuses on teacher preparation, teacher retention, teacher induction, and the assessment of teachers and preparation programs.

Learning to teach is an ongoing process. To be successful, then, schools must promote not only student learning but also teacher learning across their careers.* Embracing this notion, policymakers have called for the creation of school-based professional learning communities, including organizational structures that promote regular opportunities for teachers to collaborate with teams of colleagues** – also here and here. As the use of instructional teams becomes increasingly common, it is important to examine whether and how collaboration actually improves teaching and learning. The growing evidence, summarized below, suggests that it does. 

For many decades, educational scholars have conducted qualitative case studies documenting the nature of collaboration among particular groups of teachers working together in departmental teams, reading groups, and other types of instructional teams. This body of work has demonstrated that the kinds and content of collaboration vary substantially across contexts, has shed light on the norms and structures that promote more promising collaboration, and has set the stage for today’s policy focus on “professional learning communities.” However, these studies rarely connected collaboration to teachers’ classroom performance. Thus, they provided little information on whether teachers actually got better at teaching as a result of their participation in collaboration.

Teachers And Professional Collaboration: How Sweden Has Become The ABBA Of Educational Change

Our guest author today is Andy Hargreaves, the Brennan Chair in Education at Boston College. He is the coauthor of Professional Capital: Transforming Teaching in Every School, which won the 2015 Grawemeyer Award for the idea in education most likely to have the most effect on practice worldwide. He is also the 2016 recipient of the Horace Mann League’s Outstanding Friend of Public Education Award. An extended version of this column originally appeared in Pedagogiska Magasinet, the Swedish teachers’ magazine in February 2016.

In the 1960s and 70s, Sweden’s economic productivity and social engineering were the envy of democrats all over the world. The nation’s comprehensive schools were an inspiration for public education reformers in the United Kingdom and many other nations too. In Sweden, market prosperity and the collective good went side by side. It was a country where, like the nations’ classic pop group, Abba, people banded and bonded together really well.

In the 90s, however, Sweden entered an age of what political scientists call free-market neo-liberalism, and educational reform was at the leading edge of it. In some ways moving ahead of the US trend, Sweden introduced large numbers of competitive “free schools”, funded with public money but no longer regulated by their school districts. Hedge fund companies were the largest single group of owners of these schools. Sweden’s society and its schools were, in the titles of two of Abba’s songs, now driven by a “Winner Takes it All” culture of “Money, Money, Money!” Between 2003 and 2012, Sweden experienced the greatest deterioration in PISA scores out of all OECD countries who were performing above average in 2003. Despite the country's proud and internationally admired egalitarian tradition, its achievement gaps have been widening faster than in any other country.

Student Sorting And Teacher Classroom Observations

Although value added and other growth models tend to be the focus of debates surrounding new teacher evaluation systems, the widely known but frequently unacknowledged reality is that most teachers don’t teach in the tested grades and subjects, and won’t even receive these test-based scores. The quality and impact of the new systems therefore will depend heavily upon the quality and impact of other measures, primarily classroom observations.

These systems have been in use for decades, and yet, until recently, relatively little is known about their properties, such as their association with student and teacher characteristics, and there are, as yet, only a handful of studies of their impact on teachers’ performance (e.g., Taylor and Tyler 2012). The Measures of Effective Teaching (MET) Project, conducted a few years ago, was a huge step forward in this area, though at the time it was perhaps underappreciated the degree to which MET’s contribution was not just in the (very important) reports it produced, but also in its having collected an extensive dataset for researchers to use going forward. A new paper, just published in Educational Evaluation and Policy Analysis, is among the many analyses that have and will use MET data to address important questions surrounding teacher evaluation.

The authors, Rachel Garrett and Matthew Steinberg, look at classroom observation scores, specifically those from Charlotte Danielson’s widely employed Framework for Teaching (FFT) protocol. These results are yet another example of how observation scores share most of the widely-cited (statistical) criticisms of value added scores, most notably their sensitivity to which students are assigned to teachers.

Beyond Teacher Quality

Beyond PD: Teacher Professional Learning in High-Performing Systems is a recent report from the Learning First Alliance and the International Center for Benchmarking in Education at the National Center for Education and the Economy. The paper describes practices and policies from four high-performing school systems – British Columbia, Hong Kong, Shanghai, and Singapore – where professional learning is believed to be the primary vehicle for school improvement.

My first reaction was: This sounds great, but where is the ubiquitous discussion of “teacher quality?” Frankly, I was somewhat baffled that a report on school improvement never even mentioned the phrase.* Upon close reading, I found the report to be full of radical (and very good) ideas. It’s not that the report proposed anything that would require an overhaul of the U.S. education system; rather, they were groundbreaking because these ideas did not rely on the typical assumptions about how the youth or the adults in these systems learn and achieve mastery. Because, while things are changing a bit in the U.S. with regard to our understanding of student learning – e.g., we now talk about “deep learning” – we have still not made this transition when it comes to teachers.

In the U.S., a number of unstated but common assumptions about “teacher quality” suffuse the entire school improvement conversation. As researchers have noted (see here and here), instructional effectiveness is implicitly viewed as an attribute of individuals, a quality that exists in a sort of vacuum (or independent of the context of teachers’ work), and which, as a result, teachers can carry with them, across and between schools. Effectiveness also is often perceived as fairly stable: teachers learn their craft within the first few years in the classroom and then plateau,** but, at the end of the day, some teachers have what it takes and others just don’t. So, the general assumption is that a “good teacher” will be effective under any conditions, and the quality of a given school is determined by how many individual “good teachers” it has acquired.