The Allure Of Teacher Quality

Those following education know that policy focused on "teacher quality" is by far the dominant paradigm for improving schools over the past few years. Some (but not nearly all) components of this all-hands-on-deck effort are perplexing to many teachers, and have generated quite a bit of pushback. No matter one’s opinion of this approach, however, what drives it is the tantalizing allure of variation in teacher quality.

Fueled by the ever-increasing availability of detailed test score datasets linking teachers to students, the research literature on teachers’ test-based effectiveness has grown rapidly, in both size and sophistication. Analysis after analysis finds that, all else being equal, the variation in teachers’ estimated effects on students' test growth – the difference between the “top” and “bottom” teachers – is very large. In any given year, some teachers’ students make huge progress, others’ very little. Even if part of this estimated variation is attributable to confounding factors, the discrepancies are still larger than most any other measured "input" within the jurisdiction of education policy. The underlying assumption here is that “true” teacher quality varies to a degree that is at least somewhat comparable in magnitude to the spread of the test-based estimates.

Perhaps that's the case, but it does not, by itself, help much. The key question is whether and how we can measure teacher performance at the individual level and, more importantly, influence the distribution – that is, to raise the ceiling, the middle and/or the floor. The variation hangs out there like a drug to which we’re addicted, but haven’t really figured out how to administer. If there was some way to harness it efficiently, the potential benefits could be considerable. The focus of current education policy is in large part an effort to do anything and everything to try and figure this out. And, as might be expected given the enormity of the task, progress has been slow.

The most oft-discussed form of this effort is designing new evaluation systems, which represents an attempt to measure “true” teacher quality in a valid, reliable manner. If one can measure teacher performance, then one is, of course, in a much better position to try to improve it.

The seductive variation in teacher performance, coupled with the inadequacy of current evaluation systems in many places, has compelled some (but not all) states and districts to rush ahead with this process without field testing, and to begin using their new evaluations in high-stakes decisions, including termination. There is, as yet, little indication as to whether these new systems (and, more importantly, how the results are used) will improve outcomes. There are plenty of strong opinions either way.

But evaluations and the decisions to which the results will be tied are really only one front in this all-out campaign, even if they are the element that gets the most attention. The assumed variation in teacher “quality” is arguably the primary motivation for any number of additional efforts, including but not limited to:

Measuring the efficacy of teacher training programs, and holding them accountable for it;
Altering compensation structures to attract/retain more qualified candidates;
Videotaping lessons to see if there are practices common among more effective teachers;
Expanding alternative certification routes;
Experimenting with systems for collecting and distributing student data to inform instruction;
Weakening of teacher job protections;
Stepping up processes to screen applicants to open teaching positions;
Limiting teacher sick days;
Changing teacher layoff criteria;
Trying programs for new teachers, such as mentoring and induction;

Some of these projects have been around a long time, others are relatively new. Some have broad support while others can be rather controversial.

But they all share a common foundational purpose – to try anything to harness some portion of the variation in teacher quality, which appears so large that even small improvements, though very difficult to bring about, might yield substantial benefits.

These interventions have thus far failed to garner much evidence (in some cases because the policies are relatively new). No doubt there will be successes and failures, though we'll never know unless we evaluate these programs. The reasonable expectation is that even a set of well-designed and implemented policies (which is no guarantee) would produce small, gradual improvements in the distribution of teacher quality over a period of years and decades.

In the meantime, the allure persists, unwavering.

- Matt Di Carlo

Blog Topics

Matt, I think your list has neglected the most powerful strategy for improving the quality of teachers--developing the capacity of school-site teams to continually improve instruction. There is quite a bit of research showing that teacher conversation around instruction trumps every other approach(even more powerful than the influence of socio-economics). For example, See http://www.ssireview.org/articles/entry/the_missing_link_in_school_refo…. Also impressive are the results from instructional collaboration in successful districts and states such as Mass., Ontario, Canada, Long Beach, Sanger, and the Aspire charter network and from high-flyers internationally. In all of these places, developing successful school teams focused on improving instruction was the hall-mark of improvement efforts and such approaches offer effect size gains many multiples of any strategy you listed.

One question you do not pose, but would seem quite worthy of multiple posts, has to do with the entire list of approaches. Why is the US pursuing strategies to improve teacher quality in ways vastly different than all of the countries with greater international performance than the US? If policymakers are going to fixate on international test scores, perhaps they should pay attention to what other countries have proved already works. It seems the US is trying mightily to show that a free-market approach would work when none of the top-performing take such an approach. Having just listened to multiple spokespeople from the US Department of Education fail to provide and research evidence for the approach they are taking, one has to wonder what it would take for the US to seriously consider trying approaches that have actually been shown to be effective.

I think the field (and especially policy makers and advocacy groups) are in the middle of committing a huge fundamental attribution error. Too many people are framing the issue in terms of teacher quality, when it should be about TEACHING quality. Yes, the two are related but they are not the same. Too many policy discussions and research studies treat teaching as a black box and assume that it is enough to ensure that every student has a “good teacher,” as if mere exposure to a good teacher--placing students physically in the presence of a teacher and absorbing knowledge like light rays--is what brings about learning. Good teaching is not simply a function of good teachers, and the practices that teachers employ are not merely a function of their individual knowledge, skills, and experiences. It is also a function of the "technology" of teaching that we employ (curriculum, pedagogical models, teaching methods, instructional materials and tools), and the organizational structures and social resources in which teachers' work is embedded. One of the barriers to improving US education is the individualistic culture, and the heroic images of teaching in our culture. More than many other cultures, Americans believe that good teachers are born not made, that they are function of certain personal characteristics rather than the product of training. You either got the teaching knack or you don't. And also, it's rooted in American anti-intellectualism that sees teaching as easy work ("Those who can do, those who can't teach.")

As the other Ed pointed out, other countries have done a better job of improving teacher quality by using training to reduce the variability in TEACHING PRACTICE. Yes, many of them have also recruited more accomplished college graduates into the profession by making teaching a higher status occupation, but they also invest (as other professions in the US do) more in ensuring that all teachers are rigorously trained to enact a set of pedagogical practices. In other words, they reduce variability not through selection but through training. Well, and also having national curricula helps, too.

I agree with Ed Liu's comment re fundamental attribution error and variability in teaching vs teacher - there is a BIG difference that policy-makers aren't recognizing.

We should be re-evaluating the TEACHING PROFESSION, not teachers. The teaching profession, as others have noted, is extremely weak - not a profession in the true sense as it has little authority, expertise, or ability to peer-review its members.

Because of this, TEACHERS themselves are weak in much the same way that disadvantaged students are under-performers: both are products of a weak system. SO, isn't the solution to reevaluate the profession itself - how it screens, regulates, develops, and supports its members? The focus should be on the profession, not the teachers.

http://TheEducatedSociety.com/putting-the-focus-on-the-profession-not-t…