Shanker Blog

A Look At The Changes To D.C.'s Teacher Evaluation System

D.C. Public Schools (DCPS) recently announced a few significant changes to its teacher evaluation system (called IMPACT), including the alteration of its test-based components, the creation of a new performance category (“developing”), and a few tweaks to the observational component (discussed below). These changes will be effective starting this year.

As with any new evaluation system, a period of adjustment and revision should be expected and encouraged (though it might be preferable if the first round of changes occurs during a phase-in period, prior to stakes becoming attached). Yet, despite all the attention given to the IMPACT system over the past few years, these new changes have not been discussed much beyond a few quick news articles.

I think that’s unfortunate: DCPS is an early adopter of the “new breed” of teacher evaluation policies being rolled out across the nation, and any adjustments to IMPACT’s design – presumably based on results and feedback – could provide valuable lessons for states and districts in earlier phases of the process.

Accordingly, I thought I would take a quick look at three of these changes.

College For All; Good Jobs For A Few?

A recent study by the Center for Policy Research (CEPR) asks the question that must be on the minds of college grads, now working as coffee shop baristas: “Where Have All the Good Jobs Gone?" The answer: swallowed by corporate profits and the personal portfolios of the ultrawealthy.

Despite the fact that the American economy has experienced “enormous” productivity gains since the late 1970’s, the study finds that the number of “good jobs” (defined as those paying at least $37,000 per year, with employer-provided health insurance and an employer-sponsored retirement plan) has declined from 27.4 percent in 1979 to 24.6 percent in 2010.  This discouraging trend was strong even before the onset of the country’s economic crisis: in 2007, the year before the onset of the recession, only 25 percent of college grads had “good jobs."

CEPR notes that the prevailing explanations for the failure to share productivity gains are “technology” and lack of necessary skills among American workers. But, if this were true, the CEPR study argues, one would expect college grads to have a higher share of good jobs than they did 30 years ago. They don’t. Instead, at every age level, today’s college grads are less likely to have a “good job” than their 1970s counterparts. This is especially surprising, the researchers note, since twice as many Americans now have advanced degrees as compared to the 1970’s.

Large Political Stones, Methodological Glass Houses

Earlier this summer, the New York City Independent Budget Office (IBO) presented findings from a longitudinal analysis of NYC student performance. That is, they followed a cohort of over 45,000 students from third grade in 2005-06 through 2009-10 (though most results are 2005-06 to 2008-09, since the state changed its definition of proficiency in 2009-10).

The IBO then simply calculated the proportion of these students who improved, declined or stayed the same in terms of the state’s cutpoint-based categories (e.g., Level 1 ["below basic" in NCLB parlance], Level 2 [basic], Level 3 [proficient], Level 4 [advanced]), with additional breakdowns by subgroup and other variables.

The short version of the results is that almost two-thirds of these students remained constant in their performance level over this time period – for instance, students who scored at Level 2 (basic) in third grade in 2006 tended to stay at that level through 2009; students at the “proficient” level remained there, and so on. About 30 percent increased a category over that time (e.g., going from Level 1 to Level 2).

The response from the NYC Department of Education (NYCDOE) was somewhat remarkable. It takes a minute to explain why, so bear with me.

Common Core Opens The Second Front In The Reading Wars

Our guest author today is Kathleen Porter-Magee, Bernard Lee Schwartz policy fellow and editor of the Common Core Watch blog at the Thomas B. Fordham Institute. Previously, Ms. Porter-Magee served as both a middle and high school teacher, as well as the curriculum and professional development director for a network of public charter schools.

Up until now, the Common Core ELA standards were considered path-breaking mostly because of their reach. This isn't the first time a group attempted to write “common” standards, but it is the first time they’ve gained such widespread traction.

Yet the Common Core standards are revolutionary for another, less talked about, reason: they define rigor in reading and literature classrooms more clearly and explicitly than nearly any of the state ELA standards they’ve replaced. Now, as the full impact of these expectations is starting to take hold, the decision to define rigor—and the way the CCSS define it—is fanning the flames of a debate that threatens to open up a whole new front in America’s long running “Reading Wars."

The first and most divisive front in the reading wars was the debate over the importance of phonics to early reading instruction. Thanks to the 2000 recommendations of the National Reading Panel and the 2001 “Reading First” portion of No Child Left Behind, the phonics camp has largely won the day in this battle. Now, while there remain curricula that may marginalize the importance of phonics and phonemic awareness, there are none that ignore it completely.

School Grades For School Grades' Sake

I have reviewed, albeit superficially, the test-based components of several states’ school rating systems (e.g., OH, FL, NYC, LA, CO), with a particular focus on the degree to which they are actually measuring student performance (how highly students score), rather than school effectiveness per se (whether students are making progress). Both types of measures have a role to play in accountability systems, even if they are often confused or conflated, resulting in widespread misinterpretation of what the final ratings actually mean, and many state systems’ failure to tailor interventions to the indicators being used.

One aspect of these systems that I rarely discuss is the possibility that the ratings systems are an end in themselves. That is, the idea that public ratings, no matter how they are constructed, provide an incentive for schools to get better. From this perspective, even if the ratings are misinterpreted or imprecise, they might still “work."*

There’s obviously something to this. After all, the central purpose of any accountability system is less about closing or intervening in a few schools than about giving all schools incentive to up their respective games. And, no matter how you feel about school rating systems, there can be little doubt that people pay attention to them. Educators and school administrators do so, not only because they fear closure or desire monetary rewards; they also take pride in what they do, and they like being recognized for it. In short, my somewhat technocratic viewpoint on school ratings ignores the fact that their purpose is less about rigorous measurement than encouraging improvement.

Literacy For Life: The Role Of Career And Technical Education In Reading Proficiency

It is well established that a student’s reading proficiency level in elementary school is a good predictor of high school graduation success. The lower the reading level, the more likely it is that the student will not graduate on time. Against this background, it is sobering that many U.S. students reach high school without the reading and comprehension skills they need. According to NAEP data, in 2011, more than a third (33 percent) of 4th-graders were reading at a below basic level; among 8th-grade and 12th grade students, the percentage of students who were stuck at the below basic reading level had dropped, but only to about 25 percent. Many of these students drop out; many go on to earn a diploma, but enter the work world singularly unprepared to earn a living.

What is to be done? Certainly, intensive remediation is part of the answer, but so are practice and motivation and interest. The challenge for struggling readers at the high school level is hard to overstate; by the time they enter high school, they often display a negative and despairing attitude toward school that has been hardened by years of failure. Furthermore, most high school teachers are not trained in literacy instruction, a specialized skill which is theoretically the purview of early elementary school. Indeed, for many urban teachers, motivating kids just to come to school is the major challenge.

How do we motivate these kids, who sometimes exhibit stubborn resistance to reading or to any other kind of schoolwork?  One effective strategy is to make the purpose of reading as interesting and obvious as possible. For many youngsters, that means access to high-quality Career and Technical Education (CTE).

The New Middle East: Democratic Accountability And The Role Of Trade Unions

Our guest author today is Heba F. El-Shazli.  She has 25 years of experience in the promotion of democracy, independent trade unions, political and economic development. She has worked with institutions and leaders throughout the Middle East and North Africa (MENA) to challenge authoritarian regimes. Currently she is a visiting professor of international studies and modern languages at the Virginia Military Institute. The views expressed here are her own.

Since the shock of 9/11 and the tragedy that ensued, many policy analysts have questioned whether or not Islam is compatible with democracy, while  ignoring countries such as Indonesia (the largest Muslim nation in the world) as well as  India, Turkey, and  others with large Muslim populations.

Now, in the aftermath of Arab Spring, Islamist political parties have gained political power through elections in the Middle East and, for many analysts, the jury is still out: Can Islamist governments be responsive to the people who elected them? Will it be one person, one vote, one time?  It appears that these questions are about to be answered:  The Justice and Development Party (AKP), which governs Turkey, has been in the forefront for many years. In Morocco, a majority of voters also handed power to the Justice and Development Party (PJD), a party inspired by Turkey's moderate Islamists. Tunisia’s Al-Nahda (Renaissance) party and its prime minister were elected to office after free and fair elections.  In Egypt, Mohammed Morsi, of the Freedom and Justice Party (FJP) established by the Muslim Brotherhood, won the Presidential elections and his new prime minister has formed a cabinet.

Against this background, the fundamental challenge to these governments in the Middle East/North Africa (MENA) region is economic and not religious. The newly-minted Islamist governments are going to be tested daily and this time held accountable by voters who are no longer afraid to speak out.

The Louisiana Voucher Accountability Sweepstakes

The situation with vouchers in Louisiana is obviously quite complicated, and there are strong opinions on both sides of the issue, but I’d like to comment quickly on the new “accountability” provision. It's a great example of how, too often, people focus on the concept of accountability and ignore how it is actually implemented in policy.

Quick and dirty background: Louisiana will be allowing students to receive vouchers (tuition to attend private schools) if their public schools are sufficiently low-performing, according to their "school performance score" (SPS). As discussed here, the SPS is based primarily on how highly students score, rather than whether they’re making progress, and thus tells you relatively little about the actual effectiveness of schools per se. For instance, the vouchers will be awarded mostly to schools serving larger proportions of disadvantaged students, even if many of those schools are compelling large gains (though such progress cannot be assessed adequately using year-to-year changes in the SPS, which, due in part to its reliance on cross-sectional proficiency rates, are extremely volatile).

Now, here's where things get really messy: In an attempt to demonstrate that they are holding the voucher-accepting private schools accountable, Louisiana officials have decided that they will make these private schools ineligible for the program if their performance is too low (after at least two years of participation in the program). That might be a good idea if the state measured school performance in a defensible manner. It doesn't.

Schools Aren't The Only Reason Test Scores Change

In all my many posts about the interpretation of state testing data, it seems that I may have failed to articulate one major implication, which is almost always ignored in the news coverage of the release of annual testing data. That is: raw, unadjusted changes in student test scores are not by themselves very good measures of schools' test-based effectiveness.

In other words, schools can have a substantial impact on performance, but student test scores also increase, decrease or remain flat for reasons that have little or nothing to do with schools. The first, most basic reason is error. There is measurement error in all test scores - for various reasons, students taking the same test twice will get different scores, even if their "knowledge" remains constant. Also, as I've discussed many times, there is extra imprecision when using cross-sectional data. Often, any changes in scores or rates, especially when they’re small in magnitude and/or based on smaller samples (e.g., individual schools), do not represent actual progress (see here and here). Finally, even when changes are "real," other factors that influence test score changes include a variety of non-schooling inputs, such as parental education levels, family's economic circumstances, parental involvement, etc. These factors don't just influence how highly students score; they are also associated with progress (that's why value-added models exist).

Thus, to the degree that test scores are a valid measure of student performance, and changes in those scores a valid measure of student learning, schools aren’t the only suitors at the dance. We should stop judging school or district performance by comparing unadjusted scores or rates between years.