Shanker Blog

On Teacher Evaluations, Between Myth And Fact Lies Truth

Controversial proposals for new teacher evaluation systems have generated a tremendous amount of misinformation. It has come from both “sides," ranging from minor misunderstandings to gross inaccuracies. Ostensibly to address some of these misconceptions, the advocacy group Students First (SF) recently released a "myth/fact sheet” on evaluations.

Despite the need for oversimplification inherent in “myth/fact” sheets, the genre can be useful, especially about topics such as evaluation, about which there is much confusion. When advocacy groups produce them, however, the myths and facts sometimes take the form of “arguments we don’t like versus arguments we do like."

This SF document falls into that trap. In fact, several of its claims are a little shocking. I would still like to discuss the sheet, not because I enjoy picking apart the work of others (I don’t), but rather because I think elements of both the “myths” and “facts” in this sheet could be recast as "dual myths” in a new sheet. That is, this document helps to illustrate how, in many of our most heated education debates, the polar opposite viewpoints that receive the most attention are often both incorrect, or at least severely overstated, and usually serve to preclude more productive, nuanced discussions.

Let’s take all four of SF’s “myth/fact” combinations in turn.

Education Under Attack In Bahrain

Our guest author today is Jalila Al-Salman, a Bahraini teacher and vice president of the Bahrain Teachers' Association (BTA). A leader in the Bahraini uprisings of Arab Spring, she was arrested, held in prison under abusive conditions, tortured and sentenced to three years in prison. Released late in 2012, Ms. Al-Salman continues to advocate for a peaceful, democratic transition in Bahrain.

Since the outbreak of protests in Bahrain in February 2011, people there have faced varied and numerous forms of oppression by the Government of Bahrain. Peaceful protesters have been arrested and beaten, detainees have been tortured, public and private sector employees have been wrongfully terminated from their positions for participating in protests, and over 100 people have been killed.

Shortly after the uprising began, the Bahrain Teachers’ Association (BTA), of which I am the vice president, participated in a three-day strike from February 20 to 23. Up until then, we had been escalating our calls for improvements to the education system, so it seemed like an appropriate time to make our voices heard. In addition to calls for political reform, educators expressed frustration with the Bahrain government’s policy of hiring foreign educators from Egypt, Tunisia, Jordan and Saudi Arabia - a practice that limits opportunities for domestic educators to teach in Bahrain’s schools. Following the strike, an estimated 9,000 teachers marched to the Pearl Roundabout, a traffic circle in the heart of Manama that served as the epicenter of the 2011 protests. It was the largest protest by educators in Bahrain’s history.

A Miracle In The Mid-Atlantic

The National Assessment of Educational Progress, or NAEP, is often called “the nation’s report card." Advocates sometimes use the results of the main NAEP tests to argue for their policy agendas. One example (among many) is when supporters of the so-called “Florida Formula," a battery of market-based reforms that went into effect throughout the late 1990s and early 2000s and are currently spreading throughout the nation, make their case primarily on Florida’s NAEP performance. The basic logic is that average test scores increased during this time period, and thus whatever Florida did must have worked, and should be exported en masse to other states.

However, since 2003, the state that has brought home the most proverbial A’s on the nation’s report card is not Florida. It is Maryland. In fact, the increase in math and reading performance among fourth and eighth graders in the state’s public schools can only be called the “Maryland Miracle."

There's too much at stake for policy makers to waste time worrying about trivial details, such as policy analysis. Whatever laws and practices were in place in Maryland during this time period are clearly responsible for the Miracle. They should serve as a model for the nation.

Greetings From Due Diligence, New Jersey

Earlier this week, New Jersey Governor Chris Christie announced that the state will assume control over Camden City School District. Camden will be the fourth NJ district to undergo takeover, though this is the first time that the state will be removing control from an elected local school board, which will now serve in an advisory role (and have three additional members appointed by the Governor). Over the next few weeks, NJ officials will choose a new superintendent, and begin to revamp evaluations, curricula and other core policies.

Accompanying the announcement, the Governor’s office released a two-page "fact sheet," much of which is devoted to justifying this move to the public.

Before discussing it, let’s be clear about something - it may indeed be the case that Camden schools are so critically low-performing and/or dysfunctional as to warrant drastic intervention. Moreover, it's at least possible that state takeover is the appropriate type of intervention to help these schools improve (though the research on this latter score is, to be charitable, undeveloped).

That said, the "fact sheet" presents relatively little valid evidence regarding the academic performance of Camden schools. Given the sheer magnitude of any takeover decision, it is crucial for the state to demonstrate publicly that they have left no stone unturned by presenting a case that is as comprehensive and compelling as possible. However, the discrepancy between that high bar and NJ's evidence, at least that pertaining to academic outcomes, is more than a little disconcerting.

How Many Schools Don't Have Nurses?

In 2006, the National Association of School Nurses (NASN) commissioned the largest and, to my knowledge, most recent national survey on the availability of nursing services in U.S. public schools. It was administered to a sample of over 1,000 schools in all 50 states and D.C.

The primary purpose was to gather basic information on the health staff in these schools, as well as a few core characteristics, such as school size and student demographics.

I must confess that I was a little surprised by the results. Here is the distribution of schools by nursing availability, summarized very briefly (these proportions vary by school size, type and other characteristics):

The Ineffective Rating Fetish

In a story for Education Week, always reliable Stephen Sawchuk reports on what may be a trend in states’ first results from their new teacher evaluation systems: The ratings are skewed toward the top.

For example, the article notes that, in Michigan, Florida and Georgia, a high proportion of teachers (more than 90 percent) received one of the two top ratings (out of four or five). This has led to some grumbling among advocates and others, citing similarities between these results and those of the old systems, in which the vast majority of teachers were rated “satisfactory," and very few were found to be “unsatisfactory."

Differentiation is very important in teacher evaluations – it’s kind of the whole point. Thus, it’s a problem when ratings are too heavily concentrated toward one end of the distribution. However, as Aaron Pallas points out, these important conversations about evaluation results sometimes seem less focused on good measurement or even the spread of teachers across categories than on the narrower question of how many teachers end up with the lowest rating - i.e., how many teachers will be fired.

Poor Implementation Undermines Promise Of The Common Core

** Reprinted here in the Washington Post

Our guest author today is Stephen Lazar, a founding teacher at Harvest Collegiate High School in New York City, where he teaches Social Studies. A National Board certified teacher, he blogs at Outside the Cave. Stephen is also one of the organizers of Insightful Social Studies, a grass roots campaign of teachers to reform the newly proposed New York State Social Studies standards.

The Common Core Learning Standards (CCLS) seek to define “college and career readiness expectations." Forty-five states have adopted them, and are moving briskly towards full implementation in the coming year. Last January, I wrote that the standards “represent the greatest opportunity for history teaching and learning to be widely re-imagined since the Committee of Ten set the basic outlines for American education over a hundred years ago."

While I stand by that statement, with each step towards implementation I see the opportunity being squandered. We cannot possibly continue to move solely in the direction of “college and career readiness” in History & Social Studies education without ensuring that “civic” readiness is valued equally.  Additionally, we need to ensure that as states write new curricula, that they contain the proper balance of content, skills, and understandings.  New curricula will need to ensure students use an inquiry-based approach to go in depth with a smaller amount of content to gain the wider breadth of skills and dispositions required for civic, college, and career readiness.

A Controversial Consensus On KIPP Charter Schools

A recent Mathematica report on the performance of KIPP charter schools expands and elaborates on their prior analyses of these schools' (estimated) effects on average test scores and other outcomes (also here). These findings are important and interesting, and were covered extensively elsewhere.

As is usually the case with KIPP, the results stirred the full spectrum of reactions. To over-generalize a bit, critics sometimes seem unwilling to acknowledge that KIPP's results are real no matter how well-documented they might be, whereas some proponents are quick to use KIPP to proclaim a triumph for the charter movement, one that can justify the expansion of charter sectors nationwide.

Despite all this controversy, there may be more opportunity for agreement here than meets the eye. So, let’s try to lay out a few reasonable conclusions and see if we might find some of that common ground.

Causality Rules Everything Around Me

In a Slate article published last October, Daniel Engber bemoans the frequently shallow use of the classic warning that “correlation does not imply causation." Mr. Engber argues that the correlation/causation distinction has become so overused in online comments sections and other public fora as to hinder real debate. He also posits that correlation does not mean causation, but “it sure as hell provides a hint," and can “set us down the path toward thinking through the workings of reality."

Correlations are extremely useful, in fact essential, for guiding all kinds of inquiry. And Engber is no doubt correct that the argument is overused in public debates, often in lieu of more substantive comments. But let’s also be clear about something – careless causal inferences likely do more damage to the quality and substance of policy debates on any given day than the misuse of the correlation/causation argument does over the course of months or even years.

We see this in education constantly. For example, mayors and superintendents often claim credit for marginal increases in testing results that coincide with their holding office. The causal leaps here are pretty stunning.

Data Driving: At The Intersection Of Arbitrary And Meaningful

In his State of the City address last month, New York City Mayor Michael Bloomberg made some brief comments about the upcoming adoption of new assessments aligned with the Common Core State Standards (CCSS), including the following statement:

But no matter where the definition of proficiency is arbitrarily set on the new tests, I expect that our students’ progress will continue outpacing the rest of the State’s[,] the only meaningful measurement of progress we have.

On the surface, this may seem like just a little bit of healthy bravado. But there are a few things about this single sentence that struck me, and it also helps to illustrate an important point about the relationship between standards and testing results.