The Legend Of Last Fall

The subject of Michelle Rhee’s teaching record has recently received a lot of attention. While the controversy has been interesting, it could also be argued that it’s relatively unimportant. The evidence that she exaggerated her teaching prowess is, after all, inconclusive (though highly suggestive). A little resume inflation from a job 20 years ago might be overlooked, so long as Rhee’s current claims about her more recent record are accurate. But are they?

On Rhee’s new website, her official bio - in effect, her resume today (or at least her cover letter) - contains a few sentences about her record as chancellor of D.C Public Schools (DCPS), under the header "Driving Unprecedented Growth in the D.C. Public Schools." There, her test-based accomplishments are characterized as follows:

Under her leadership, the worst performing school district in the country became the only major city system to see double-digit growth in both their state reading and state math scores in seventh, eighth and tenth grades over three years.

But the statement is false.

Of course, it contains the usual problems that pervade our NCLB-style misunderstanding of education statistics: The mistaken idea that you can easily compare districts in different states with different tests and different definitions of proficiency; the false concept that cohort-to-cohort changes are “growth” (you’re comparing two completely different groups of students, especially at the grade level); and the common mislabeling (and misinterpretation) of proficiency rates as “scores” (DCPS does not release its actual test score data).

Let’s put aside these common issues, and also ignore the fact that the claim in the bio seems to be comparing two different kinds of measures – absolute performance level (“worst performing district”) and increases in that performance level (“double-digit growth”) – perhaps in order to create the illusion of a “last to first” transformation.  Let’s just look quickly at each of these claims in turn.

DCPS was not the “worst performing school district in the country” when Rhee took over after the 2006-07 school year, at least not by the measure she specifies. Since she refers to a three-year period and results for seventh and tenth graders, we can infer that she is talking about state tests (in DC’s case, the DC-CAS), rather than NAEP, which is administered (math and reading) every two years (and not to seventh or tenth graders).  I would also add that NAEP only provides separate estimates for only 11 districts in 2007 (the year in which Rhee claims DCPS was the “worst performing district”), so it cannot really be used for national comparisons of all “major city systems."

A centralized collection of state test data is available from the New America Foundation, which (in a truly public service) collects proficiency rates and other data from all U.S. school districts every year. These data only allow one to examine one of the three grades that Rhee brings up in her bio - eighth grade. (Meaning that we could potentially falsify her claims, but not fully confirm them.)

Out of the 200 largest districts in 2007, there were 14 with math proficiency rates lower than those of DCPS, including “major districts” such as Denver, Oakland, Baltimore, and Los Angeles. There were five districts with lower reading proficiency rates, though the differences are negligible (all within 1-2 percent). So, in at least of one the grades that Rhee references in her claim (eighth grade) the first part of her claim is - put charitably - exaggerated.

Then there is the far more important component of her argument—that DCPS was the only large district in the nation to see double-digit “score” gains in seventh, eighth, and tenth grades. The New America Foundation’s dataset only goes up to 2009 (I assume the 2010 data are still being collected). Just by eyeballing the changes between 2007 and 2009, and confirming the 2010 rates on the state's education data website, I noticed that Baltimore (a "major city system" by any definition) also had double-digit increases in both math and reading (eighth grade) proficiency between 2007 and 2010, going from 24.0 percent to 38.9 in math, and from 43.9 to 61.5 in reading.

Now, if one manually checked all other large districts in all three grades and both subjects, one might also find a few more (though probably not many) that did the same in one of these three grades. In either case, once again, Rhee’s claim is incorrect.

Even the bio’s header - “Driving Unprecedented Growth in the D.C. Public Schools”- is blatantly misleading. She doesn't specify a test here, but since DC adopted the DC-CAS test in 2006, we cannot really compare Rhee’s tenure with previous years using DC-CAS results, as doing so would require an intra-district comparison of results from two totally different tests over time. We must instead rely on NAEP (which it might be wise to do regardless, since it is the better test and, for this comparison, we don’t need data for other districts). A quick look at NAEP scores shows that DCPS performance had been improving for years. In fact, the growth rate under Rhee was either equivalent to or lower than that of her two predecessors. There is nothing “unprecedented” about it.

Overall, I have argued that we really need to wait for a thorough, independent analysis of Rhee’s testing record, using longitudinal student-level data, before passing any judgments—and that the publicly-available data tentatively suggest that there was some progress, at least in math, but that it has been overstated (for details, please see my earlier post on this issue).

Nevertheless, one could easily argue that the proficiency rate increases under Rhee are laudable. DCPS is a largely low-income, low-performing district, and we should celebrate all of its successes. A couple of districts may have seen higher gains on their state tests, but that doesn’t detract from the accomplishment.

Yet that’s what makes Rhee’s unchecked, overstated claims so perplexing.

Her unfailing use of superlatives in describing her record – "highest performing," “the only major city system," "unprecedented growth” – smacks of unnecessarily aggressive self-promotion. Trying to improve performance in a tough district is, well...tough. Real progress is almost always slow and sustained, and school leaders need not preside over large test score gains to be considered successful. We should not expect miracles from Rhee or anyone else. We should, however, expect administrators to be obsessively careful about accurately characterizing their results, especially when they, like Rhee, are running a national campaign based on their records and reputations. On a similar note, there is quite a bit of irony in the fact that her entire approach to education is based on human capital and “data-driven” personnel policies, yet both her old and new “resumes” contain misleading information.

More fundamentally, the problem is that Michelle Rhee projects a level of intense certainty about her policy ideas that could only be characterized as simplistic or, perhaps, given the level of her influence, a little frightening. And she often justifies this certainty using grandiose claims about her own testing results. When those claims don’t quite add up, it signals that there are no certainties or simple answers when it comes to assessing and improving public education in the U.S. We don't need "rock stars." We need millions of skilled, professional musicians. I believe that Michelle Rhee understands this. I wish she would show it.