A New Idea For Test-Based Accountability In DC: Actual Test Scores

The Washington Post reports on an issue that we have discussed here on many occasions: The incompleteness of the testing results released annually by the District of Columbia Public Schools (DCPS), or, more accurately, the Office of the State Superintendent of Education (OSSE), which is responsible for testing in DC schools.

Here’s the quick backstory: For the past 7-8 years or so, DCPS/OSSE have not released a single test score for the state assessment (the DC-CAS). Instead, they have released only the percentage of students whose scores meet the designated cutoff points for the NCLB-style categories of below basic, basic, proficient and advanced. I will not reiterate all of the problems with these cutpoint-based rates and how they serve to distort the underlying data, except to say that they are by themselves among the worst ways to present these data, and there is absolutely no reason why states and districts should not release both rates and average scale scores.

The Post reports, however, that one organization -- the Broader, Bolder Approach to Education -- was able to obtain the actual scale score data (by subgroup and grade) for 2010-2013, and that this group published a memo-style report alleging that DCPS’ public presentation of their testing results over the past few years has been misleading. I had a mixed reaction to this report and the accompanying story.

On the one hand, I give a lot of credit to Broader, Bolder for doing this – for acknowledging the importance of the scale scores, for going to the trouble of requesting them, and then for publishing them in full. By so doing, this project has served to shine a light on DCPS/OSSE’s inexplicable failure to have released these data previously (and they're not alone - several other states also do not release actual scores). One can only hope that these actions will also exert pressure on D.C., as well as on other states, to do so going forward.

On the other hand, I’m concerned that this whole issue may turn into yet another bone of contention between the various factions of the education policy debate. The Broader, Bolder report is a very direct condemnation – it claims that DCPS/OSSE misled the public in how they presented trends in testing performance over the past few years by “cherry-picking” numbers and painting a “false picture of progress." The Post headline -- "Critics of D.C. education policies question test score gains" -- picks up on this narrative (note that the changes are not actually "gains," but that's a different story).

And, frankly, there is more than a little truth behind this suspicion – when it comes to presenting testing data in an overblown, misleading fashion, DCPS is among the worst offenders.

That said, I really hope this doesn’t turn into another back-and-forth between supporters and opponents of the D.C. reforms, in which both sides pick apart DC-CAS and NAEP scores and offer exaggerated misinterpretations of the data that are only moderately more appropriate than DCPS/OSSE’s own press releases. The reality is that changes between years, whether in scores or rates, are more often due to unmeasurable variations among different cohorts of test-takers than to “real” progress (and, in any case, changes in neither the rates nor the scores are actually valid school or district performance measures). As I’ve said before, there’s more to assessing test-based performance than subtraction.

Instead, the core issue here -- the sine qua non of this whole story -- is DCPS/OSSE’s failure to release the data in the first place. And this issue should not be politicized. If we, as a nation, are going to place so much faith in these test results, it is simply unacceptable to withhold any data that does not compromise privacy. One can only hope that this is a basic point on which most of us can agree, and that it does not get lost in a barrage of politically-motivated attacks and counter-attacks.

- Matt Di Carlo