The Bewildering Arguments Underlying Florida's Fight Over ELL Test Scores

The State of Florida is currently engaged in a policy tussle of sorts with the U.S. Department of Education (USED) over Florida’s accountability system. To make a long story short, last spring, Florida passed a law saying that the test scores of English language learners (ELLs) would only count toward schools’ accountability grades (and teacher evaluations) once the ELL students had been in the system for at least two years. This runs up against federal law, which requires that ELLs’ scores be counted after only one year, and USED has indicated that it’s not willing to budge on this requirement. In response, Florida is considering legal action.

This conflict might seem incredibly inane (unless you’re in one of the affected schools, of course). Beneath the surface, though, this is actually kind of an amazing story.

Put simply, Florida’s argument against USED's policy of counting ELL scores after just one year is a perfect example of the reason why most of the state's core accountability measures (not to mention those of NCLB as a whole) are so inappropriate: Because they judge schools’ performance based largely on where their students’ scores end up without paying any attention to where they start out.

Consider the comments of Miami-Dade superintendent Alberto Carvalho, who supports the “two year rule," and whose district, which serves a very large proportion of ELL students, would be affected most by the change. Superintendent Carvalho notes that to expect “that after one single year of English-language instruction, a child who does not speak English would be ready to sit for an exam and demonstrate equal proficiency to one who was born in a community that spoke English is not only irrational, it is unfair and unreasonable."

This is an excellent point. Schools and districts do not control the students assigned to them, and it is indeed unreasonable and irrational to expect that, after only one year, a school should make a student who is just learning English score proficiently on English exams or on subjects that are tested wholly in English (frankly, it’s difficult to understand how even two years could be considered reasonable).

By the same logic, however, what about a child from a disadvantaged background who is a native speaker but enters the school or the year far behind? For the purposes of school accountability, is it any more reasonable to require that that child, paraphrasing Superintendent Carvalho, “sit for an exam and demonstrate equal proficiency to a student who was born in an affluent community and who was at or above the proficiency level before they even began the school year?" Moreover, is it fair to punish schools or label them as "failing" if they happen to serve large proportions of these struggling students and fail to help most of them achieve well over year’s worth of growth, and often 2-3 years, in a single year’s time?

Yet these are exactly the terms by which Florida largely grades its schools, relying heavily on absolute performance levels (how highly students score at year’s end), rather than predominantly on growth measures, which gauge (albeit imperfectly) how much students have learned over time.*

(This is not to mention NCLB, which depends even more heavily on absolute performance levels to judge schools’ performance.)

In other words, these policies represent the same approach to which Florida now objects in the case of ELLs—instead of measuring student growth over time and/or controlling for factors that are known to influence performance, they judge schools based on whether students meet a (somewhat arbitrary) proficiency threshold, regardless of those students’ characteristics (including ELL status) or where they start out.

In this sense, Florida's (very reasonable) arguments in support of their "two year rule" are at their core arguments against the state's dominant school accountability measures.

Making things even more confusing, Florida education commissioner Pam Stewart, in recent comments about the ELL issue, defended the "two year rule" using the same assumptions as the USED policy (the "one year rule") to which the state currently objects. She claimed that Florida’s Hispanic students are among the nation's highest-scoring in national assessments," and so, in her words, it’s “impossible to understand” why the federal government would want to “micromanage” Florida’s accountability system (i.e., contest the "two year rule"), given all its purported successes.

This is perplexing on a couple of levels. For one thing, a large proportion of Florida’s Hispanic students are not ELLs (nor, of course, are all ELL students Hispanic)—meaning that “success” with one group should not be equated as “success” with the other. Second, and most importantly, once again, how highly students (or student subgroups) score on tests tells you very little about how well their schools have performed—for the very same reasons that, according to Florida, it is unfair to count ELL students’ testing results after just one year - that is, they don’t account for where these students start out. Put differently, Commissioner Stewart’s argument seems to be that Florida shouldn't judge schools by how highly their ELL students score after just one year because Florida’s Hispanic students score highly in a given year.

In short, then Florida is fighting USED on an approach to measuring school performance that is deeply flawed, but that is also the backbone of its core school accountability measures. Their education commissioner is, with justification, supporting the state's alternative policy, but is doing so using arguments that reflect the very same assumptions that undergird the policy they oppose. It's a great example of the baffling discourse and policy surrounding educational measurement in the U.S., in which laudable intentions and worthwhile goals are routinely compromised by misunderstandings of the measures employed and what they are actually telling us.

- Matt Di Carlo

*****

* As discussed here, Florida does (to its credit) employ growth-based measures, which count for 50 percent of its school grades. However, one of the measures (representing 25 percent of a school's grade, or half of the growth-based portion) is by its design largely redundant with proficiency rates. As a result, Florida's school grades are, overall, heavily reliant to absolute performance measures, and thus correlated strongly with the characteristics of the students they serve.