Do Teachers Really Come From The "Bottom Third" Of College Graduates?

** Also posted here on 'Valerie Strauss' Answer Sheet' in the Washington Post

The conventional wisdom among many education commentators is that U.S. public school teachers “come from the bottom third” of their classes. Most recently, New York City Mayor Michael Bloomberg took this talking point a step further, and asserted at a press conference last week that teachers are drawn from the bottom 20 percent of graduates.

All of this is supposed to imply that the U.S. has a serious problem with the “quality” of applicants to the profession.

Despite the ubiquity of the “bottom third” and similar arguments (which are sometimes phrased as massive generalizations, with no reference to actual proportions), it’s unclear how many of those who offer them know what specifically they refer to (e.g., GPA, SAT/ACT, college rank, etc.). This is especially important since so many of these measurable characteristics are not associated with future test-based effectiveness in the classroom, while those that are are only modestly so.

Still, given how often it is used, as well as the fact that it is always useful to understand and examine the characteristics of the teacher labor supply, it’s worth taking a quick look at where the “bottom third” claim comes from and what it might or might not mean.

Most people who put forth this assertion cite one of two sources, both from the McKinsey & Company consulting organization. The first is an influential 2007 report , which simply notes that “we are now recruiting our teachers from the bottom third of high school students going to college." The authors fail to specify how “bottom third” is defined, or whether their data refer to graduates who planned to teach versus those who actually got a job (the latter method is, of course, far preferable).

The citation for this claim is a 2007 report from the New Commission on the Skills of the American Workforce, which was issued by the National Center on Education and Economy (NCEE). The full report is not freely available online, but it turns out (thanks to the work of California teacher Larry Ferlazzo) that its source is the National Center for Education Statistics’ annual “Condition of Education” (CoE) report (2002 edition).

There don’t seem to be any breakdowns in the cited report that permit one to examine precisely how many teachers come from the “bottom third," but the CoE does include a few tables on the SAT/ACT scores of teachers who received a bachelor’s degree in 1992-93 and had actually taught by the time 1997 rolled around (and for whom such data were available).

One table lists directly the percent of teachers who scored in the top half – 40.9 percent. Using figures in a different table to very roughly ballpark the proportion of 1992-93 graduates-turned-teachers in the bottom quartile (lowest 25 percent), it would be a little under 30 percent.*

Overall, then, 1992-93 graduates who chose teaching were somewhat overrepresented in the bottom of the distribution, and underrepresented in the top. The blanket characterization of these results by McKinsey (via NCEE) – that we are “recruiting our teachers from the bottom third” – seems more than a little misleading.

The second standard source for the “bottom third” claims is more clear and well-documented. It is a subsequent McKinsey report (2010), one which doesn’t rely on questionable interpretations from indirect sources, but rather its own analysis. That report claims, “The U.S. attracts most of its teachers from the bottom two-thirds of college classes, with nearly half coming from the bottom third."

According to a footnote, these data are “derived from the U.S. Department of Education, NCES, 2001 Baccalaureate and Beyond Longitudinal Survey." The appendix to the report confirms that the “top-" and “bottom” third figures are also based on SAT/ACT scores, specifically those of 1999 graduates whose first job (at least by 2001) was teaching. The breakdown for these graduates is as follows: 23 percent came from the “top third;" 47 percent from the “bottom third;" and 29 percent from the “middle third." This presents a somewhat more negative picture than the CoE data discussed above.

Why the differences? Because these studies are looking at different groups of teachers. In the CoE data, it’s 1993 graduates who had taught by 1997 (four years later), while the data used in the second McKinsey include 1999 graduates who, in 2001 (two years later), said their first job was (or is) teaching. In other words, each set of results is based on two different cohorts of college graduates, who are also identified in different ways, at different points after graduation.

Neither sample is necessarily representative of the teacher workforce as a whole, or of prior and subsequent cohorts.

Overall, then, the blanket assertion that teachers are coming from the “bottom third” of graduates is, at best, an incomplete picture. It’s certainly true that, when the terciles are defined in terms of SAT/ACT scores, there is consistent evidence that new teachers are disproportionately represented in this group (see here and here for examples from the academic literature). But the differences are not always as large as is sometimes suggested. They vary by year and sample identification, as well as by other variables, such as school-level characteristics (e.g., poorer schools) and teacher characteristics such as race and gender. (And, by the way, their relative standing as graduates is based on tests that most took in high school.)

Finally, it's very important to note that the “bottom-" and “top third” may not be a particularly good conceptualization of either the problem or the solution. The connection between SAT/ACT scores and future (test-based) effectiveness, though it’s among the only associations strong enough to be discerned statistically, is still a highly imprecise predictor of "quality." There's some useful information there, but it’s unclear whether it merits the high-profile attention it receives.**

There's no question that we should try and get as many highly-qualified applicants as possible into the classroom (though, again, how to do so is an open question). But we should also be very careful about oversimplifying the issue of aggregate teacher quality – and making sweeping statements about their qualifications based on a limited body of evidence – for the sake of intuitive, easy-to-understand talking points.

- Matt Di Carlo

*****

* There are differences in these outcomes by school and teacher characteristics. Most notably, the percentage of teachers scoring in the top half is very low in schools with the highest proportions of low-income students.

** One important question about the meaning of all these estimates is how the distribution for U.S. teachers stacks up against other professions (there are a few comparisons in the appendix of the second McKinsey report) and, perhaps, teachers in other nations (which is discussed in both McKinsey reports). But the inter-occupational comparisons would also have to be interpreted cautiously, since it's likely that the association between SAT/ACT scores and future productivity varies by job.