• Income And Educational Outcomes

    The role of poverty in shaping educational outcomes is one of the most common debates going on today. It can also be one of the most shallow.

    The debate tends to focus on income. For example (and I’m generalizing a bit here), one “side” argues that income and test scores are strongly correlated; the other “side” points to the fact that many low-income students do very well and cautions against making excuses for schools’ failure to help poor kids.

    Both arguments have merit, but it bears quickly mentioning that the focus on the relationship between income and achievement is a rather crude conceptualization of the importance of family background (and non-schooling factors in general) for education outcomes. Income is probably among the best widely available proxies for these factors, insofar as it is correlated with many of the conditions that can hinder learning, especially during a child’s earliest years. This includes (but is not at all limited to): peer effects; parental education; access to print and background knowledge; parental involvement; family stressors; access to healthcare; and, of course, the quality of neighborhood schools and their teachers.

    And that is why, when researchers try to examine school performance – while holding constant the effect of factors outside of schools’ control – income or some kind of income-based proxy (usually free/reduced price lunch) can be a useful variable. It is, however, quite limited.

  • Trial And Error Is Fine, So Long As You Know The Difference

    It’s fair to say that improved teacher evaluation is the cornerstone of most current education reform efforts. Although very few people have disagreed on the need to design and implement new evaluation systems, there has been a great deal of disagreement over how best to do so – specifically with regard to the incorporation of test-based measures of teacher productivity (i.e., value-added and other growth model estimates).

    The use of these measures has become a polarizing issue. Opponents tend to adamantly object to any degree of incorporation, while many proponents do not consider new evaluations meaningful unless they include test-based measures as a major element (say, at least 40-50 percent). Despite the air of certainty on both sides, this debate has mostly been proceeding based on speculation. The new evaluations are just getting up and running, and there is virtually no evidence as to their effects under actual high-stakes implementation.

    For my part, I’ve said many times that I'm receptive to trying value-added as a component in evaluations (see here and here), though I disagree strongly with the details of how it’s being done in most places. But there’s nothing necessarily wrong with divergent opinions over an untested policy intervention, or with trying one. There is, however, something wrong with fully implementing such a policy without adequate field testing, or at least ensuring that the costs and effects will be carefully evaluated post-implementation. To date, virtually no states/districts of which I'm aware have mandated large-scale, independent evaluations of their new systems.*

    If this is indeed the case, the breathless, speculative debate happening now will only continue in perpetuity.

  • Beyond Anecdotes: The Evidence About Financial Incentives And Teacher Retention

    ** Also posted here on "Valerie Strauss' Answer Sheet" in the Washington Post

    Our guest author today is Eleanor Fulbeck, who earned her Ph.D. in education policy from the University of Colorado at Boulder in 2011, and is currently a post-doctoral fellow at the University of Pennsylvania.

    A couple of weeks ago, an article in the New York Times, written by reporter Sam Dillon, took a look at the new incentive program being used by the District of Columbia Public Schools (DCPS). Under this plan (called “Impact Plus”), teachers rated “highly effective” by the district’s new evaluation system are eligible for large cash bonuses and/or permanent salary increases.

    Dillon notes that, “The profession is notorious for losing thousands of its brightest young teachers within a few years, which many experts attribute to low starting salaries and a traditional step-raise structure that rewards years of service and academic degrees rather than success in the classroom." He also profiles several teachers who received the bonuses, most of whom say it played a role in their decision to remain in the classroom.

    Putting aside these anecdotes and characterizations of “experts’” views, the idea that financial incentives – such as bonuses for performance or teaching in hard-to-staff schools – is a key to boosting teacher retention is a complex empirical question, and an open one at that.

  • Are Americans Exceptional In Their Attitudes Toward Government's Role In Reducing Inequality?

    As discussed in a previous post, roughly half of Americans believe that government should take some active role in reducing income differences between rich and poor, though, as one would expect, this view is less prevalent among Republicans, more educated and higher earning survey respondents.

    These data, however, lack a frame of reference. That is, they don’t tell us whether American support for government redistribution is “high” or “low” compared with that in other nations. The conventional wisdom in this area is that Americans generally prefer a more limited government, especially when it comes to things like income redistribution.

    It might therefore be interesting to take a quick look at how the U.S. stacks up against other nations in terms of these redistributive preferences.

  • The Persistence Of Both Teacher Effects And Misinterpretations Of Research About Them

    In a new National Bureau of Economic Research working paper on teacher value-added, researchers Raj Chetty, John Friedman and Jonah Rockoff present results from their analysis of an incredibly detailed dataset linking teachers and students in one large urban school district. The data include students’ testing results between 1991 and 2009, as well as proxies for future student outcomes, mostly from tax records, including college attendance (whether they were reported to have paid tuition or received scholarships), childbearing (whether they claimed dependents) and eventual earnings (as reported on the returns). Needless to say, the actual analysis includes only those students for whom testing data were available, and who could be successfully linked with teachers (with the latter group of course limited to those teaching math or reading in grades 4-8).

    The paper caused a remarkable stir last week, and for good reason: It’s one of the most dense, important and interesting analyses on this topic in a very long time. Much of the reaction, however, was less than cautious, specifically the manner in which the research findings were interpreted to support actual policy implications (also see Bruce Baker’s excellent post).

    What this paper shows – using an extremely detailed dataset and sophisticated, thoroughly-documented methods – is that teachers matter, perhaps in ways that some didn’t realize. What it does not show is how to measure and improve teacher quality, which are still open questions. This is a crucial distinction, one which has been discussed on this blog numerous times (also here and here), as it is frequently obscured or outright ignored in discussions of how research findings should inform concrete education policy.

  • New Report: Does Money Matter?

    Over the past few years, due to massive budget deficits, governors, legislators and other elected officials are having to slash education spending. As a result, incredibly, there are at least 30 states in which state funding for 2011 is actually lower than in 2008. In some cases, including California, the amounts are over 20 percent lower.

    Only the tiniest slice of Americans believe that we should spend less on education, while a large majority actually supports increased funding. At the same time, however, there’s a concerted effort among some advocates, elected officials and others to convince the public that spending more money on education will not improve outcomes, while huge cuts need not do any harm.

    Often, their evidence comes down to some form of the following graph:

  • Is California's "API Growth" A Good Measure Of School Performance?

    California calls its “Academic Performance Index” (API) the “cornerstone” of its accountability system. The API is calculated as a weighted average of the proportions of students meeting proficiency and other cutoffs on the state exams.

    It is a high-stakes measure. “Growth” in schools’ API scores determines whether they meet federal AYP requirements, and it is also important in the state’s own accountability regime. In addition, toward the middle of last month, the California Charter Schools Association called for the closing of ten charter schools based in part on their (three-year) API “growth” rates.

    Putting aside the question of whether the API is a valid measure of student performance in any given year, using year-to-year changes in API scores in high-stakes decisions is highly problematic. The API is cross-sectional measure – it doesn’t follow students over time – and so one must assume that year-to-year changes in a school’s index do not reflect a shift in demographics or other characteristics of the cohorts of students taking the tests. Moreover, even if the changes in API scores do in fact reflect “real” progress, they do not account for all the factors outside of schools’ control that might affect performance, such as funding and differences in students’ backgrounds (see here and here, or this Mathematica paper, for more on these issues).

    Better data are needed to test these assumptions directly, but we might get some idea of whether changes in schools’ API are good measures of school performance by testing how stable they are over time.

  • Teacher Retention: Estimating The Effects Of Financial Incentives In Denver

    Our guest author today is Eleanor Fulbeck, who earned her Ph.D. in education policy from the University of Colorado at Boulder in 2011, and is currently a post-doctoral fellow at the University of Pennsylvania.

    There is currently much interest in improving access to high-quality teachers (Clotfelter, Ladd, & Vigdor, 2010; Hanushek, 2007) through improved recruitment and retention. Prior research has shown that it is difficult to retain teachers, particularly in high-poverty schools (Boyd et al., 2011; Ingersoll, 2004). Although there is no one reason for this difficulty, there is some evidence to suggest teachers may leave certain schools or the profession in part because of dissatisfaction with low salaries (Ingersoll, 2001).

    Thus, it is possible that by offering teachers financial incentives, whether in the form of alternative compensation systems or standalone bonuses, they would become more satisfied with their jobs and retention would increase. As of yet, however, support for this approach has not been grounded in empirical research.

    Denver’s Professional Compensation System for Teachers ("ProComp") is one of the most prominent alternative teacher compensation reforms in the nation.* Via a combination of ten financial incentives, ProComp seeks to increase student achievement by motivating teachers to improve their instructional practices and by attracting and retaining high-quality teachers to work in the district.

    My research examines ProComp in terms of: 1) whether it has increased retention rates; 2) the relationship between retention and school quality (defined in terms of student test score growth); and 3) the reasons underlying these effects. I pay special attention to the effects of ProComp on schools that serve high concentrations of poor students – “Hard to Serve” (HTS) schools where teachers are eligible to receive a financial incentive to stay. The quantitative findings are discussed briefly below (I will discuss my other results in a future post).

  • Do Half Of New Teachers Leave The Profession Within Five Years?

    You’ll often hear the argument that half or almost half of all beginning U.S. public school teachers leave the profession within five years.

    The implications of this statistic are, of course, that we are losing a huge proportion of our new teachers, creating a “revolving door” of sorts, with teachers constantly leaving the profession and having to be replaced. This is costly, both financially (it is expensive to recruit and train new teachers) and in terms of productivity (we are losing teachers before they reach their peak effectiveness). And this doesn’t even include teachers who stay in the profession but switch schools and/or districts (i.e., teacher mobility).*

    Needless to say, some attrition is inevitable, and not all of it is necessarily harmful, Many new teachers, like all workers, leave (or are dismissed) because they are just aren’t good at it – and, indeed, there is test-based evidence that novice leavers are, on average, less effective. But there are many other excellent teachers who exit due to working conditions or other negative factors that might be improved (for reviews of the literature on attrition/retention, see here and here).

    So, the “almost half of new teachers leave within five years” statistic might serve as a useful diagnosis of the extent of the problem. As is so often the case, however, it's rarely accompanied by a citation. Let’s quickly see where it comes from, how it might be interpreted, and, finally, take a look at some other relevant evidence.

  • New Policy Brief: The Evidence On Charter Schools And Test Scores

    In case you missed it, today we released a new policy brief, which provides an accessible review of the research on charter schools’ testing effects, how their varying impacts might be explained, and what this evidence suggests about the ongoing proliferation of these schools.

    The brief is an adaptation of a three-part series of posts on this blog (here is part one, part two and part three).

    Download the policy brief (PDF)

    The abstract is pasted directly below.