Education

Income And Educational Outcomes

The role of poverty in shaping educational outcomes is one of the most common debates going on today. It can also be one of the most shallow.

The debate tends to focus on income. For example (and I’m generalizing a bit here), one “side” argues that income and test scores are strongly correlated; the other “side” points to the fact that many low-income students do very well and cautions against making excuses for schools’ failure to help poor kids.

Both arguments have merit, but it bears quickly mentioning that the focus on the relationship between income and achievement is a rather crude conceptualization of the importance of family background (and non-schooling factors in general) for education outcomes. Income is probably among the best widely available proxies for these factors, insofar as it is correlated with many of the conditions that can hinder learning, especially during a child’s earliest years. This includes (but is not at all limited to): peer effects; parental education; access to print and background knowledge; parental involvement; family stressors; access to healthcare; and, of course, the quality of neighborhood schools and their teachers.

And that is why, when researchers try to examine school performance – while holding constant the effect of factors outside of schools’ control – income or some kind of income-based proxy (usually free/reduced price lunch) can be a useful variable. It is, however, quite limited.

Trial And Error Is Fine, So Long As You Know The Difference

It’s fair to say that improved teacher evaluation is the cornerstone of most current education reform efforts. Although very few people have disagreed on the need to design and implement new evaluation systems, there has been a great deal of disagreement over how best to do so – specifically with regard to the incorporation of test-based measures of teacher productivity (i.e., value-added and other growth model estimates).

The use of these measures has become a polarizing issue. Opponents tend to adamantly object to any degree of incorporation, while many proponents do not consider new evaluations meaningful unless they include test-based measures as a major element (say, at least 40-50 percent). Despite the air of certainty on both sides, this debate has mostly been proceeding based on speculation. The new evaluations are just getting up and running, and there is virtually no evidence as to their effects under actual high-stakes implementation.

For my part, I’ve said many times that I'm receptive to trying value-added as a component in evaluations (see here and here), though I disagree strongly with the details of how it’s being done in most places. But there’s nothing necessarily wrong with divergent opinions over an untested policy intervention, or with trying one. There is, however, something wrong with fully implementing such a policy without adequate field testing, or at least ensuring that the costs and effects will be carefully evaluated post-implementation. To date, virtually no states/districts of which I'm aware have mandated large-scale, independent evaluations of their new systems.*

If this is indeed the case, the breathless, speculative debate happening now will only continue in perpetuity.

Beyond Anecdotes: The Evidence About Financial Incentives And Teacher Retention

** Also posted here on "Valerie Strauss' Answer Sheet" in the Washington Post

Our guest author today is Eleanor Fulbeck, who earned her Ph.D. in education policy from the University of Colorado at Boulder in 2011, and is currently a post-doctoral fellow at the University of Pennsylvania.

A couple of weeks ago, an article in the New York Times, written by reporter Sam Dillon, took a look at the new incentive program being used by the District of Columbia Public Schools (DCPS). Under this plan (called “Impact Plus”), teachers rated “highly effective” by the district’s new evaluation system are eligible for large cash bonuses and/or permanent salary increases.

Dillon notes that, “The profession is notorious for losing thousands of its brightest young teachers within a few years, which many experts attribute to low starting salaries and a traditional step-raise structure that rewards years of service and academic degrees rather than success in the classroom." He also profiles several teachers who received the bonuses, most of whom say it played a role in their decision to remain in the classroom.

Putting aside these anecdotes and characterizations of “experts’” views, the idea that financial incentives – such as bonuses for performance or teaching in hard-to-staff schools – is a key to boosting teacher retention is a complex empirical question, and an open one at that.

New Report: Does Money Matter?

Over the past few years, due to massive budget deficits, governors, legislators and other elected officials are having to slash education spending. As a result, incredibly, there are at least 30 states in which state funding for 2011 is actually lower than in 2008. In some cases, including California, the amounts are over 20 percent lower.

Only the tiniest slice of Americans believe that we should spend less on education, while a large majority actually supports increased funding. At the same time, however, there’s a concerted effort among some advocates, elected officials and others to convince the public that spending more money on education will not improve outcomes, while huge cuts need not do any harm.

Often, their evidence comes down to some form of the following graph:

Is California's "API Growth" A Good Measure Of School Performance?

California calls its “Academic Performance Index” (API) the “cornerstone” of its accountability system. The API is calculated as a weighted average of the proportions of students meeting proficiency and other cutoffs on the state exams.

It is a high-stakes measure. “Growth” in schools’ API scores determines whether they meet federal AYP requirements, and it is also important in the state’s own accountability regime. In addition, toward the middle of last month, the California Charter Schools Association called for the closing of ten charter schools based in part on their (three-year) API “growth” rates.

Putting aside the question of whether the API is a valid measure of student performance in any given year, using year-to-year changes in API scores in high-stakes decisions is highly problematic. The API is cross-sectional measure – it doesn’t follow students over time – and so one must assume that year-to-year changes in a school’s index do not reflect a shift in demographics or other characteristics of the cohorts of students taking the tests. Moreover, even if the changes in API scores do in fact reflect “real” progress, they do not account for all the factors outside of schools’ control that might affect performance, such as funding and differences in students’ backgrounds (see here and here, or this Mathematica paper, for more on these issues).

Better data are needed to test these assumptions directly, but we might get some idea of whether changes in schools’ API are good measures of school performance by testing how stable they are over time.

Teacher Retention: Estimating The Effects Of Financial Incentives In Denver

Our guest author today is Eleanor Fulbeck, who earned her Ph.D. in education policy from the University of Colorado at Boulder in 2011, and is currently a post-doctoral fellow at the University of Pennsylvania.

There is currently much interest in improving access to high-quality teachers (Clotfelter, Ladd, & Vigdor, 2010; Hanushek, 2007) through improved recruitment and retention. Prior research has shown that it is difficult to retain teachers, particularly in high-poverty schools (Boyd et al., 2011; Ingersoll, 2004). Although there is no one reason for this difficulty, there is some evidence to suggest teachers may leave certain schools or the profession in part because of dissatisfaction with low salaries (Ingersoll, 2001).

Thus, it is possible that by offering teachers financial incentives, whether in the form of alternative compensation systems or standalone bonuses, they would become more satisfied with their jobs and retention would increase. As of yet, however, support for this approach has not been grounded in empirical research.

Denver’s Professional Compensation System for Teachers ("ProComp") is one of the most prominent alternative teacher compensation reforms in the nation.* Via a combination of ten financial incentives, ProComp seeks to increase student achievement by motivating teachers to improve their instructional practices and by attracting and retaining high-quality teachers to work in the district.

My research examines ProComp in terms of: 1) whether it has increased retention rates; 2) the relationship between retention and school quality (defined in terms of student test score growth); and 3) the reasons underlying these effects. I pay special attention to the effects of ProComp on schools that serve high concentrations of poor students – “Hard to Serve” (HTS) schools where teachers are eligible to receive a financial incentive to stay. The quantitative findings are discussed briefly below (I will discuss my other results in a future post).

Do Half Of New Teachers Leave The Profession Within Five Years?

You’ll often hear the argument that half or almost half of all beginning U.S. public school teachers leave the profession within five years.

The implications of this statistic are, of course, that we are losing a huge proportion of our new teachers, creating a “revolving door” of sorts, with teachers constantly leaving the profession and having to be replaced. This is costly, both financially (it is expensive to recruit and train new teachers) and in terms of productivity (we are losing teachers before they reach their peak effectiveness). And this doesn’t even include teachers who stay in the profession but switch schools and/or districts (i.e., teacher mobility).*

Needless to say, some attrition is inevitable, and not all of it is necessarily harmful, Many new teachers, like all workers, leave (or are dismissed) because they are just aren’t good at it – and, indeed, there is test-based evidence that novice leavers are, on average, less effective. But there are many other excellent teachers who exit due to working conditions or other negative factors that might be improved (for reviews of the literature on attrition/retention, see here and here).

So, the “almost half of new teachers leave within five years” statistic might serve as a useful diagnosis of the extent of the problem. As is so often the case, however, it's rarely accompanied by a citation. Let’s quickly see where it comes from, how it might be interpreted, and, finally, take a look at some other relevant evidence.

New Policy Brief: The Evidence On Charter Schools And Test Scores

In case you missed it, today we released a new policy brief, which provides an accessible review of the research on charter schools’ testing effects, how their varying impacts might be explained, and what this evidence suggests about the ongoing proliferation of these schools.

The brief is an adaptation of a three-part series of posts on this blog (here is part one, part two and part three).

Download the policy brief (PDF)

The abstract is pasted directly below.

Today's Forecast: Cloud Computing In Education

It’s hard to tell whether cloud computing is "the next big thing" or just another buzz word, but, according to a recent survey of 5,300 organizations in 38 countries, change is already taking place: "the promises of reduced cost, improved performance and greater scalability" are driving interest in "moving to cloud."

But what does cloud computing mean to those of us who care about education, teaching and learning?

When an organization "goes cloud" it means that the organization no longer deals directly with many of its computing/IT needs – e.g., software, updates, storage etc. The key to understanding this model and its broader implications is to appreciate the transition it represents: from viewing computing as a product to viewing it as a service. Much like public utilities, IT resources are delivered to users through the internet, just like electricity is distributed to our homes through the power grid. Users pay according to their consumption level, and the service provider takes care of the rest – see here.

Evidently, by moving to the cloud, organizations (including schools and universities) can save on IT infrastructure and maintenance. Some have noted that the model could also bring about changes in the IT sector, perhaps require a different type of (and/or fewer) IT professionals. Second, cloud computing should also help increase accessibility to educational content and convenience. For example, if lessons and assignments are be posted and stored in the cloud (i.e., on the shared server), students can work from anywhere, collaborate/interact with their peers etc.

But what else is cloud computing?

The Deafening Silence Of Unstated Assumptions

Here’s a thought experiment. Let’s say we were magically granted the ability to perfectly design our public education system. In other words, we were somehow given the knowledge of the most effective policies and how to implement them, and we put everything in place. How quickly would schools improve? Where would we be after 20 years of having the best possible policies in place? What about after 50 years?

I suspect there is much disagreement here, and that answers would vary widely. But, since there is a tendency in education policy to shy away from even talking realistically about expectations, we may never really know. We sometimes operate as though we expect immediate gratification - quick gains, every single year. When schools or districts don't achieve gains, even over a short period of time, they are subject to being labeled as failures.

Without question, we need to set and maintain high expectations, and no school or district should ever cease trying to improve. Yet, in the context of serious policy discussions, the failure to even discuss expectations in a realistic manner hinders our ability to interpret and talk about evidence, as it often means that we have no productive standard by which to judge our progress or the effects of the policies we try.