Learning From Teach For America

There is a small but growing body of evidence about the (usually test-based) effectiveness of teachers from Teach for America (TFA), an extremely selective program that trains and places new teachers in mostly higher needs schools and districts. Rather than review this literature paper-by-paper, which has already been done by others (see here and here), I’ll just give you the super-short summary of the higher-quality analyses, and quickly discuss what I think it means.*

The evidence on TFA teachers focuses mostly on comparing their effect on test score growth vis-à-vis other groups of teachers who entered the profession via traditional certification (or through other alternative routes). This is no easy task, and the findings do vary quite a bit by study, as well as by the group to which TFA corps members are compared (e.g., new or more experienced teachers). One can quibble endlessly over the methodological details (and I’m all for that), and this area is still underdeveloped, but a fair summary of these papers is that TFA teachers are no more or less effective than comparable peers in terms of reading tests, and sometimes but not always more effective in math (the differences, whether positive or negative, tend to be small and/or only surface after 2-3 years). Overall, the evidence thus far suggests that TFA teachers perform comparably, at least in terms of test-based outcomes.

Somewhat in contrast with these findings, TFA has been the subject of both intensive criticism and fawning praise. I don’t want to engage this debate directly, except to say that there has to be some middle ground on which a program that brings talented young people into the field of education is not such a divisive issue. I do, however, want to make a wider point specifically about the evidence on TFA teachers – what it might suggest about the current focus to “attract the best people” to the profession.

This goal – recruiting and retaining talented people into teaching – is shared by most everyone, but it is among the most central emphases of the diverse group that might be called market-based reformers. Their idea is to change compensation structures, performance evaluations and other systems in order to create the kind of environment that will be appealing to high-achieving, less risk-averse people, as well as to ensure that those who aren't cut out for the job are compelled to leave. This will, so the argument goes, create a “dynamic profession” more in line with the high risk, high reward model common among the private sector firms competing for the same pool of young workers.

No matter your feelings on TFA, it’s more than fair to say that their corps members fit this profile perfectly. On paper, they aren’t just "top third," but top third of the top third. TFA cohorts enter the labor market having been among the highest achievers in the best colleges and universities in the nation. Getting accepted to the program is very, very difficult. Those who make it are not only service-oriented, but also smart, hard-working and ambitious. They are exactly the kind of worker that employers crave, and market-based reformers have made it among their central purposes to attract to the profession.

Yet, at least by the standard of test-based productivity, TFA teachers really don’t do better, on average, than their peers, and when there are demonstrated differences, they are often relatively small and concentrated in math (the latter, by the way, might suggest the role of unobserved differences in content knowledge). Now, again, there is some variation in the findings, and the number and scope of these analyses are limited – we’re nowhere near some kind of research consensus on these comparisons of test-based productivity, to say nothing of other sorts of student outcomes.

(It’s also very important to note that, for all we know, TFA teachers would get better results with more extensive preparation. After all, even the most well-designed five-week training regimen would have trouble preparing teachers for placement in some of the highest-needs schools and districts in the nation.)

Still, even these admirable young people, who could probably have their choice of jobs outside education, end up being just hard-working teachers, struggling to manage classrooms, plan lessons and get as much learning as possible from their students, often under less-than-ideal conditions. This squares with the related literature showing that the majority of measurable pre-service characteristics, such as the selectivity of undergraduate institution, GPA, etc., are, at best, inconsistently predictive of future classroom performance (at least as measured by growth model estimates). The variation within these groups completely overwhelms the variation between them.

This is one of the reasons why, whenever I hear someone talk about the need to “attract the best people” to teaching, I wonder as to their conceptualization of the “best people." In most cases, they’re talking about the kind of folks that come through TFA.

And I’m all for getting these people into teaching - we should have as many of them in classrooms as we can. Sure, in TFA's case, they commit to only a couple of years and most do leave, but some do stay beyond that commitment, and it's worth noting that attrition and mobility are also extremely high among traditionally-certified teachers who, like TFA'ers, work in high-needs schools and districts. Moreover, it's not irrelevant that many of the TFA teachers who leave the classroom pursue leadership positions in education (positions which many teachers believe require classroom experience).

But, to me, one of the big, underdiscussed lessons of TFA is less about the program itself than what the test-based empirical research on its corps members suggests about the larger issue of teacher recruitment. Namely, it indicates that "talent" as typically gauged in the private sector may not make much of a difference in the classroom, at least not by itself. This doesn't necessarily mean that market-based policies won't lure great teachers, but it does suggest that, if we’re going to enact massive changes in personnel policy to attract a certain “type” of person to teaching, we might reexamine our assumptions on who we’re trying to attract and what they want.

- Matt Di Carlo

*****

* Probably the most rigorous study in this area, albeit one that is not easily generalized, is this Mathematica evaluation (later published in a peer-reviewed journal), which exploits random assignment of students to classrooms. A few high-quality examples of quasi-experimental treatments include: this published analysis of New York City teachers; this paper, which was also subsequently published and also used NYC data; and this recent working paper comparing alternatively- and traditionally-certified teachers in Florida. Finally, a 2009 working paper was the first to  compare TFA teachers placed in high schools.

Blog Topics
Education
Education Reform
Education Research
ESEA
Teach for America
Teacher Quality
Value-Added
Issues Areas
K-12 Education

Submitted by Bob Calder on March 19, 2012

Permalink

Turn it upside down and see if you recognize it.

The existing national cohort of K-12 teachers compares favorably to a TFA cohort of top performers recruited from top institutions.

This tends to indicate that reformers targeting teachers are doing so with an ideological goal rather than that of improving education. That's not surprising since the same is true of the charter school movement.

TFA has been used shamefully by the people supporting it.

Submitted by JB on March 19, 2012

Permalink

Two related critiques:

1.Consider the recruiting practices of TFA. It's hard to imagine that they are simply driven by a wholesale desire to recruit, as you state, "the best people." It's clear that different strategies are employed for different regions from year to year, such that reputations are developed for different groups of recruits from different recruitment pools across the nation. You seem to redeem TFA by re-affirming its self-proclaimed mantra that it in fact DOES recruit those that, to use an oft-repeated term, “add value” to private sector industries, without taking the time to think about the ways in which this might not be the case. TFA’s high numbers of applicants are due to aggressive and institutionally embedded recruitment practices that are sustained by the myth that being a TFA teacher is just as valuable in the marketplace as a stint as a two year analyst at any big investment bank. It is important to understand this as, in part, a marketing strategy and unpack the many ways in which recruits might deviate from this “model.”

2.You take issue with the notion that those who would do well in the private sector would make good teachers, but is that the right place to begin your critique? You seem to elevate the position of TFA recruits, suggesting that their skills- which you spent a large portion of the piece praising- are not fit for the classroom. In doing so, you are replaying the TFA driven narrative that it’s recruits are somehow superior to all other teachers (if not to all other people!) without questioning the structures of social and economic inequalities that have perpetuated the existence of such an “elite” group.

Submitted by Chris Tessone on March 20, 2012

Permalink

Excellent post, Matt.

Other countries that out perform us do tend to have teachers drawn from the slice of their college graduates that looks like TFA's talent pool. I suspect (though I don't know, of course) that our K-12 working environment is not really a place where anyone can thrive on average. (Lots of factors reward mediocrity, labor-management relations are not great for reasons one could blame on either side depending on the place, etc.) So the fact that TFAers are only as effective as traditionally-trained educators isn't that revealing to me.

I also suspect that the real value TFA has provided historically is in the entrepreneurs, principals, and other folks who have come out of their pipeline, which isn't captured in the classroom effectiveness data directly. Our experience as a charter school authorizer in Ohio has certainly borne this out--TFA is a great source of school leaders and school founders.

Anyhow, I share your sense that this should not be as polarizing as it is!

Submitted by Ed on March 20, 2012

Permalink

Matt,

Interesting take on the TFA research. I agree that there has been too much focus on how to attract the “best and the brightest," whatever that means. In the business literature on human resource management, Baron and Kreps (http://www.amazon.com/Strategic-Human-Resources-Frameworks-Managers/dp/…) argue that different firms and industries must understand the type of labor force they should seek out based on the nature of their work and technology (methods of production).

In some industries--such as investment banking, law, sports, and entertainment--a few “stars” often generate the majority of a firm’s profits and success. Identifying individuals who have the potential to be stars is thus important and taking some risks in hiring is not problematic. In other words, finding a few stars can more than make up for a few hiring duds. Other industries depend on consistent, reliable performance of a large number of “foot-soldiers,” and success comes from improving the average performance of workers and reducing variation in their performance via training, technology, organization and process improvements.

In still other industries (often involving safety and security), a few mistakes can be disastrous for a firm, so firms need “guardians” and try to make safe hires even if this might mean also reducing the number of potential high performers. Certification and credentialing is thus often important. For these firms/industries, the costs of a mistake are greater than the gains of a high performer.

In education, it seems as if we sometimes swing from the extremes of taking a guardian approach (emphasizing rigid credentialing to avoid “mistakes”) and focusing on hiring a few stars through targeted programs.

I think, however, given that the teaching force is so huge--there are 3-4 million teachers--it's unrealistic to take a "star" approach, but that is precisely the approach TFA promotes by arguing that they can produce enough leverage by creating a group of transformative leaders in education.

Ultimately, however, teaching is huge profession that relies on millions of foot soldiers. This is not to say that we shouldn’t be interested in attracting some of the best and brightest. We should. However, we should focus our main efforts on policies that hold promise for improving the average quality of teachers and that reduce the variability in their performance. This points to different strategies--improving pre-service and in-service training, improving the tools that are available to teachers in the form of better curriculum, assessments, technology, etc., and reorganizing schools to better leverage teachers skills and promote teacher learning and continuous improvement.

Submitted by CW on March 21, 2012

Permalink

Very interesting post. You have appropriately asked the right question. What does teaching "talent" look like in a preservice teacher? What are the characteristics we should be after in future teachers? TFA, in general, takes the approach that high GPAs in rigorous content areas are the right ingredients. And while the overall test-based data is flat I have talked with administrators that hire TFAers and they note that in some areas the increased content knowledge separates TFA candidates from their standard applicant pool.

An area this post reopens for me is that for years there have been "off the shelf" HR screening tools that through independent research have been predictive of higher student test performance. These tools tend to four on underlying beliefs and demeanors. Perhaps some application of those tools with TFA and traditional candidates could shed some light on this idea of identifying the proper talent set.

Submitted by Joe on March 21, 2012

Permalink

If TFA teachers are performing comparably to teachers with much more extensive training, that seems like a sign they're actually doing pretty well. Nonetheless, as you state, the evidence is pretty shaky, so it's hard to draw any conclusions.

It is pretty obvious public education needs a big shake up, though. Making the teaching profession more attractive almost can't be a bad way to do this, assuming we're willing to pay for it.

Submitted by ceolaf on March 21, 2012

Permalink

I think that there are two points that one MUST keep in mind when thinking about the "success" of TFA:

1) If you judge the success of a group based on the measures they choose of to focus on, you are choosing the most generous basis for that judgement. TFA focuses on test scores. Whether that is the right or wrong thing to do, there can be no more generous way to judge it. That TFA teachers do NOT produce substantially better test scores is a MUCH greater condemnation of TFA that it would be of other programs that do NOT focus on test scores.

2) TFA also points to the impact that those who leave TFA and the classroom can have, as public leaders and citizens. This is another key aspect of their program, and the reason why it TFA suggests that TFA should not be judged by its retention rates. Well, what IS the impact of the experience of being put in so many of our nation's most dysfunctional schools, with the most challenging populations and often among the worst leadership. If one's only experience as an adult in public education is in poorly run organization in which the challenges far surpass the local organizatonal capacities to address challenges, what is the impact? What is the impact of that non-representative/worst-case experience on future the opinions, judgements and forecasts of TFA corps members?

I begin all consideration of TFA with these two ideas in mind. In order to learn from TFA, we need to understand what what TFA's goals are, and where its efforts have been focused. Then we can get to model and implementation.

Frankly, I do not see what TFA can tell about us about teacher recruitment, "at least not by itself." I tend to think that it tells us more about teacher prepartion and training than recruitment, but all we can be sure of is what it tells us about a particular combination of recruitment and preparation.