Education

Talking About But Not Learning From Finland

Finland’s education system has become an international celebrity. Their remarkable results are being trumpeted, usually in the “What can we learn from them?" context. Yet a lot of the recent discussion about what we can learn – as far as concrete policies – has been rather shallow. 

Right now, the factoid that is getting the most play is that Finnish teachers come from the “top ten percent” of those entering the labor force, whereas U.S. teachers don’t. But without knowing the reasons behind this difference, this fact is not particularly useful.

Although there has been some interesting research on these issues (see here, here, here, here, here, and here), I still haven’t really seen a simple comparison of Finnish vs. American policies that can help us understand what they’re doing right (and perhaps what we’re doing wrong). I am not an expert in comparative education, but I have assembled a few quick lists of features and policies. Needless to say, I am not suggesting that we do everything Finland does, and cease doing everything they don’t. It's very difficult to isolate the unique effects of each of these policies. Also, more broadly, Finland is small (less than six million residents), homogeneous, and their welfare state keeps poverty and inequality at one of the lowest levels among all developed nations (the U.S. is among the highest).

But if we are going to learn anything from the Finnish system, it is important to lay out the concrete differences (I inevitably missed things, so please leave a comment if you have additions).

Performance-Enhancing Teacher Contracts?

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post.

Please check out our two other posts (here and here), which present summaries and discussions of the actual evidence on the relationship between unions and test scores.

For years, some people have been determined to blame teachers’ unions for all that ails public education in America. This issue has been around a long time (see here and here), but, given the tenor of the current debate, it seems to bear rehashing.  According to this view, teachers unions negatively affect student achievement primarily through the mechanism of the collective bargaining agreement, or contract. These contracts are thought to include “harmful” provisions, such as seniority-based layoffs and unified salary schedules that give raises based on experience and education rather than performance.

But a fairly large proportion of public school teachers are not covered under legally-binding contracts.  In fact, there are ten states in which there are no legally binding K-12 teacher contracts at all (AL, AZ, AR, GA, LA, MS, NC, SC, TX, and VA). Districts in a few of these states have entered into what are called “meet and confer” agreements about salary, benefits, and other working conditions, but administrators have the right to break these agreements at will. For all intents and purposes, these states are free of many of the alleged “negative union effects."

Here’s a simple proposition: If teacher union contracts are the problem, then we should expect to see higher achievement outcomes in the ten states where there are no binding teacher contracts.

So, let’s take a quick look at how states with no contracts compare with the states that have them.

Three Questions For Those Who Dismiss The Nashville Merit Pay Study

The reaction from many performance pay advocates to the Nashville evaluation released last week has been that the study is relatively meaningless (see here and here for examples).  The general interpretation: The results show that the pay bonuses do not improve student achievement, but short-term test score gains are not the "true purpose" of these incentive programs. What they are really supposed to improve, so the line goes, is the quality of people who pursue teaching as a career, as well as their retention rates.

While I disagree that the findings are not important (they are, if for no other reason than they discredit the idea that teachers are holding their effort hostage to more money), I am sympathetic towards the view that the study didn’t tackle the big issues. Attracting the best possible people into the profession – and keeping them there – are much more meaningful goals than short-term test score gains, and they are not addressed in this study (though some results for retention are reported).

But this argument also begs a few important questions that I hope we can answer before the Nashville study fades into evaluation oblivion.  I have three of them.

Remorse Code (Or Comments From A Crib Strangler)

Those who publicly advocate for the kind of education policies put forth in "Waiting for Superman" are now seeing the equivalent of a letter-high fastball down the middle. They can wait on it and crank it out, using the buzz created by a major motion picture to advance the movie’s/campaign’s arguments at face value. I'm a little late on this (at least by blog standards), but over at Fordham’s Flypaper blog, Mike Petrilli saw this fastball, yet instead of unloading, he sacrifice bunted the runner into scoring position for the good of the team.

Responding to an interview in which Davis Guggenheim, the film’s director, claims that charter schools have "cracked the code" on how to educate even the poorest kids, Petrilli warns against the hubris of thinking that we are anywhere beyond first steps when it comes to fixing urban schools. He points out that charters like KIPP benefit from selection effects (more motivated and informed parents seek out the schools), and that the degree to which these schools have actually "closed the gap" between poor and affluent schools has been somewhat oversold. Petrilli also notes that while some of these schools seem to have "cracked the code," there is still little idea of how to expand them to serve more than a tiny minority of poor kids.

Thoughtful comments like these should remind those of us who care about expanding quality education that, although we may have canyon-sized differences between us on what needs to be done (Petrilli claims that those who disagree with him are trying to "strangle" reforms "in their crib"), there may be a few important respects in which we are closer than we may appear. Still (in addition to the crib-strangling allegations), I would take issue with one of Petrilli’s central points – that charters like KIPP may have "cracked the code," and the main problem now is how to scale them up. From my perspective, the "code" is specific policies and practices that produce results. And on this front, we’re practically still using decoder rings from cereal boxes.

Educational Unilateralism

In New York City this week, a special "plenary summit" of the UN General Assembly met to encourage the world to step up support for the Millennium Development Goals (MDG) first okayed by the UN in 2000. These eight goals – which include slashing poverty, combating disease, fighting hunger, protecting the environment, and boosting education – had a 2015 target date for their achievement. Ten years on, the summit reviewed progress and urged participants to speed up the pace.While the eradication of disease and hunger was named as the key priority, the nations of the world also recognized the crucial importance of education. Goal 2 focuses on the right of all kids to at least a primary school education. Goal 3 promotes the right of girls to have the same access to education as boys – a major problem in much of the developing world. 

Although the U.S. is the world’s largest donor country, surveys show that few Americans have heard about it. President Obama, who during his campaign pledged to fund a $2 billion Global Fund for Education, has done little – what with the financial crisis and debates over both the means and ends of foreign assistance programs getting in the way. In the meanwhile, critics call MDG little more than a laundry list of needs, with no real strategy on how to achieve them. Still, the goals are well worth reading, if only as a reflection of what the world believes (at least on paper) are the rock-bottom problems facing humanity in the 21^st century.

Persistently Low-Performing Incentives

Today, the National Center on Performance Incentives (NCPI) and the RAND Corporation released a long-awaited experimental evaluation of teacher performance pay in Nashville, Tenn. It finds that performance bonuses have virtually no effect on student math test scores (there were small but significant gains by fifth graders, but only in two of the three years examined, and the gains did not last into sixth grade).

Since this is such a politically contentious issue, these findings are likely to spark a lot of posturing and debate. So it’s worth trying to put them in context. As I discussed in a prior post, we now have at least preliminary results from three randomized experimental evaluations of merit pay in the U.S., the first contemporary, high-quality evidence of its kind.  This Nashville report and the two previously-released studies – one from Chicago and one from New York City's schoolwide bonus program – reached the same conclusion: Performance bonuses for teachers have little or no discernable effect on student test scores. 

Although the NYC and Chicago findings are preliminary (the evaluations are still in progress), the NYC program provides schoolwide and not individual bonuses, and one additional study (Round Rock, Tex.) is yet to be released, the three already-released reports do represent a fairly impressive, though still very tentative, body of evidence on merit pay’s utility as a means to improve test scores.

And at this point, it’s a good bet that, when all the evaluations are final and the smoke has cleared, we will have to conclude that performance bonuses are, at the very least, a very unpromising policy for producing short-term test score gains. 

Research Wars

Weeks before the fact, a Sept. 29 forum sponsored by the Economic Policy Institute and the National Education Policy Center has sparked some interesting debate over at the National Journal. The event, centered around the recent book, Think Tank Research Quality: Lessons for Policy Makers, the Media and the Public, is an effort to separate "the junk research from the science."

The crux of the debate is whether the recent explosion of self-published reports by various educational think tanks has helped or hindered the effort to improve the quality of educational research. (Full disclosure: The Albert Shanker Institute is often called a "think tank" and we frequently self-publish.) The push and pull of dueling experts and conflicting reports, say some, has turned education research into a political football—moved down the field by one faction, only to be punted all the way to the other end by a rival faction—each citing "research" as their guide.

"My research says this works and that doesn’t," can always be countered by, "Oh yeah, well my research says that works and this doesn’t." There are even arguments about what "what works" means because, except for performance on standardized tests, our goals remain diverse, decentralized and subject to local control. As a result, public education is plagued by trial and error policies that rise and fall, district by district and state by state, like some sort of crazed popularity contest.

Teacher Quality On The Red Carpet; Accuracy Swept Under The Rug

The media campaign surrounding “Waiting for Superman," which has already drawn considerable coverage, only promises to get bigger. While I would argue – at least in theory – that increased coverage of education is a good thing, it also means that this is a critically important (and, in some respects, dangerous) time. Millions of new people will be tuning in, believing that they are hearing serious discussions about the state of public education in America and “what the research says” about how it can be improved.

It’s therefore a sure bet that what I’ve called the “teacher effects talking point” will be making regular appearances. It goes something like this: Teachers are the most important schooling factor influencing student achievement. This argument provides much of the empirical backbone for the current push toward changes in teacher personnel policies. It is an important finding based on high-quality research, one with clear policy implications. It is also potentially very misleading.

The same body of evidence that shows that teachers are the most important within-school factor influencing test score gains also demonstrates that non-school factors matter a great deal more. The first wave of high-profile articles in our newly-energized education debate not only seem to be failing to provide this context, but are ignoring it completely. Deliberately or not, they are publishing incorrect information dressed up as empirical fact, spreading it throughout a mass audience new to the topic, to the detriment of us all.

More Than One Way Of Winning

Our guest author today is James R. Stone, professor and director of the National Research Center for Career & Technical Education at the University of Louisville.

In recent years, a consensus has emerged among education researchers and policymakers that all students should graduate from high school both "college- and career-ready." President Obama has made this part of his education agenda. And numerous advocacy organizations have championed the notion. But what does the phrase actually mean?

"College-ready" usually means not needing remedial courses once in college, and "career-ready" is usually equated with college-ready. High standards and expectations are the means recommended to prepare college-ready graduates. This means rigorous courses aligned with standards, and tests to ensure that students meet those standards. Presumably, career-readiness comes with the same requirements. The evidence contradicts the rhetoric, however. Paul Barton at ETS, Peter Cappelli at the Wharton School, and other labor market experts argue that being prepared for college is not the same as being prepared for a successful transition into the workforce.

Perhaps we ought to consider an alternative framework that more clearly defines what college- and career-ready means.

Extra Time: More From The Magazine's Education Poll

** Also posted here on "Valerie Strauss’ Answer Sheet" in the Washington Post.

A recent education poll conducted by Time Magazine has gotten a lot of attention. Many of the questions are worded so badly that the results are rather meaningless. The question on merit pay, for example, defines the practice as “paying teachers according to their effectiveness” (who would oppose that, if it could be accurately measured?). Other questions are very interesting, such as the one asking whether respondents would pay higher taxes to improve public schools (56 percent would). Or the finding that, when asked what will “improve student achievement the most," more than twice as many people choose “more involved parents” (54 percent) over “more effective teachers” (24 percent).

But, as is sometimes the case, a few of the survey’s most interesting results were not included in the published article, which highlighted only 11 out of 40-50 or so total questions (the full set of results is available here). Here are three or four unpublished items that caught my eye (the sample size is 1,000, with a margin of error of +/- 3 percent):