Education

Are Value-Added Models Objective?

In recent discussions about teacher evaluation, some people try to distinguish between "subjective" measures (such as principal and peer observations) and "objective" measures (usually referring to value-added estimates of teachers’ effects on student test scores).

In practical usage, objectivity refers to the relative absence of bias from human judgment ("pure" objectivity being unattainable). Value-added models are called "objective" because they use standardized testing data and a single tool for analyzing them: All students in a given grade/subject take the same test and all teachers’ "effects" in a given district or state are estimated by the same model. Put differently, all teachers are treated the same (at least those 25 percent or so who teach grades and subjects that are tested), and human judgment is relatively absent.

By this standard, are value-added models objective? No. And it is somewhat misleading to suggest that they are.

Does Language Shape Thought?

Do the words we use frame the thoughts that we have? And, if so, does the language we speak affect how we think?

It turns out that linguists and cognitive scientists have been going back and forth on this issue for years. There is a fascinating article on the subject in last weekend’s New York Times Magazine (which I’m only now getting around to reading). It’s a piece by Guy Deutscher, an honorary research fellow at the School of Languages, Linguistics and Cultures at the University of Manchester, and author of a forthcoming book, Through the Language Glass: Why the World Looks Different in Other Languages.

First popularized in the 1940s, the Sapir-Whorf hypothesis of linguistic relativity "seduced a whole generation into believing that our mother tongue restricts what we are able to think." The lack of a future and past tense in a given language, for example, was supposed to limit some speakers’ ability to comprehend the concepts of future and past.

Although such ethnocentric and romantic aspects of the theory have been totally discredited, new research does suggest that language can have an effect on both thought and perception. For example, it has recently "been demonstrated in a series of ingenious experiments that we even perceive colors through the lens of our mother tongue." As it turns out, different languages "carve up the spectrum of visible light" in different ways, with, for instance, many languages considering blue and green to be variations of the same color. And, astonishingly, "our brains are trained to exaggerate the distance between shades of color if these have different names in our language." So, "as strange as it may sound, our experience of a Chagall painting actually depends to some extent on whether our language has a word for blue."

Help The Economy: Put A Teacher's Aide In Every Classroom

Economist Robert Shiller (co-creator of the Case-Shiller Home Price Index, an essential tool for investors and economists) has an interesting idea for stimulating the economy: Put a teacher’s aide in every classroom.

Why? As reported by the Wall Street Journal, "Not only would it employ millions, but it would be good for the children," who would benefit from "the extra attention of another person."

Shiller is regarded as one of the most important economists today. The Arthur M. Okun professor of economics at Yale University and professor of finance at the Yale School of Management, he forewarned about both the dot.com bust and the housing bubble. For years, he criticized the so-called efficient markets model of economics, which many today cite as a key driver of the policies that led to the financial crisis. He is also the author of many books, including, Irrational Exuberance in 2000, which warned that the peaking real estate and stock markets were in bubble territory.

Shiller is worried about today’s economy. He estimates that the likelihood of a double-dip recession is growing and that we are "teetering" on the brink of a dangerous deflationary spiral. What to do?

The Test-Based Language Of Education

A recent poll on education attitudes from Gallup and Phi Delta Kappan got a lot of attention, including a mention on ABC’s "This Week with Christian Amanpour," which devoted most of its show to education yesterday. They flashed results for one of the poll’s questions, showing that 72 percent of Americans believe that "each teacher should be paid on the basis of the quality of his or her work," rather than on a "standard-scale basis."

Anyone who knows anything about survey methodology knows that responses to questions can vary dramatically with different wordings (death tax, anyone?). The wording of this Gallup/PDK question, of course, presumes that the "quality of work" among teachers might be measured accurately. The term "teacher quality" is thrown around constantly in education circles, and in practice, it is usually used in the context of teachers’ effects on students’ test scores (as estimated by various classes of "value-added" models).

But let’s say the Gallup/PDK poll asked respondents if "each teacher should be paid on the basis of their estimated effect on their students’ standardized test scores, relative to other teachers." Think the results would be different? Of course. This doesn’t necessarily say anything about the "merit" of the compensation argument, so to speak, nor does it suggest that survey questions should always emphasize perfect accuracy over clarity (which would also create bias of a different sort). But has anyone looked around recently and seen just how many powerful words, such as "quality," are routinely used to refer to standardized test score-related measures? I made a tentative list.

The Cost Of Success In Education

Many are skeptical of the current push to improve our education system by means of test-based “accountability” - hiring, firing, and paying teachers and administrators, as well as closing and retaining schools, based largely on test scores. They say it won’t work. I share their skepticism, because I think it will.

There is a simple logic to this approach: when you control the supply of teachers, leaders, and schools based on their ability to increase test scores, then this attribute will become increasingly common among these individuals and institutions. It is called “selecting on the dependent variable," and it is, given the talent of the people overseeing this process and the money behind it, a decent bet to work in the long run.

Now, we all know the arguments about the limitations of test scores. We all know they’re largely true. Some people take them too far, others are too casual in their disregard. The question is not whether test scores provide a comprehensive measure of learning or subject mastery (of course they don’t). The better question is the extent to which teachers (and schools) who increase test scores a great deal are imparting and/or reinforcing the skills and traits that students will need after their K-12 education, relative to teachers who produce smaller gains. And this question remains largely unanswered.

This is dangerous, because if there is an unreliable relationship between teaching essential skills and the boosting of test scores, then success is no longer success. And by selecting teachers and schools based on those scores, we will have deliberately engineered our public education system to fail in spite of success.

It may be only then that we truly realize what we have done.

Accountability For Us, No Way; We're The Washington Post

In his August 4th testimony before the Senate’s Committee on Health, Education, Labor and Pensions, Government Accountability Office (GAO) official Gregory D. Kutz offered an earful of scandalous stories about how for-profit, post-secondary institutions use misrepresentation, fraud, and generally unethical practices to tap the federal loan and grant-making trough. One of these companies, so says the Washington Post itself, is Kaplan Inc, a profit-making college that contributes a whopping amount to the paper’s bottom line (67 percent of the Washington Post Company’s $92 million in second quarter earnings, according to the Washington Examiner; 62 percent according to the Post’s Ombudsman Andrew Alexander).

One might assume that the Post's deep financial involvement in Kaplan Inc. would prompt its editorial board to recuse itself from comment on new proposed federal regulations designed to correct the problems. Instead of offering "point-counterpoint" op-eds on this issue, this bastion of journalistic integrity has launched a veritable campaign in support of its corporate education interests, and offered up its op-ed page to education business allies. It is a sad and disappointing chapter in the history of this once-great institution.

Data-Driven Decisions, No Data

According to an article in yesterday’s Washington Post, the outcome of the upcoming D.C. mayoral primary may depend in large part on gains in students’ “test scores” since Mayor Adrian Fenty appointed Michelle Rhee to serve as chancellor of the D.C. Public Schools (DCPS).

That struck me as particularly interesting because, as far as I can tell, Michelle Rhee has never released any test scores to the public. Not an average test score for any grade level or for any of the district’s schools or any subgroup of its students. None.

What About Curriculum Effects?

Our guest author today is Barak Rosenshine, emeritus professor of educational psychology at the University of Illinois at Urbana-Champaign.

Bill Gates, the Los Angeles Times, and others have argued that teachers should be held accountable for the achievement of their students. This has led to heated debates over the validity and proper use of value added statistical measures. But no one seems to be talking about curriculum effects. What if an excellent teacher is in a school that has selected a curriculum for mathematics or for reading that isn’t very good. How accountable should the teacher be in those circumstances?

Think You Know About Teacher Attrition?

Every four years, with the release of the Teacher Follow-Up Survey (TFS), we get what is virtually our only source of reliable information on the rates of and reasons for teacher attrition in the U.S.

The survey is a supplement to the much larger Schools and Staffing Survey (SASS), which is also conducted every four years by the National Center for Education Statistics. The SASS is an extensive survey of characteristics, conditions, and other variables of over 30,000 teachers across the nation. The TFS simply contacts a sample of the SASS participants the following year to see if they’re still teaching, and if not, what they’re doing. For this round, the 2007-08 SASS respondents were contacted again in 2008-09 for the TFS.

The TFS divides respondents into three categories: stayers (teachers in the same school as last year); movers (teachers who are still teaching but in a different school and/or district); and leavers (teachers who left teaching for whatever reason, including retirement). Overall, among public school teachers in 2008-09, 84.5 percent were stayers, 7.6 percent were movers, and 7.9 percent were leavers (note that these are annual, not cumulative rates).

While some of the TFS results are unsurprising, there may be plenty about teacher attrition that you thought you knew but didn’t.

Education Reform, Redux

Ever get the feeling that we are having the same old educational debate, over and over? A glance through the archives of the Atlantic Monthly helps to cement the notion.

One writer describes schools as “society's dumping ground,…a vast refuse heap for any and every unwanted service or task that other social or governmental institutions and agencies find too tough to handle. The community, the home, and to some extent even the church have used the public schools to relieve their consciences of feelings of guilt by passing on unfinished business which they have found [too] difficult …or just burdensome." That was 1959.

Another pleads for “education reform," while admitting that the term has been so overused as to become virtually meaningless. “America has been oversold on pedagogical gadgets which never perform up to expectations," he says. But, since “standards in American public education are deplorably and inexcusably low," something must be done. In a democracy, he writes, every citizen deserves “an education… [grounded] in learning, in mastery, in growing insight, in standards which really operate – and not just in going to school. So when multitudes of young people accumulate credits, pass courses, carry off elegant [diplomas], and come out knowing little or nothing, it is simply intolerable." That was 1939.