Shanker Blog

Investment Counselors

NOTE: With this post, we are starting a new “feature” here at Shanker Blog – periodically summarizing research papers that carry interesting and/or important implications for the education policy debates. We intend to focus on papers that are either published in peer-reviewed journals or are still in working paper form, and are unlikely to get significant notice. Here is the first:

Are School Counselors a Cost-Effective Education Input?

Scott E. Carrell and Mark Hoekstra, Working paper (link to PDF), September 2010

Most teachers and principals will tell you that non-instructional school staff can make a big difference in school performance. Although we may all know this, it’s always useful to have empirical research to confirm it, and to examine the size and nature of the effects. In this paper, economists Scott Carrell and Mark Hoekstra put forth one of the first rigorous tests of how one particular group of employees – school counselors – affect both discipline and achievement outcomes. The authors use a unique administrative dataset of third, fourth, and fifth graders in Alachua County, Florida, a diverse district that serves over 30,000 students.  Their approach exploits year-to-year variation in the number of counselors in each school – i.e., whether the outcomes of a given school change from the previous year when a counselor is added to the staff.

Their results are pretty striking: The addition of a single full-time counselor is associated with a 0.04 standard deviation increase in boys’ achievement (about 1.2 percentile points). These effects are robust across different specifications (including sibling and student fixed effects). The disciplinary effects are, as expected, even more impressive. A single additional counselor helps to decrease boys’ disciplinary infractions between 15 to 26 percent.

Middle Class Overvalues

It is a staple of American politics that elected officials routinely frame their appeals to the "middle class." The idea is simple: Since the vast majority of Americans consider themselves members of the middle class, it makes sense to use this label as shorthand for "people like you." The practice of people locating themselves within a class structure – rather than being "assigned" to classes based on particular characteristics, such as income or occupation – is often called "subjective class identification."

Now, I don’t know whether the daily use of the "middle class" appeal is smart politics (I assume it has been poll-tested and focus-grouped ad nauseum). But I can say that the underlying assumption – that virtually all Americans identify subjectively as middle class – is not correct. Or, more accurately, it depends on how you ask the question.

If you give people a three-category class structure (upper, middle, lower) in which they must place themselves, a huge plurality opts for the middle. On the other hand, in the simple tabulation below, I demonstrate what happens when you add a fourth category to the menu of options – "working class." The data come from the General Social Survey (GSS) for 2010, and it’s a representative, unrestricted sample of all Americans over age 18. One of the survey questions is: "If you were asked to use one of four names for your social class, would you say you belong in: the lower class, the working class, the middle class, or the upper class?"

Great Expectations

A couple of years ago, Eat Pray Love author Elizabeth Gilbert explored the negative side of our unrealistically high expectations for artists and, more generally, for those who rely on their creativity to make a living. In ancient Rome, Gilbert recounts, creativity was associated with a sort of divine spirit that came to human beings from some distant and unknowable source, for distant and unfathomable reasons. The Romans referred to this intangible spirit as a genius. An individual was not a genius, but rather had a genius - a magical entity who was believed to live in the walls of an artist's studio and who would come out and invisibly assist the artist with his/her work. The lesson Gilbert draws is one of humility (i.e., successes are not entirely ours – don’t be such a narcissist) and emancipatory relief (i.e., failures are not completely our fault either – can’t hurt to try).

What does all this have to do with education and teachers? It seems to me that our expectations for both teachers and artists are sometimes unrealistic and unproductive, if not detrimental. Great teachers are often portrayed as superheroes, unencumbered by anything that might distract them from their teaching crusade – "refusing to surrender to the combined menaces of poverty, bureaucracy, and budgetary shortfalls." As a recent article in The Atlantic explained, Teach for America now asks applicants to talk about how they have overcome the challenges in their lives and uses these answers to rate their perseverance.

Yet the meaning of "Great Teacher" rarely gets analyzed. Instead, our definition of greatness – or even competence – remains a convenient black box, leading some to suggest that the question of what makes a teacher great is less important than separating the wheat from the chaff. In turn, this reveals a simplistic and, in my view, negative assumption that greatness, unlike Gilbert’s genius, is a stable, static, innate, and independent attribute. You either have it or you don’t.

No Excuses In Anti-Poverty Policy As Well

Much of the current education debate consists of a constant, ongoing argument about the role of poverty. One “side” is accused of using poverty as an excuse for not improving schools, and of saying that poverty is destiny in regard to educational outcomes. The other “side” is accused of completely ignoring the detrimental effects of poverty, and of arguing that market-based reforms can by themselves transform our public education system.

Both portrayals are inaccurate, and both “sides” know it, yet the accusations continue. Of course there is a core of truth in the characterizations, but the differences are far more nuanced than the opponents usually communicate. It’s really a matter of degree. In addition to differences in the specifics of what should be done, a lot boils down to variations in how much improvement we believe can be gained by teacher-focused education reform (or by education reform in general) by itself. In other words, some people have higher expectations than others.

I have previously argued that the reasonable expectation for teacher quality-based reforms is that, if everything goes perfectly (which is far from certain), they will generate very slow, gradual improvement over a period of years and decades. This means we should make these changes, but be very careful to design them sensibly, monitor their effects, and maintain realistic expectations (for the record, I think we are, in many respects, falling short on all three counts).

But the thing that I find a little frustrating about the whole poverty/education thing is that, while nobody should use poverty as an excuse in education policy, it’s not uncommon to hear education used as an excuse, of sorts, in discussions about anti-poverty policy.

In Census Finance Data, Most Charters Are Not Quite Public Schools

Last month, the U.S. Census Bureau released its annual public K-12 school finance report (and accompanying datasets). The data, which are for FY 2009 (there’s always a lag in finance data), show that spending increased roughly two percent from the previous year. This represents much slower growth than usual.

These data are a valuable resource that has rightfully gotten a lot of attention. But there’s a serious problem within them, which, while slightly technical, hasn’t received any attention at all: The vast majority of public charter schools are not included in the data.

To gather its data, the Census Bureau relies on reporting from “government entities." Some charter schools fit this description neatly, such as those operated by governments or government-affiliated bodies, including states, districts, counties, and public universities. But most charter schools are operated by private organizations (mostly non-profits), and finance figures for these schools are not included in the report (the Census classifies them as "private charter schools").

What does this mean? Well, for one thing, it means that the overall spending figures (total dollar amounts) are a bit understated. Charters only account for a relatively small proportion of all public school enrollments (around 5-6 percent); still, given the huge amounts of money we’re dealing with here (the U.S. spends roughly $600 billion a year), we’re talking about quite a bit in absolute terms. Perhaps more important is the potential effect on per-pupil spending figures – the way that education financing is usually expressed.

Another Tiananmen Anniversary: Will There Be A Reckoning?

This Saturday, June 4, 2011, marks the 22nd anniversary of the 1989 Tiananmen Square massacre, where thousands of pro-democracy activists were killed, injured or imprisoned by Chinese authorities.  This year’s Tiananmen anniversary comes at a time of greatly increased political repression in China.  According to the Congressional-Executive Commission on China (CECC), “Chinese authorities have launched a broad crackdown against rights defenders, reform advocates, lawyers, petitioners, writers, artists, and Internet bloggers in what international observers have described as one of the harshest crackdowns in years."

Over the last several months, activist groups such as Chinese Human Rights Defenders (CHRD) have repeatedly tried to draw attention to this harsh renewal of repression in China. In an article entitled “Missing before Action” in the March issue of Foreign Policy Magazine, a CHRD writer noted that hundreds of Chinese human rights activists, lawyers, and pro-democracy dissidents from across the country have been affected by the crackdown. Police have used “violence, arbitrary detention, "disappearances," and other forms of harassment and intimidation” to put a damper on any nascent protest movement.  Other dissidents --or non-dissident citizens walking the streets -- have been picked up for questioning.

Although authorities  began tightening the political screws in the period leading up to  the 2008 Beijing Olympics, it appears that the recent democratic uprisings in the Middle East have given added impetus to this policy.

When It Comes To How We Use Evidence, Is Education Reform The New Welfare Reform?

** Also posted here on “Valerie Strauss’ Answer Sheet” in the Washington Post

In the mid-1990s, after a long and contentious debate, the U.S. Congress passed the Personal Responsibility and Work Opportunity Reconciliation Act of 1996, which President Clinton signed into law. It is usually called the “Welfare Reform Act," as it effectively ended the Aid to Families with Dependent Children (AFDC) program (which is what most people mean when they say “welfare," even though it was [and its successor is] only a tiny part of our welfare state). Established during the New Deal, AFDC was mostly designed to give assistance to needy young children (it was later expanded to include support for their parents/caretakers as well).

In place of AFDC was a new program – Temporary Assistance for Needy Families (TANF). TANF gave block grants to states, which were directed to design their own “welfare” programs. Although the states were given considerable leeway, their new programs were to have two basic features: first, for welfare recipients to receive benefits, they had to be working; and second, there was to be a time limit on benefits, usually 3-5 years over a lifetime, after which individuals were no longer eligible for cash assistance (states could exempt a proportion of their caseload from these requirements). The general idea was that time limits and work requirements would “break the cycle of poverty”; recipients would be motivated (read: forced) to work, and in doing so, would acquire the experience and confidence necessary for a bootstrap-esque transformation.

There are several similarities between the bipartisan welfare reform movement of the 1990s and the general thrust of the education reform movement happening today. For example, there is the reliance on market-based mechanisms to “cure” longstanding problems, and the unusually strong liberal-conservative alliance of the proponents. Nevertheless, while calling education reform “the new welfare reform” might be a good soundbyte, it would also take the analogy way too far.

My intention here is not to draw a direct parallel between the two movements in terms of how they approach their respective problems (poverty/unemployment and student achievement), but rather in how we evaluate their success in doing so. In other words, I am concerned that the manner in which we assess the success or failure of education reform in our public debate will proceed using the same flawed and misguided methods that were used by many for welfare reform.

The Ethics of Testing Children Solely To Evaluate Adults

The recent New York Times article, “Tests for Pupils, but the Grades Go to Teachers," alerts us of an emerging paradox in education – the development and use of standardized student testing solely as a means to evaluate teachers, not students. “We are not focusing on teaching and learning anymore; we are focusing on collecting data," says one mother quoted in the article. Now, let’s see: collecting data on minors that is not explicitly for their benefit – does this ring a bell?

In the world of social/behavioral science research, such an enterprise – collecting data on people, especially on minors – would inevitably require approval from the Institutional Review Board (IRB). For those not familiar, IRB is a committee that oversees research that involves people and is responsible for ensuring that studies are designed in an ethical manner. Even in conducting a seemingly harmless interview on political attitudes or observing a group studying in a public library, the researcher would almost certainly be required to go through a series of steps to safeguard participants and ensure that the norms governing ethical research will be observed.

Very succinctly, IRBs’ mission is to see that (1) the risk-benefit ratio of conducting the research is favorable; (2) any suffering or distress that participants may experience during or after the study is understood, minimized, and addressed; and (3) research participants’ agreed to participate freely and knowingly – usually, subjects are requested to sign an informed consent which includes a description of the study’s risks and benefits, a discussion of how confidentiality will be guaranteed, a statement on the voluntary nature of involvement, and a clarification that refusal or withdrawal at any time will involve no penalty or loss of benefits. When the research involves minors, parental consent and sometimes child assent are needed.

In short, IRB procedures exist to protect people. To my knowledge, student evaluation procedures and standardized testing are exempt from this sort of scrutiny. So the real question is: Should they be? Perhaps not.

Value-Added In Teacher Evaluations: Built To Fail

With all the controversy and acrimonious debate surrounding the use of value-added models in teacher evaluation, few seem to be paying much attention to the implementation details in those states and districts that are already moving ahead. This is unfortunate, because most new evaluation systems that use value-added estimates are literally being designed to fail.

Much of the criticism of value-added (VA) focuses on systematic bias, such as that stemming from non-random classroom assignment (also here). But the truth is that most of the imprecision of value-added estimates stems from random error. Months ago, I lamented the fact that most states and districts incorporating value-added estimates into their teacher evaluations were not making any effort to account for this error. Everyone knows that there is a great deal of imprecision in value-added ratings, but few policymakers seem to realize that there are relatively easy ways to mitigate the problem.

This is the height of foolishness. Policy is details. The manner in which one uses value-added estimates is just as important – perhaps even more so – than the properties of the models themselves. By ignoring error when incorporating these estimates into evaluation systems, policymakers virtually guarantee that most teachers will receive incorrect ratings. Let me explain.

As Membership Has Declined, Have Attitudes Toward Unions Changed Too?

The sharp decline in U.S. union membership over the past 30-40 years is well known, but does it reflect a change in attitudes towards organized labor? In other words, is decreasing union membership accompanied by decreasing support for labor?

Of course, if attitudes have in fact changed, they might be both exogenous (membership declines because support decreases, leading to fewer unionization drives and less political support) as well as endogenous (support decreases because membership declines, as fewer people are exposed to unions and to the benefits of membership) to unionization levels. And, to some degree, attitudes and membership likely change independent of each other.

In any case, it’s worth taking a look at how attitudes towards labor have changed over the past few decades. In the graph below, I present simple trend data from the General Social Survey (GSS), which has been administered either annually or semi-annually since 1972. Every year, the GSS queries respondents’ confidence in a number of major societal institutions, including organized labor. Granted, there is a difference between having confidence in unions and supporting them per se, but I think it’s safe to assume that the former is a decent indicator of the latter.