Shanker Blog

Weaning Educational Research Off Of Steroids

Our guest authors today are Hunter Gehlbach and Carly D. Robinson. Gehlbach is an associate professor of education and associate dean at the University of California, Santa Barbara’s Gevirtz Graduate School of Education, as well as Director of Research at Panorama Education. Robinson is a doctoral candidate at Harvard’s Graduate School of Education.

Few people confuse academics with elite athletes. As a species, academics are rarely noted for their blinding speed, raw power, or outrageously low resting heart rates. Nobody wants to see a calendar of scantily clad professors. Unfortunately, recent years have surfaced one commonality between these two groups—a commonality no academic will embrace. And one with huge implications for educational policymakers’ and practitioners’ professional lives.

In the same way that a 37 year-old Barry Bonds did not really break the single-season home run record—he relied on performance-enhancing drugs—a substantial amount of educational research has undergone similar “performance enhancements” that make the results too good to be true.

To understand, the crux of the issue, we invite readers to wade into the weeds (only a little!), to see what research “on steroids” looks like and why it matters. By doing so, we hope to reveal possibilities for how educational practitioners and policymakers can collaborate with researchers to correct the problem and avoid making practice and policy decisions based on flawed research.

The Teacher Diversity Data Landscape

This week, the Albert Shanker Institute released a new research brief, authored by myself and Klarissa Cervantes. It summarizes what we found when we contacted all 51 state education agencies (including the District of Columbia) and asked whether data on teacher race and ethnicity was being collected, and whether and how it was made available to the public. This survey was begun in late 2017 and completed in early 2018.

The primary reason behind this project is the growing body of research to suggest that all students, and especially students of color, benefit from a teaching force that reflects the diverse society in which they must learn to live, work and prosper. ASI’s previous work has also documented that a great many districts should turn their attention to recruiting and retaining more teachers of color (see our 2015 report). Data are a basic requirement for achieving this goal – without data, states and districts are unable to gauge the extent of their diversity problem, target support and intervention to address that problem, and monitor the effects of those efforts. Unfortunately, the federal government does not require that states collect teacher race and ethnicity data, which means the responsbility falls to individual states. Moreover, statewide data are often insufficient – teacher diversity can vary widely within and between districts. Policymakers, administrators, and the public need detailed data (at least district-by-district and preferably school-by-school), which should be collected annually and be made easily available.

The results of our survey are generally encouraging. The vast majority of state education agencies (SEAs), 45 out of 51, report that they collect at least district-by-district data on teacher race and ethnicity (and all but two of these 45 collect school-by-school data). This is good news (and, frankly, better results than we anticipated). There are, however, areas of serious concern.

Teaching – And Defending – American Democracy

If recent history demonstrates anything, it is the old truth that American democracy is a work in progress, and that it can suffer reversals as well as advances. The teaching of civics in our schools should convey the complex and fluid character of American government, and the concurrent responsibility of citizens to be actively involved in politics in order to defend and expand the rights and freedoms of American democracy. At a moment of great risk for democracy, both in the United States and abroad, it is especially important for young people to understand that the moral arc of history does not bend on its own, but only by the active intervention of ordinary people. We may still have a republic, even in this moment of dangerous turmoil, but—as Benjamin Franklin famously opined—only if the citizenry can keep it.

Seen in this light, the crash course on how to teach civics offered by the Fordham Institute’s Checker Finn is an exemplar of what NOT to do. In an age of the rise of authoritarian and racist populisms of the far right, including that found at the pinnacle of American government, Finn is exercised about the emergence of an embryonic democratic socialist current in American politics. Of particular concern is what he sees as an “appalling” New York Times op-ed by two young editors of the socialist journal Jacobin, which argued that “subversion of democracy was the explicit intent of the framers” of the Constitution, and advocated constitutional reform to make the American system more democratic. 

The idea that the 1789 Constitution contained significant anti-democratic elements seems to be anathema to Finn. Armed with an exegesis of Federalist Paper 10 which misses the essence of James Madison’s argument, he asserts that the purpose of the Constitution was the promotion and defense of democracy, full stop, and that is how it must be taught in civics courses.

Why Teacher Evaluation Reform Is Not A Failure

The RAND Corporation recently released an important report on the impact of the Gates Foundation’s “Intensive Partnerships for Effective Teaching” (IPET) initiative. IPET was a very thorough and well-funded attempt to improve teaching quality in schools in three districts and four charter management organizations (CMOs). The initiative was multi-faceted, but its centerpiece was the implementation of multi-measure teacher evaluation systems and the linking of ratings from those systems to professional development and high stakes personnel decisions, including compensation, tenure, and dismissal. This policy, particularly the inclusion in teacher evaluations of test-based productivity measures (e.g., value-added scores), has been among the most controversial issues in education policy throughout the past 10 years.

The report is extremely rich and there's a lot of interesting findings in there, so I would encourage everyone to read it themselves (at least the executive summary), but the headline finding was that the IPET had no discernible effect on student outcomes, namely test scores and graduation rates, in the districts that participated, vis-à-vis similar districts that did not. Given that IPET was so thoroughly designed and implemented, and that it was well-funded, it can potentially be viewed as a "best case scenario" test of the type of evaluation reform that most states have enacted. Accordingly, critics of these reforms, who typically focus their opposition on the high stakes use of evaluation measures, particularly value-added and other test-based measures, in these evaluations, have portrayed the findings as vindication of their opposition. 

This reaction has merit. The most important reason why is that evaluation reform was portrayed by advocates as a means to immediate and drastic improvements in student outcomes. This promise was misguided from the outset, and evaluation reform opponents are (and were) correct in pointing this out. At the same time, however, it would be wise not to dismiss evaluation reform as a whole, for several reasons, a few of which are discussed below.

What Do Teachers Think About How Teachers' Unions Affect Schools?

You don’t have to follow education policy debates for long to notice that teachers’ unions invoke a wide range of opinions; they are admired by some, and abhorred by others. Although they are often portrayed as one big monolithic organization (“the teachers’ union”), there are in fact over ten thousand local teachers’ unions across the nation. All of them are led by teachers who are elected by teachers, their contracts are approved by teachers, and their policy advocacy tends to reflect the preferences of their members, who are teachers.

Given that the vast majority of U.S. students are educated in public schools, which are funded by tax dollars, it makes sense that public scrutiny of teachers’ unions tends to be more extensive than it is for, say, steel or auto worker unions. The leaders of teachers’ unions and their members understand (even welcome) this. Communicating with parents and the community is a big part of being a teacher. People are very serious about education, and rightfully so. Everyone should participate in the debate over how to run our schools. I dare say most teachers would agree that this debate, while sometimes overly contentious, has a net positive effect.

Yet all the debate about the effect of teachers’ unions often omits an interesting (and, perhaps, important) question: What do teachers think about this issue? You don’t have to agree with all policy stances taken by teachers’ unions; such disagreement is not “teacher bashing,” as is sometimes alleged. If, however, you respect teachers and their opinions about how to run schools, and if teachers tend to agree with their unions, then it makes sense at least to keep this in mind when expressing opinions about their unions.

We Can't Graph Our Way Out Of The Research On Education Spending

The graph below was recently posted by U.S. Education Department (USED) Secretary Betsy DeVos, as part of her response to the newly released scores on the 2017 National Assessment of Educational Progress (NAEP), administered every two years and often called the “nation’s report card.” It seems to show a massive increase in per-pupil education spending, along with a concurrent flat trend in scores on the fourth grade reading version of NAEP. The intended message is that spending more money won’t improve testing outcomes. Or, in the more common phrasing these days, "we can't spend our way out of this problem."

Some of us call it “The Graph.” Versions of it have been used before. And it’s the kind of graph that doesn’t need to be discredited, because it discredits itself. So, why am I bothering to write about it? The short answer is that I might be unspeakably naïve. But we’ll get back to that in a minute.

First, let’s very quickly run through the graph. In terms of how it presents the data, it is horrible practice. The double y-axes, with spending on the left and NAEP scores on the right, are a textbook example of what you might call motivated scaling (and that's being polite). The NAEP scores plotted range from a minimum of 213 in 2000 to a maximum of 222 in 2017, but the graph inexplicably extends all the way up to 275. In contrast, the spending scale extends from just below the minimum observation ($6,000) to just above the maximum ($12,000). In other words, the graph is deliberately scaled to produce the desired visual effect (increasing spending, flat scores). One could very easily rescale the graph to produce the opposite.

Where Al Shanker Stood: Labor Law Reform

This month marks the 50th anniversary of the assassination of Martin Luther King, Jr. in Memphis, Tennessee, where he was working in support of the union rights of striking African American sanitation workers. We thought it was an opportune time to reprint this July 17, 1977 piece, in which Al Shanker turned over his weekly column to his friend and mentor Bayard Rustin, advisor to King on nonviolent protest strategies, chief organizer of the 1963 March on Washington, and founding president of the A. Philip Randolph Institute.

The nation's labor laws need to be reformed to give workers a fair chance to organize. Enlightened opinion has long recognized that unions are essential if workers are to have any hope of dealing on an equal basis with their employers.

The nation's basic labor relations policy was expressed in the Wagner Act of 1935 as "encouraging the practice and procedure of collective bargaining" and "protecting the exercise by workers of full freedom of association, self-organization and designation of representatives of their own choosing." The Taft-Hartley and Landrum-Griffin amendments to the Wagner Act undermined those principles by creating an imbalance in favor of employers.

Although companies no longer employ the brutal anti-union methods of the past, many have adopted a sophisticated arsenal of devices -- legal, illegal, and extralegal -- to interfere with and frustrate the rights of workers to organize and bargain collectively.

What Happened To Teacher Quality?

Starting around 2005 and up until a few years ago, education policy discourse and policymaking was dominated by the issue of improving “teacher quality.” We don’t really hear too much about it the past couple of years, or at least not nearly as much. One of the major reasons why is that the vast majority of states have enacted policies ostensibly designed to improve teacher quality.

Thanks in no small part to the Race to the Top grant program, and the subsequent ESEA waiver program, virtually all states reformed their teacher evaluation systems, the “flagship” policy of the teacher quality push. Many of these states also tied their new evaluation results to high stakes personnel decisions, such as granting tenure, dismissals, layoffs, and compensation. Predictably, the details of these new systems vary quite a bit, both within and between states. Many advocates are unsatisfied with how the new policies were designed, and one could write a book on all the different issues. Yet it would be tough to deny that this national policy effort was among the fastest shifts in recent educational history, particularly given the controversy surrounding it.

So, what happened to all the attention to teacher quality? It was put into practice. The evidence on its effects is already emerging, but this will take a while, and so it is still a quiet time in teacher quality land, at least compared to the previous 5-7 years. Even so, there are already many lessons out there, too many for a post. Looking back, though, one big picture lesson – and definitely not a new one – is about how the evaluation reform effort stands out (in a very competitive field) for the degree to which it was driven by the promise of immediate, large results.

Let's Not Forget About Dreamers

March has arrived, and there is still no action on DACA. Around 800,000 people remain in limbo, those who voluntarily registered under the provisions of President Obama’s Deferred Action for Childhood Arrivals. Let me repeat: 800,000 people are at risk – people who grew up in this country as Americans, people who might very well be your friends, colleagues, students, classmates, and neighbors and you might not even know. They have had moments of hope and promise taken from them repeatedly. From one moment to the next, their lives change on a political whim.

Since before President Trump took office, there have been promises of bipartisan legislation. Under the Trump administration, there have been debates and stalemates, budget fights and threats of government shutdowns, and yet nothing has been done. DACA’s Dreamers have been used as leveraging tools in an attempt to secure money for “the wall” along the Mexican border and stricter immigration laws. They are being treated as bargaining chips and not as human beings. It is time for a clean “Dream Act”. The time for pointing fingers is over. How much longer do Dreamers have to wait in uncertainty, fearing that their lives may change every time there is a new U.S. president?

When the program was rescinded in September 2017, Attorney General Sessions said the following: “The effect of this unilateral executive amnesty, among other things, contributed to a surge of unaccompanied minors on the southern border that yielded terrible humanitarian consequences. It also denied jobs to hundreds of thousands of Americans by allowing those same jobs to go to illegal aliens” (Full speech here). This was misleading on many levels.