Education Research

Evidence In Education: Is Supply Meeting Teachers' Demand

In today’s public schools, teachers and administrators are constantly pressured to implement new reforms and initiatives, most of which claim to be research- or science-based. Schools are often viewed as the recipients of new policies and new knowledge generated by research. In So Much Reform, So Little Change, Charles Payne states “Best Practice discourse lends itself to decontextualized thinking, reducing the problem of urban schooling to a cognitive one: if only our teachers and principals knew how to do it in the Big City...we should spend some time thinking with school people about what those reasons might be rather than just issuing more exhortations from on high” (p. 63). Including educators in the conversation is crucial for a successful rollout of any evidence-based intervention. Additionally, in order to best support our teachers, it is imperative we develop a system for them to have access to evidence-informed practices that are contextualized and can be tailored to their varying needs. 

While there are many problems with the “research to practice” approach, one barrier is that schools are frequently not given enough training, resources, and support to make sense of educational research. Educators need evidence, innovation, and new approaches; after all, much of their job is to constantly adapt their ways of teaching to meet the unique and changing needs of their students. But right now, too many educators are doing this work without the necessary tools to be successful. In this post I would like to sketch out a tool that could improve access to and applicability of research. By access, I refer not only to the ability of finding and downloading journal articles and other sources, but also to having the knowledge to discern questionable research. According to Paula J. Stanovich and Keith E. Stanovich (2003), a failure to discern good and bad research can cause teachers to try new methods that are not strongly supported by peer-reviewed research. Applicability refers to the extent to which research has practical implications for teachers.

One Page Summaries Of Your State's School Finance System

For the past few years, the Shanker Institute has been collaborating with Bruce Baker and Mark Weber of Rutgers University to publish the School Finance Indicators Database (SFID), a collection of finance and resource allocation measures for policymakers, journalists, parents, and the public. 

The State Indicators Database (SID), the primary product of the SFID, is freely available to the public, but it includes about 125 variables. So, even if you know exactly the types of measures you are looking for, compiling the data for a state or a group of states might present a challenge. While we have tried to make the data accessible for non-researchers, we realize that it can still be difficult for a lot of people. 

We have therefore just published 51 state school finance profiles (with help from ASI fellow Lauren Schneider), which pull together a digestible amount of information into one place for each state (and D.C.). You can download the profiles individually or as a group.

How Much Segregation Is There Within Schools?

Our national discourse on school segregation, whether income- or race-/ethnicity-based, tends to focus on the separation of students between schools within districts. There are good reasons for this, including the fact that the majority of desegregation efforts have been within-district efforts. Sometimes lost in this focus, however, is the importance of segregation between districts.

This distinction can be confusing, so consider a large metro area with a central city district surrounded by a group of suburban districts. There may be extensive racial/ethnic segregation of students between schools within those districts, with students of color concentrated in some schools and their White peers concentrated in others. But total segregation across the entire metro area is also a function of segregation between districts - i.e., the degree to which students of certain races or ethnicities are concentrated in some districts and not others (e.g., students of color in the city, white students in the suburbs). In a sense, if we view diversity as a resource, there are multiple "chokepoints" at which that resource is distributed down to the next level—from states to metro areas to districts to schools—and this can exacerbate segregation.

A recent working paper provides one of relatively few pieces of recent evidence suggesting that, in addition to racial and ethnic segregation between districts and between schools within districts, there may be an additional important "layer": segregation within schools.

Research And Evidence Can Help Guide Teachers During The Pandemic

This post is part of our series entitled Teaching and Learning During a Pandemic, in which we invite guest authors to reflect on the challenges of the Coronavirus pandemic for teaching and learning. Our guests today are Sara Kerr, Vice President of Education Policy Implementation at Results for America, and Nate Schwartz, Professor of Practice at Brown University’s Annenberg Institute for School Reform. Other posts in the series are compiled here.

Teachers are used to playing many different roles, but this year they are facing the most complex challenges of their careers. They are being asked to be public health experts. Tech support specialists. Social workers to families reeling from the effects of layoffs and illness. Masters of distance learning and trauma-responsive educational practices. And they are being asked to take on these new responsibilities against a backdrop of rising COVID-19 cases in many parts of the country, looming budget cuts for many school districts, and a hyper-polarized political debate over the return to school.

To make any of this possible, educators need to be armed with the best available science, data, and evidence, not only about the operational challenges of reopening that have dominated the news cycle but also about how to to meet the increasingly complex social-emotional and academic needs of students and their families. They don’t have time to sift through decades of academic papers for answers. Fortunately, the nation’s education researchers are eager and ready to help.

Why School Climate Matters For Teachers And Students

Our guest authors today are Mathew A. Kraft, associate professor of education and economics at Brown University, and Grace T. Falken, a research program associate at Brown’s Annenberg Institute. This article originally appeared in the May 2020 issue of The State Education Standard, the journal of the National Association of State Boards of Education.

Over the past decade, education reformers have focused much of their attention on raising teacher quality. This makes sense, given the well-evidenced, large impacts teachers have on student outcomes and the wide variation in teacher effectiveness, even within the same school (Goldhaber 2015; Jackson et al. 2014). Yet this focus on individual teachers has caused policymakers to lose sight of the importance of the organizational contexts in which teachers work and students learn. 

The quality of a school’s teaching staff is greater than the sum of its parts. School environments can enable teachers to perform to their fullest potential or undercut their efforts to do so. 

When we think of work environments, we often envision physical features: school facilities, instructional resources, and the surrounding neighborhood. State and district policies that shape curriculum standards, class size, and compensation also come to mind. These things matter, but so do school climate factors that are less easily observed or measured. Teachers’ day-to-day experiences are influenced most directly by the culture and interpersonal environment of their schools.

Interpreting School Finance Measures

Last week we released the second edition of our annual report, "The Adequacy and Fairness of State School Finance Systems," which presents key findings from the School Finance Indicators Database (SFID). The SFID, released by the Shanker Institute and Rutgers Graduate School of Education (with my colleagues and co-authors Bruce Baker and Mark Weber), is a free collection of sophisticated finance measures that are designed to be accessible to the public. At the SFID website, you can read the summary of our findings, download the full report and datasets, or use our online data visualization tools.

The long and short of the report is that states vary pretty extensively, but most fund their schools either non-progressively (rich and poor districts receive roughly the same amount of revenue) or regressively (rich districts actually receive more revenue), and that, in the vast majority of states, funding levels are inadequate in all but the most affluent districts (in many cases due to a lack of effort).

One of the difficulties in producing this annual report is that the our "core" measures upon which we focus (effort, adequacy, and progressivity) are state-level, and it's not easy to get attention for your research report when you basically have 51 different sets of results. One option is assigning states grades, like a school report card. Often, this is perfectly defensible and useful. We decided against it, not only because assigning grades would entail many arbitrary decisions (e.g., where to set the thresholds), but also because assigning grades or ratings would risk obscuring some of the most useful conclusions from our data. Let's take a quick look at an example of how this works.

Interpreting Effect Sizes In Education Research

Interpreting “effect sizes” is one of the trickier checkpoints on the road between research and policy. Effect sizes, put simply, are statistics measuring the size of the association between two variables of interest, often controlling for other variables that may influence that relationship. For example, a research study may report that participating in a tutoring program was associated with a 0.10 standard deviation increase in math test scores, even controlling for other factors, such as student poverty, grade level, etc.

But what does that mean, exactly? Is 0.10 standard deviations a large effect or a small effect? This is not a simple question, even for trained researchers, and answering it inevitably entails a great deal of subjective human judgment. Matthew Kraft has an excellent little working paper that pulls together some general guidelines and a proposed framework for interpreting effect sizes in education. 

Before discussing the paper, though, we need to mention what may be one of the biggest problems with the interpretation of effect sizes in education policy debates: They are often ignored completely.

The Offline Implications Of The Research About Online Charter Schools

It’s rare to find an educational intervention with as unambiguous a research track record as online charter schools. Now, to be clear, it’s not a large body of research by any stretch, its conclusions may change in time, and the online charter sub-sector remains relatively small and concentrated in a few states. For now, though, the results seem incredibly bad (Zimmer et al. 2009; Woodworth et al. 2015). In virtually every state where these schools have been studied, across virtually all student subgroups, and in both reading and math, the estimated impact of online charter schools on student testing performance is negative and large in magnitude.

Predictably, and not without justification, those who oppose charter schools in general are particularly vehement when it comes to online charter schools – they should, according to many of these folks, be closed down, even outlawed. Charter school supporters, on the other hand, tend to acknowledge the negative results (to their credit) but make less drastic suggestions, such as greater oversight, including selective closure, and stricter authorizing practices.

Regardless of your opinion on what to do about online charter schools’ poor (test-based) results, they are truly an interesting phenomenon for a few reasons.

We Need To Reassess School Discipline

It has been widely documented that, in American schools, students of color are disproportionately punished for nonviolent behaviors, and are targeted for exclusionary discipline within schools more often than their white peers. Exclusionary discipline is defined as students being removed from their learning environment, whether by in-school suspension, out-of-school suspension, or expulsion. 

In a national study, Sullivan et al. (2013) found that “Black students were more than twice as likely as White students to be suspended, whereas Hispanic and Native American students were 10 and 20 percent more likely to be suspended.” Out of all the racial minority groups, Asians had the lowest suspension rates across the board. Across all the racial groups, “males were twice as likely as female students to be suspended, and Black males had the highest rates of all subgroups.”

One reason that students of color are at a performance disadvantage to their White counterparts is because, put simply, they are being removed from the classroom much more often. This is true nationally, but it seems to be a particularly pronounced issue in the Commonwealth of Virginia. The Center for Public Integrity released a 2015 study demonstrating that schools in Virginia “referred students to law enforcement agencies at a rate nearly three times the national rate” (Ferriss, 2015). According to the U.S. Department of Education, Virginia’s Black student population, which is 23 percent of all students, received 59 percent of short-term arrests and 43 percent of expulsions (Lum, 2018).

Weaning Educational Research Off Of Steroids

Our guest authors today are Hunter Gehlbach and Carly D. Robinson. Gehlbach is an associate professor of education and associate dean at the University of California, Santa Barbara’s Gevirtz Graduate School of Education, as well as Director of Research at Panorama Education. Robinson is a doctoral candidate at Harvard’s Graduate School of Education.

Few people confuse academics with elite athletes. As a species, academics are rarely noted for their blinding speed, raw power, or outrageously low resting heart rates. Nobody wants to see a calendar of scantily clad professors. Unfortunately, recent years have surfaced one commonality between these two groups—a commonality no academic will embrace. And one with huge implications for educational policymakers’ and practitioners’ professional lives.

In the same way that a 37 year-old Barry Bonds did not really break the single-season home run record—he relied on performance-enhancing drugs—a substantial amount of educational research has undergone similar “performance enhancements” that make the results too good to be true.

To understand, the crux of the issue, we invite readers to wade into the weeds (only a little!), to see what research “on steroids” looks like and why it matters. By doing so, we hope to reveal possibilities for how educational practitioners and policymakers can collaborate with researchers to correct the problem and avoid making practice and policy decisions based on flawed research.