• Actual Growth Measures Make A Big Difference When Measuring Growth

    As a frequent critic of how states and districts present and interpret their annual testing results, I am also obliged (and indeed quite happy) to note when there is progress.

    Recently, I happened to be browsing through New York City’s presentation of their 2014 testing results, and to my great surprise, on slide number four, I found proficiency rate changes between 2013 and 2014 among students who were in the sample in both years (which they call “matched changes”). As it turns out, last year, for the first time, New York State as a whole began publishing these "matched" year-to-year proficiency rate changes for all schools and districts. This is an excellent policy. As we’ve discussed here many times, NCLB-style proficiency rate changes, which compare overall rates of all students, many of whom are only in the tested sample in one of the years, are usually portrayed as “growth” or “progress.” They are not. They compare different groups of students, and, as we’ll see, this can have a substantial impact on the conclusions one reaches from the data. Limiting the sample to students who were tested in both years, though not perfect, at least permits one to measure actual growth per se, and provides a much better idea of whether students are progressing over time.

    This is an encouraging sign that New York State is taking steps to improve the quality and interpretation of their testing data. And, just to prove that no good deed goes unpunished, let’s see what we can learn using the new “matched” data – specifically, by seeing how often the matched (longitudinal) and unmatched (cross-sectional) changes lead to different conclusions about student “growth” in schools.

  • Sample Size And Volatility In School Accountability Systems

    It is generally well-known that sample size has an important effect on measurement and, therefore, incentives in test-based school accountability systems.

    Within a given class or school, for example, there may be students who are sick on testing day, or get distracted by a noisy peer, or just have a bad day. Larger samples attenuate the degree to which unusual results among individual students (or classes) can influence results overall. In addition, schools draw their students from a population (e.g., a neighborhood). Even if the characteristics of the neighborhood from which the students come stay relatively stable, the pool of students entering the school (or tested sample) can vary substantially from one year to the next, particularly when that pool is small.

    Classes and schools tend to be quite small, and test scores vary far more between- than within-student (i.e., over time). As a result, testing results often exhibit a great deal of nonpersistent variation (Kane and Staiger 2002). In other words, much of the differences in test scores between schools, and over time, is fleeting, and this problem is particularly pronounced in smaller schools. One very simple, though not original, way to illustrate this relationship is to compare the results for smaller and larger schools.

  • Preparing Effective Teachers For Every Community

    Our guest authors today are Frank Hernandez, Corinne Mantle-Bromley and Benjamin Riley. Dr. Hernandez is the dean of the College of Education at the University of Texas of the Permian Basin, and previously served as a classroom teacher and school and district administrator for 12 years. Dr. Mantle-Bromley is dean of the University of Idaho’s College of Education and taught in rural Idaho prior to her work preparing teachers for diverse K-12 populations. Mr. Riley is the founder of Deans for Impact, a new organization composed of deans of colleges of education working together to transform educator preparation in the US. 

    Students of color in the U.S., and those who live in rural communities, face unique challenges in receiving a high-quality education. All too often, new teachers have been inadequately prepared for these students’ specific needs. Perhaps just as often, their teachers do not look like them, and do not understand the communities in which these students live. Lacking an adequate preparation and the cultural sensitivities that come only from time and experience within a community, many of our nation’s teachers are thrust into an almost unimaginably challenging situation. We simply do not have enough well-prepared teachers of color, or teachers from rural communities, who can successfully navigate the complexities of these education ecosystems.

    Some have described the lack of teachers of color and teachers who will serve in rural communities as a crisis of social justice. We agree. And, as the leaders of two colleges of education who prepare teachers who serve in these communities, we think the solution requires elevating the expectations for every program that prepares teachers and educators in this country.

  • The Debate And Evidence On The Impact Of NCLB

    There is currently a flurry of debate focused on the question of whether “NCLB worked.” This question, which surfaces regularly in the education field, is particularly salient in recent weeks, as Congress holds hearings on reauthorizing the law.

    Any time there is a spell of “did NCLB work?” activity, one can hear and read numerous attempts to use simple NAEP changes in order to assess its impact. Individuals and organizations, including both supporters and detractors of the law, attempt to make their cases by presenting trends in scores, parsing subgroups estimates, and so on. These efforts, though typically well-intentioned, do not, of course, tell us much of anything about the law’s impact. One can use simple, unadjusted NAEP changes to prove or disprove any policy argument. And the reason is that they are not valid evidence of an intervention's effects. There’s more to policy analysis than subtraction.

    But it’s not just the inappropriate use of evidence that makes these “did NCLB work?” debates frustrating and, often, unproductive. It is also the fact that NCLB really cannot be judged in simple, binary terms. It is a complex, national policy with considerable inter-state variation in design/implementation and various types of effects, intended and unintended. This is not a situation that lends itself to clear cut yes/no answers to the “did it work?” question.

  • The Increasing Academic Ability Of New York Teachers

    For many years now, a common talking point in education circles has been that U.S. public school teachers are disproportionately drawn from the “bottom third” of college graduates, and that we have to “attract better candidates” in order to improve the distribution of teacher quality. We discussed the basis for this “bottom third” claim in this post, and I will not repeat the points here, except to summarize that “bottom third” teachers (based on SAT/ACT scores) were indeed somewhat overrepresented nationally, although the magnitudes of such differences vary by cohort and other characteristics.

    A very recent article in the journal Educational Researcher addresses this issue head-on (a full working version of the article is available here). It is written by Hamilton Lankford, Susanna Loeb, Andrew McEachin, Luke Miller and James Wyckoff. The authors analyze SAT scores of New York State teachers over a 25 year period (between 1985 and 2009). Their main finding is that these SAT scores, after a long term decline, improved between 2000 and 2009 among all certified teachers, with the increases being especially large among incoming (new) teachers, and among teachers in high-poverty schools. For example, the proportion of incoming New York teachers whose SAT scores were in the top third has increased over 10 percentage points, while the proportion with scores in the bottom third has decreased by a similar amount (these figures define “top third” and “bottom third” in terms of New York State public school students who took the SAT between 1979 and 2008).

    This is an important study that bears heavily on the current debate over improving the teacher labor supply, and there are few important points about it worth discussing briefly.

  • New York Public Schools And Governor Andrew Cuomo: An Essay, In List Form

    A point-by-point commentary on Governor Andrew Cuomo’s newly-announced education plan.*

    1. New York State now has most racially and economically segregated schools in the nation, worse than Mississippi.
    2. New York is violating Campaign for Fiscal Equity ruling of highest state court to provide full, equitable funding to high poverty schools.
    3. As a result, New York State owes $6 billion it had promised to school districts with concentrations of poverty.
    4. One would think that a Democratic Governor would be focused on correcting such educational injustices.  But not Andrew Cuomo.
    5. Cuomo is proposing tax credits (aka vouchers) that would divert funds and resources from underfunded public schools to private schools.
    6. Poor and working class kids, students of color who attend public schools would be hurt.
    7. Cuomo is 1st ever Democratic Governor to propose tax credits for private schools, says conservative Checker Finn.
    8. League of Women Voters, Civil Liberties Union, school board ass., sup'ts ass't., teachers union all opposed to Cuomo’s tax credit scheme.
    9. The problem with our public schools, Cuomo says, is teachers.
    10. Teachers think: how convenient that Cuomo, who ignores his responsibilities regarding school segregation and funding, blames us.

  • The Persistent Misidentification Of "Low Performing Schools"

    In education, we hear the terms “failing school” and “low-performing school” quite frequently. Usually, they are used in soundbyte-style catchphrases such as, “We can’t keep students trapped in ‘failing schools.’” Sometimes, however, they are used to refer to a specific group of schools in a given state or district that are identified as “failing” or “low-performing” as part of a state or federal law or program (e.g., waivers, SIG). There is, of course, interstate variation in these policies, but one common definition is that schools are “failing/low-performing” if their proficiency rates are in the bottom five percent statewide.

    Putting aside the (important) issues with judging schools based solely on standardized testing results, low proficiency rates (or low average scores) tell you virtually nothing about whether or not a school is “failing.” As we’ve discussed here many times, students enter their schools performing at different levels, and schools cannot control the students they serve, only how much progress those students make while they’re in attendance (see here for more).

    From this perspective, then, there may be many schools that are labeled “failing” or “low performing” but are actually of above average effectiveness in raising test scores. And, making things worse, virtually all of these will be schools that serve the most disadvantaged students. If that’s true, it’s difficult to think of anything more ill-advised than closing these schools, or even labeling them as “low performing.” Let’s take a quick, illustrative look at this possibility using the “bottom five percent” criterion, and data from Colorado in 2013-14 (note that this simple analysis is similar to what I did in this post, but this one is a little more specific; also see Glazerman and Potamites 2011; Ladd and Lauen 2010; and especially Chingos and West 2015).

  • Relationships Matter: Putting It All Together

    About six months ago, we published a post entitled The Importance Of Relationships In Educational Reform, by Kara S. Finnigan and Alan J. Daly. This post was the first of an ongoing series on the social side of education. In addition to Finnigan and Daly, scholars such as Carrie R. Leana and Frits K. Pil, Kenneth Frank, and William Penuel have joined this effort by writing about their research and sharing their perspective.

    If there is one take away about the social side approach, it is the idea that relationships matter in education. Teaching and learning are not solo but rather social endeavors and, as such, they are best achieved by working together. The social side perspective: (1) shifts the focus from the individual to the broader context in which individuals operate; (2) highlights the importance of interdependencies at all levels of the system – e.g., among teachers within a school, leaders across a district, schools and the community; and (3) recognizes that crucial resources (e.g., information, advice, support) are exchanged through interpersonal relationships.

    In my previous post I shared a list of resources (e.g., videos, news articles, papers etc.) that I compiled, and which I will periodically update, on the research underpinning the social side lens. Today I want to share two additional materials: First, a short video that I created, which summarizes, in a visual way, the ideas outlined above; second, an interactive image to help you explore our collection of content on this topic.

  • Who Should Be Allowed To Teach?

    The conventional wisdom is that Americans are becoming more tolerant over time. One of the common ways to measure this tolerance is to ask survey respondents whether they would be willing to have members of different groups – for example, people with different ethnicities, religions, sexual orientations, etc. – serve in positions of societal importance or trust, such as President, family doctor, or, of course, teacher.

    Granted, people are not always forthcoming when asked sensitive questions of this sort, and one should always regard the distribution of responses with caution. That said, from an educational perspective, it might be interesting to take a look at Americans’ stated views about whether members of different groups should be allowed to teach, particularly whether and how these opinions have changed over time.

    The General Social Survey includes several questions about who should be allowed to teach in a college or university, and the survey has asked these questions since 1972. We’ll start with four questions that are worded as follows: “There are always some people whose ideas are considered bad or dangerous by other people. For instance, somebody who is X. Should such a person be allowed to teach in a college or university, or not?"

  • Update On Teacher Turnover In The U.S.

    Every four years, the National Center for Education Statistics provides the public with the best available national estimates of teacher attrition and mobility. The estimates come from the Teacher Follow-Up Survey (TFS), which is a supplement to the Schools and Staffing Survey (SASS), a much larger national survey of teachers that is also conducted every four years. Put simply, the TFS is a sub-sample of SASS respondents, who are contacted the following year to find out if and where they are still teaching.

    The conventional wisdom among many commentators, particularly those critical of test-based accountability and recent education reform, is that teacher attrition (teachers leaving the profession) and mobility (teachers switching schools) are on the rise. As discussed in a previous post, this was indeed the case, at least at the national level, between the 1991-92 and 2004-05 school years, but ceased to be true between 2004-05 and 2008-09, during which time attrition and mobility was basically flat. A few months ago, results from the latest administration of the TFS, which tracked teachers between 2011-12 and 2012-13, were released, and it’s worth taking a quick look at the findings.

    As you can see in the graph below, the proportion of public school teachers who left the profession entirely (“leavers”), as well as the proportion who switched schools (“movers”), were again relatively flat between 2008-09 and 2012-13 (and the change is not statistically significant).