• The Ever-Changing NAEP Sample

    The results of the latest National Assessment of Educational Progress long term trend tests (NAEP-LTT) were released last week. The data compare the reading and math scores of 9-, 13- and 17-year olds at various points since the early 1970s. This is an important way to monitor how these age cohorts’ performance changes over the long term.

    Overall, there is ongoing improvement in scores among 9- and 13-year olds, in reading and especially math, though the trend is inconsistent and increases are somewhat slow in recent years. The scores for 17-year olds, in contrast, are relatively flat.

    These data, of course, are cross-sectional – i.e., they don’t follow students over time, but rather compare children in the three age groups with their predecessors from previous years. This means that changes in average scores might be driven by differences, observable or unobservable, between cohorts. One of the simple graphs in this report, which doesn't present a single test score, illustrates that rather vividly.

  • A Few Points About The New CREDO Charter School Analysis

    A new report from CREDO on charter schools’ test-based performance received a great deal of attention, and rightfully so - it includes 27 states, which together serve 95 percent of the nation's charter students.

    The analysis as a whole, like its predecessor, is a great contribution. Its sheer scope, as well as a few specific parts (examination of trends), are new and important. And most of the findings serve to reaffirm the core conclusions of the existing research on charters' estimated test-based effects. Such an interpretation may not be particularly satisfying to charter supporters and opponents looking for new ammunition, but the fact that this national analysis will not settle anything in the contentious debate about charter schools once again suggests the need to start asking a different set of questions.

    Along these lines, as well as others, there are a few points worth discussing quickly. 

  • No Presentation Without Representation

    I tend to comment on newly-released teacher surveys, primarily because I think the surveys are important and interesting, but also because teachers' opinions are sometimes misrepresented in our debate about education reform. So, last year, I wrote about a report by the advocacy organization Teach Plus, in which they presented results from a survey focused on identifying differences in attitudes by teacher experience (an important topic). One of my major comments was that the survey was "non-scientific" – it was voluntary, and distributed via social media, e-mail, etc. This means that the results cannot be used to draw strong conclusions about the population of teachers as a whole, since those who responded might be different from those that did not.

    I also noted that, even if the sample was not representative, this did not preclude finding useful information in the results. That is, my primary criticism was that the authors did not even mention the issue, or make an effort to compare the characteristics of their survey respondents with those of teachers in general (which can give a sense of the differences between the sample and the population).

    Well, they have just issued a new report, which also presents the results of a teacher survey, this time focused on teachers’ attitudes toward the evaluation system used in Memphis, Tennessee (called the “Teacher Effectiveness Measure," or TEM). In this case, not only do they raise the issue of representativeness, but they also present a little bit of data comparing their respondents to the population (i.e., all Memphis teachers who were evaluated under TEM).

  • What Some Call Delay Is At Times Just Good Policy Making

    U.S. Secretary of Education Arne Duncan recently announced that states will be given the option to postpone using the results of their new teacher evaluations for high-stakes decisions during the phase-in of the new Common Core-aligned assessments. The reaction from some advocates was swift condemnation – calling the decision little more than a “delay” and a “victory for the status quo."

    We hear these kinds of arguments frequently in education. The idea is that change must be as rapid as possible, because “kids can’t wait." I can understand and appreciate the urgency underlying these sentiments. Policy change in education (as in other arenas) can sometimes be painfully slow, and what seem likes small roadblocks can turn out to be massive, permanent obstacles.

    I will not repeat my views regarding the substance of Secretary Duncan’s decision – see this op-ed by Morgan Polikoff and myself. I would, however, like to make one very quick point about these “we need change right now because students can’t wait” arguments: Sometimes, what is called “delay” is actually better described as good policy making, and kids can wait for good policy making.

  • What Should The Results Of New Teacher Evaluations Look Like?

    In a previous post, I discussed the initial results from new teacher evaluations in several states, and the fact that states with implausibly large proportions of teachers in the higher categories face a difficult situation – achieving greater differentiation while improving the quality and legitimacy of their systems.

    I also expressed concern that pre-existing beliefs about the "proper" distribution of teacher ratings -- in particular, how many teachers should receive the lowest ratings -- might inappropriately influence the process of adjusting the systems based on the first round of results. In other words, there is a risk that states and districts will change their systems in a crude manner that lowers ratings simply for the sake of lowering ratings.

    Such concerns of course imply a more general question: How should we assess the results of new evaluation systems? That’s a complicated issue, and these are largely uncharted waters. Nevertheless, I'd like to offer a few thoughts as states and districts move forward.

  • Charter School Authorization And Growth

    If you ask a charter school supporter why charter schools tend to exhibit inconsistency in their measured test-based impact, there’s a good chance they’ll talk about authorizing. That is, they will tell you that the quality of authorization laws and practices -- the guidelines by which charters are granted, renewed and revoked -- drives much and perhaps even most of the variation in the performance of charters relative to comparable district schools, and that strengthening these laws is the key to improving performance.

    Accordingly, a recently-announced campaign by the National Association of Charter School Authorizers aims to step up the rate at which charter authorizers close “low-performing schools” and are more selective in allowing new schools to open. In addition, a recent CREDO study found (among other things) that charter middle and high schools’ performance during their first few years is more predictive of future performance than many people may have thought, thus lending support to the idea of opening and closing schools as an improvement strategy.

    Below are a few quick points about the authorization issue, which lead up to a question about the relationship between selectivity and charter sector growth.

  • Relationship Counseling

    A correlation between two variables measures the strength of the linear relationship between them. Put simply, two variables are positively correlated to the extent that individuals with relatively high or low values on one measure tend to have relatively high or low values on the other, and negatively correlated to the extent that high values on one measure are associated with low values on the other.

    Correlations are used frequently in the debate about teacher evaluations. For example, researchers might assess the relationship between classroom observations and value-added measures, which is one of the simpler ways to gather information about the “validity” of one or the other – i.e., whether it is telling us what we want to know. In this case, if teachers with higher observation scores also tend to get higher value-added scores, this might be interpreted as a sign that both are capturing, at least to some extent, "true" teacher performance.

    Yet there seems to be a tendency among some advocates and policy makers to get a little overeager when interpreting correlations.

  • Unreliable Sources: Education Revenue During The Recession

    For the better part of the past century, U.S. public education revenue has come predominantly from state and local sources, with the federal government contributing only a relatively small share. For most of this time, local revenue (primarily property taxes) comprised the largest proportion, but this began to shift gradually during the 1970s, to the point where state funds constituted a slightly larger share of overall revenue.

    As you can see in the simple graph below, which uses data from the U.S. Census Bureau, this situation persisted throughout the 1990s and most of the 2000s. During this period, states provided roughly 50 percent of total revenue, localities about 45 percent, and the federal government approximately 5-8 percent. Needless to say, these overall proportions varied quite a bit by state. Vermont represents one of the most extreme examples, where, as a result of a 1997 State Supreme Court decision, education funding comes almost entirely from the state. Conversely, since Hawaii’s education system consists of a single statewide district, revenue on paper is dominated by state sources (though, in Hawaii's case, you might view the state and local levels as the same).

    That said, the period of 2008 to 2010 was a time of pretty sharp volatility in the overall proportions contributed by each level of government.

  • Red Card For Morsi

    Our guest author today is Heba F. El-Shazli. Currently a visiting professor at Georgetown University’s Center for Democracy and Civil Society, Prof. El-Shazli  has 25 years of international experience in political and economic development, including democracy promotion programs and  support  for independent trade unions. She has worked with trade unions, political parties, and leaders throughout the Middle East and North Africa (MENA). The views expressed here are her own.

    Egypt has become a land of stark, astounding contradictions. There is so little stability that political pundits cannot predict or even explain what is happening in the political sphere or society. As it is in politics, it is in daily life. Some examples:

    Driving in Cairo has become the art of the detour. Demonstrations and protests can materialize any time of day and anywhere. If a protesting mass of humanity does not impede your way, then one of the new cement block walls will. These walls are erected by the government across major streets to contain demonstrations. Surely the time is right for some inventor to launch a new app -- “Avoid Protests - Egypt” -- to guide drivers and protesters alike.

    Times are not tough for everyone. In the affluent areas of Cairo, youngsters still sport the latest fashions, shoppers carry home their purchases, and restaurants brim with high class clientele. New restaurants are opening and one even sells fancy “cupcakes," the craze imported from the US. One would hardly believe that, for the average Egyptian, the country is indeed sinking into a financial abyss. But that is what's happening.

  • The FCAT Writing, On The Wall

    The annual release of state testing data makes the news in every state, but Florida is one of those places where it is to some degree a national story.*

    Well, it’s getting to be that time of year again. Last week, the state released its writing exam (FCAT 2.0 Writing) results for 2013 (as well as the math and reading results for third graders only).  The Florida Department of Education (FLDOE) press release noted: “With significant gains in writing scores, Florida’s teachers and students continue to show that higher expectations and support at home and in the classroom enable every child to succeed.” This interpretation of the data was generally repeated without scrutiny in the press coverage of the results.

    Putting aside the fact that the press release incorrectly calls the year-to-year changes “gains” (they are actually comparisons of two different groups of students; see here), the FLDOE's presentation of the FCAT Writing results, though common, is, at best, incomplete and, at worst, misleading. Moreover, the important issues in this case are applicable in all states, and unusually easy to illustrate using the simple data released to the public.