Revisiting The Effect Of Teachers' Unions On Student Test Scores
The Wisconsin protests have predictably spurred a great deal of information-seeking, with union supporters and opponents alike searching for evidence that supports their cases. One of the most prevalent topics over the past week or so is the effect of teacher collective bargaining on student test scores. As a result, a couple of our previous posts have been shared widely. The first (also republished here) compares NAEP scores in states that allow binding teacher contracts with those in states that do not (or have only one or two); the second, follow-up post offers some additional, multivariate analysis.
Although it is true that the first post (which was at least partially satirical - see the last few sentences) shows that states without binding contracts are among the lowest-performing in the nation, I want to clear something up: As I noted in both posts, neither the data nor my argument offer any conclusive proof that teacher contracts act to increase student test scores. The intention of those posts was to address the age-old counter claim – that teacher contracts are somehow injurious to student achievement – and to provide very tentative evidence that the contracts appear to have little discernible impact either way (which is what the follow-up post, using state-level models that controlled for basic student characteristics, indicated, along with the requisite caveats).
This speaks directly to those who seek to blame unions for poor achievement in the U.S. - if union contracts were in fact a major contributing cause of low test performance, it might be reasonable to expect to find at least some clear differences between states that did and did not allow them. Although my analysis was extremely limited, I found no such evidence.
But this also applies to those who have been claiming recently – many in the Wisconsin context – that teacher bargaining clearly improves these outcomes.
A few such arguments made the rounds over the past week. They were addressed thoughtfully on one blog, which examined the validity of the data, and pointed out (correctly) that it remained unclear whether differences were attributable to “union effects” or to student characteristics. The situation, however, is actually even more complicated that that.
Establishing a direct causal relationship between union contracts and achievement is extremely difficult, especially at the state-level. There is tremendous variation within and between states in the strength of individual locals (there are thousands) and in the terms of the contracts they have negotiated, to say nothing of all the other factors that might influence achievement (e.g., student/family characteristics, school-level policies/practices, resources, etc.). Such differences usually seem to elude those who seek to blame "the teachers' union" for various problems.
Making things worse, even some states that don’t allow bargaining or contracts do have policies commonly associated with unions, such as seniority, and unions can affect policy outside of the collective bargaining process (for example, by waging public campaigns). Finally, of course, teachers in many unionized districts aren’t union members, so a simple “yes/no” categorization is a rather imperfect measure of union strength, even at the district-level. The manner in which one measures union presence or strength is critically important. (For our posts, I divided states by whether or not states allow binding contracts, since there are a few states in which districts are not required to bargain with teachers, but contracts are legally-binding if the two sides complete the process; simpler schemes, such as CB/non-CB, miss this.)
Nevertheless, there is actually a bunch of research studies on the union/test score relationship. For instance, one review of 17 previous analyses concluded that most (but not all) show unions with a small positive benefit overall, but a possible negative effect for the lowest-performing students. Results varied by the outcome used (e.g., graduation rates, SAT/ACT, etc.), and by other characteristics, such as race.
Generally, though, the body of evidence is inconclusive (see chapter six in this book) and there is little basis for strong arguments either way.
(Side note: Even if there was solid evidence, one could not reasonably blame unions without placing equal blame on the administrators who bargained and approved the contracts. It is remarkably ironic – and hypocritical – for some superintendents to criticize unions for the policies that they themselves negotiated. Similarly, it is important to realize that many policies supported by teachers’ unions, such as peer review, might have a positive effect if implemented more widely.)
Some union opponents, on the other hand, focus less on achievement and more on their fiscal impact – i.e., teacher contracts lead to higher pay and benefits, which boosts costs for states and districts (or shifts costs to less productive inputs). There are also, of course, serious impediments to this line of research. Differences within and between states in the cost of living, state pension systems, membership structures, and other factors are difficult to control for (i.e., more aggregate data bias results). In addition, even if unions did increase costs, this might still be offset by positive financial (and educational) benefits – for example, improved organizational efficiency, better communication, or more standardized practices.
Accordingly, most studies of union effects on teacher compensation and overall costs find at least some positive association (see the review of in this older paper), but it is not entirely clear the degree to which this is a direct causal effect of unions per se. For instance, besides the aforementioned measurement and contextual issues, it is tough to prove that unions increase costs or affect productivity unless you can look at outcomes before and after unionization, both within states and between them. One very influential study that examined the union/cost relationship (as well as dropout rates) over time found that unionization increased pay and overall expenditures, while also increasing dropout rates. A more recent, important study, in contrast, employed hand-gathered union certification data (which the authors argue is more accurate), and found little effect on either pay or dropout rates, along with only short-run effects on overall costs.
So, overall, it is remarkably difficult to isolate union effects or how they might arise. The evidence is mixed and inconclusive, especially for student achievement, and any strong, blanket statements – whether for or against unions – should be taken with a grain of salt. They make for good talking points, but their evidentiary basis is, in most cases, shaky.
What is clear is that unions do accomplish other goals – giving teachers (and other workers) a voice in their profession, their compensation, and their working conditions. On this score, as the Wisconsin protests demonstrate, the evidence is rather compelling.
I'm betting there's a stronger correlation between state's $$ spent per pupil AND parent education level achieved in the home and student test scores than anything else.
This is probably followed by the economic condition of the area around students' home.
There is a very strong correlation between student participation rates in SAT tests and average scores. The scores of students in Maine ranked dead last in the country, but 92% of the kids there took the test. In Iowa, ranked first, only 4% of the kids took the test. What is most interesting is that teacher unions have historically opposed mandatory standardized testing. So maybe there is indeed a cause-effect, just not quite what it first appeared to be.
"On this score, as the Wisconsin protests demonstrate, the evidence is rather compelling."
you left us hanging. how is the evidence rather compelling? that a group of people gathered to protest? that takes a union? were the millions who protested the iraq invasion all union members? and what about unions with collective bargaining vs without? is it still possible to have a voice in compensation and conditions without collective bargaining?
"there is little basis for strong arguments either way." but if the student results are at best slightly improved by collective bargaining, does the value merit the increase in pay and compensation? are taxpayers getting their money's worth?
please don't leave us hanging!
other questions worth considering:
how do various union/nonunion schools perform vs private schools vs home schooling? after all, long before there was a public school system, people were being educated.
to focus on unions/nonunions and to compare public school systems as if they were the only option seems to leave out the larger questions regarding education in general. is the taxpayer getting fair market value with public schools? are they worth the system and its associated costs, structures, and standards? is society actually better off with or without public schools (or, is less more)? what would fill the vacuum if public schools didn't exist? i'm guessing education would move away from the current structured, uniformity (in public and many private schools) to something more free-form (or, this being the computer age, cloud based). it would seem that there would be more diversity in education. which begs the questions: how important are consistent standards, and which ones have merit?
i'm sure these and similar questions have been asked before, revisited, and reevaluated within and without the education field. lots to be answered for sure, and surely some of the answers are still in the process of being revealed.
"Making things worse, even some states that don’t allow bargaining or contracts do have policies commonly associated with unions, such as seniority, and unions can affect policy outside of the collective bargaining process (for example, by waging public campaigns."
This point is worth emphasizing, given how many ideologues there are who try to compare education in union and "non-union" states, as if that meant anything.
As Kevin Williamson recently wrote:
" In Texas, for instance, the teachers’ unions are an extraordinarily powerful political force, with the Texas State Teachers’ Association running an influential PAC that reliably doles out great heaps of money, largely to Democrats, in multi-thousand-dollar increments. How powerful is the TSTA? Powerful enough that it was able to persuade school districts to use their payroll departments to collect PAC donations out of teachers’ paychecks, in violation of state law. Meditate on that for a second: These weren’t union dues being deducted out of government employees’ paychecks, but PAC donations. This is the equivalent of the Environmental Protection Agency deducting contributions to the coal lobby’s PAC out of its employees’ paychecks — and this is in Texas, where the government unions are allegedly as defenseless as newborn babes in the woods. Texas’s teachers’ unions, like their counterparts across the country, have been extraordinarily effective in fighting most meaningful school-reform efforts, from vouchers to school choice to greater accountability measures — and all of this without collective-bargaining power."
I agree w/the comment contributed by ateacherinfl. The discussion concerning unions and test scores is not beneficial to improving student performance. There is very little discussion re: the impact of the home environment and parents in a child's academic performance. I think that is in part due to the fact that no one is comfortable saying that many parents of students in low performing urban schools are not providing a suitable environment. It is a touchy subject due to the fact that our kids with the lowest scores are African American & Latino-for the most part.
Before anyone fires off an message saying I harbor prejudice against African Americans & Latinos let me inform you that 1)I am an immigrant (came to America at age 8 44 yrs ago from Latin America of African descent& 2)I tutor a very bright African American 6th grader who reads at the 4th grade level.
What I have observed from working with "Henry" is that his family cares about hime, but they have NO IDEA of what they need to do to help him reach his full potential. Imagine, no one supervised homework or checked it. I sit with him as my mother (a teacher) did with me and check work as he is doing it.
Now this family has an excellent work ethic. One of Henry's aunt's is a chemical engineer and the other aunt is a financial analyst at a major commercial real estate firm. His dad had difficulties with the law as a younger man, but he is a whiz with mechanical things. He works in the building trades & is in a union. He went to private Catholic school but did poorly due to a very bad home life with his father who was a workoholic.
All this to say, (even with a sample size of one) we need to work more closely with families to help them create that supportive home environment. We may need the equivalent of community outreach workers who make home visits and are well-regarded as informal leaders to help these families understand the basics-no TV during the school week, a quiet well-lit place for study, and 30-40 minutes of reading six times/wk above and beyond school work.
My middle class friends (primary mom's-dad's are usually only good to play the bad cop) with college degrees know what it takes to help a child succeed in school.
A trained cadre of women from the neighborhood whose kids are doing ok in school understands what other famlies face & can relate to them. Pairing something of that nature with properly staffed (not a ratio of 20 kids to one tutor) afterschool tutoring and enrichment programs, might be helpful. We cannot expect parents to do what is necessary if they do not know what steps need to be taken.
Another voice is needed to bring the discussion around to interventions that focus on preparing parents to help their children learn and work in partnership with teachers. Dancing around the parent issue has led us to the point where we are today.
I can't speak on Texas schools, but your ideas of school reform that speak to vouchers and school choice which really equal the privitzation of public education. That money goes to private companies who have an interest in making money, not helping your child succeed. Also these schools can deny entry to the neediest of students, while public schools cannot. There are good charters out there, but there are far more people who want to take the public money and make a big profit.
Teachers are not against accountability, they are against being held accountable for things they do not control. Students scores on standardized tests are measures of the students abilities (or temperment, how they feel, whether they care on the day of the test), not the teachers. In places where teacher accountability is seen as beneficial and something that improves their abilities it is more welcomed. Peer observations in addition to principal and other district leadership are welcomed by many teachers, because those teachers want to be better. I am always frustrated by my observations because they offer me little in the way of actual advice for doing a better job.
dj kumquat wrote: "after all, long before there was a public school system, people were being educated."
Before there was a public school system, only the rich got educated. Do you really want to limit education to people who can afford private school or home schooling? Private school costs $20-$40k per kid per year. Home schooling costs the income of the schooling parent.
Teaching is not an ART...
Teaching is a Craft and it needs a Core Curriculum. This means something very important. It means that Teachers CANNOT (yet) be held accountable for student progress. In all likelihood teachers have been denied access to the Best Tools; Teacher Education is all hit and miss. Courses with identical titles can vary very significantly from school to school, and professor to professor. This is a recipe for near chaos, ironically there is no "crisis" in education, and we should not act as if one exists if for no other reason than because CRISIS conjures panic, a search for culprits and competing disruptive reformers with vested interests in everything but education. However, it is past time to take some measured evolutionary steps whose benefits could be globally far reaching.
Continued at: http://www.bestmethodsofinstruction.com/
Here are some examples of How Best to Teach… anthony-manzo.blogspot.com/
1. Guided Reading Procedure* for accurate reading comprehension, Recall and Study Reading
2. Enabling Questions (EQ) For Active Student-Centered Inquiry & Verbal Learning* [http://anthony-manzo.blogspot.com/2010/06/enabling-questions-for-self-d…]
3. The Informal Reading-Thinking Inventory (IR-TI)
The Informal Reading-Thinking Inventory:
Assessment Formats for Discovering Typical & Otherwise Unrecognized Reading & Writing Needs – and Strengths [http://anthony-manzo.blogspot.com/2010/06/informal-reading-thinking-inv…]
4. Help with Reading Mathematics: Content Area Literacy Teaching [http://anthony-manzo.blogspot.com/2010/06/help-with-reading-mathematics…]
5. Dyslexia as Specific Psychological Disorder -Conversion Reaction Syndrome [http://anthony-manzo.blogspot.com/2010/06/dyslexia-as-specific-psycholo…]
6. iREAP: Improving Reading, Writing, Thinking and Aesthetics in the Wired Classroom [http://anthony-manzo.blogspot.com/2010/06/ireap-improving-reading-writi…]
7. A Fun High Frequency, Whole Word Flash Card Practice Routine - "Say it like a Barbie!"* [http://anthony-manzo.blogspot.com/2010/06/fun-high-frequency-whole-word…]