The Impact Of The Principal In The Classroom

Direct observation is way of gathering data by watching behavior or events as they occur; for example, a teacher teaching a lesson. This methodology is important to teacher induction and professional development, as well as teacher evaluation. Yet, direct observation has a major shortcoming: it is a rather obtrusive data gathering technique. In other words, we know the observer can influence the situation and the behavior of those being observed. We also know people do not behave the same way when they know they are being watched. In psychology, these forms of reactivity are known as the Hawthorne effect, and the observer- or experimenter- expectancy effect (also here).

Social scientists and medical researchers are well aware of these issues and the fact that research findings don’t mean a whole lot when the researcher and/or the study participants know the purpose of the research and/or are aware that they are being observed or tested. To circumvent these obstacles, techniques like “mild deception” and “covert observation” are frequently used in social science research.

For example, experimenters often take advantage of “cover stories” which give subjects a sensible rationale for the research while preventing them from knowing (or guessing) the true goals of the study, which would threaten the experiment’s internal validity – see here. Also, researchers use double-blind designs, which, in the medical field, mean that neither the research participant nor the researcher know when the treatment or the placebo are being administered.

But even if one has no background in research methods or psychology, it is not hard to imagine that a group of students will behave differently when the principal is sitting in the back of the classroom. At a minimum, the novelty will be distracting. Also, the principal’s authority can be intimidating to some students, making them more timid and possibly less likely to participate. This means we could easily end up in a situation in which the principal’s efforts to measure the extent to which the teacher can elicit students’ engagement, are undermined by his/her very presence in the classroom.

The same basic point applies to teachers. What they do while being observed, especially when criteria are explicit, does not always represent what they usually do in the classroom. This would be the equivalent of expecting research participants to behave candidly after telling them about purpose of the study.

Frankly, given my training and the fact that all these are well known effects and biases, it still surprises me to hear that in-person observations are being used to make high-stakes decisions about teachers. Although all performance evaluation is necessarily biased, direct observation might be among the worst forms.

Drawing conclusions about a teacher’s effectiveness based on a series of classroom observations seems to me like approving a new drug based on clinical trials that are not double-blind. How can we be sure that the new drug really works when we are comparing reactions of groups who knew if they were given the drug or the placebo?  Poorly-designed  research is not very informative. At a minimum, important decisions should not be made based on it.

Of course, the real question here is whether there’s an alternative. In other words, what can be done to conduct classroom observations such that the data gathered have increased validity and can be used to improve instruction?

I think cameras in the classroom are a viable option, since, while their presence would be known to all, actual recording and/or reviewing could occur without the knowledge of teachers and students. This could prove useful in at least two ways. First, using cameras would be far less obtrusive and could avoid the above mentioned effects. Second, recordings could be watched and reproduced multiple times and viewed and shared among evaluators and teachers for self-reflection and training purposes. For evaluation purposes, teachers could be allowed to choose from among various recordings so that they could pick the one that they believe best represents their typical lesson/day.

The Measures of Effective Teaching (MET) project is in fact testing a method of video capture which, if done correctly, has potential to improve teacher evaluation and professional development.

Technology alone is not a quick fix, and it’s fraught with complications. For one thing, installing cameras in classrooms would require up-front investment. More importantly, there are ethical considerations involved, which would have to be addressed prior, during and after implementation. Finally, it would be essential to understand how/why video capture could be used to improve the limitations I described here – i.e., what happens when we know we are being observed and/or why – as well as our faulty judgments of others, which I wrote about elsewhere.

- Esther Quintero