On Focus Groups, Elections, and Predictions

Focus groups, a method in which small groups of subjects are questioned by researchers, are widely used in politics, marketing, and other areas. In education policy, focus groups, particularly those comprised of teachers or administrators, are often used to design or shape policy. And, of course, during national election cycles, they are particularly widespread, and there are even television networks that broadcast focus groups as a way to gauge the public’s reaction to debates or other events.

There are good reasons for using focus groups. Analyzing surveys can provide information regarding declaratory behaviors and issues’ rankings at a given point in time, and correlations between these declarations and certain demographic and social variables of interest. Focus groups, on the other hand, can help map out the issues important to voters (which can inform survey question design), as well investigate what reactions certain presentations (verbal or symbolic) evoke (which can, for example, help frame messages in political or informational campaigns).

Both polling/surveys and focus groups provide insights that the other method alone could not. Neither of them, however, can answer questions about why certain patterns occur or how likely they are to occur in the future. That said, having heard some of the commentary about focus groups, and particularly having seen them being broadcast live and discussed on cable news stations, I feel strongly compelled to comment, as I do whenever data are used improperly or methodologies are misinterpreted.

The problem, put simply, is that focus groups are frequently misused (as are survey results, for that matter). The observations are interpreted erroneously; for example, as an indication of how many voters in the general population would switch their opinion after being presented with a particular argument, or as evidence that this argument was the reason for (cause of) the opinion change. Given how often focus groups are used and interpreted, it may be helpful to discuss these issues briefly.

Focus groups, in qualitative methodology, are used for very specific purposes. Regardless of how the participants in such a group or groups are chosen, the goal of convening focus groups is to get insight into how the participants react to certain stimuli in the presence of and interacting with other group members. That is, they are useful in gathering information about people’s views on a given issue in a collective context.

What this implies is that views and feelings of focus groups’ participants cannot be interpreted outside of the context in which they were observed. That is, they are solicited as much by moderators as by other participants’ opinions, and the group dynamic is as important as (if not more important than) individuals’ responses. This group aspect is what differentiates focus groups from in-depth interviews, also a common qualitative approach. The latter are conducted in order to gain insight into an individual’s subjective interpretation of events, whereas the former is to observe subjects’ reactions while interacting with others.

Focus groups are informative if we are interested, for example, in what vocabulary, figures of speech, or cultural reference points members of the group use to connect/explain themselves to others, while in-depth interviews are useful in understanding how they explain their opinions and actions to themselves.

For the reasons highlighted above, focus groups cannot provide information regarding individual people per se. The data should be analyzed in the context of the group – participants’ reactions may be different in a different group, even one consisting of seemingly similar persons. (And, of course, reactions could also be different in a group not broadcast on national TV.)

Moreover, focus groups are more useful for illustrating differences than consensus in individuals’ opinions. People interacting with each other have a tendency to converge toward a common opinion, and to suppress dissenting views. Consensus, therefore, may be artifactual. This is especially important in light of the fact that focus group members are usually selected into a given group based on their similarities rather than differences.

The reactions solicited during a focus group interview also cannot provide information regarding shifts in subjects’ opinions, feelings or beliefs. You cannot say that how a subject interpreted a given stimulus is why the subject formed (or changed) his or her view. The narratives offered by participants can only help to reconstruct the process of “at the moment” interpretation of events/stimuli presented, as focus groups provide a one time observation. For example, when a participant says that watching a presidential debate made him or her angry (or happy, or hesitant, or whatever) because of a certain word or expression a given candidate used, you cannot say that the word or expression itself caused the anger. There is a range of alternative reasons why the person could react with a given emotion. For instance, that person could have been angry with the candidate in general, and the word or phrase might simply have triggered that feeling. You can, however, probe for meanings that subjects associate with a given term or expression, which may become useful when thinking about, say, possible communication strategies.

Another, perhaps more fundamental reason why causal inferences based on what participants say in focus groups are not warranted is the following: People often do not know why they do what they do, or why they think what they think. That is not meant to be condescending. It is an established scientific fact. Human thought processes are unbelievably complicated. Plenty of research into decision making (here, here, and here, for example) demonstrates that what people identify as the reasons for their actions often are not consistent with the range of possible objective reasons. Moreover, it is unreasonable to expect individuals to be aware of how their interpretations are constrained by the opportunities they were given (here, here, and here, for example).

Put simply, when people react to stimuli, such as a political advertisement or a debate, they can be asked how they justify that reaction. The purpose of focus groups is to explore those justifications (words, cultural references, symbolism, etc.) expressed in the presence of and while interacting with other people, not to identify why they actually feel the way that they do.

As mentioned above, focus groups are a form of qualitative data, and no qualitative data can ever be used for making generalizations about a given population, regardless of the focus group’s composition. (Though, in the case of cable news broadcasts of focus groups, viewers aren’t even told anything about how the groups are selected or who the members of the group are.)

There are many ways to select qualitative samples, but none of these qualitative sampling methods gives any basis for generalizations to populations from which these groups were chosen. In other words, qualitative generalizations can extend to qualitative ideas, such as when you want to build a map of connotations brought up by a given term, but that doesn’t mean those connotations are shared by the population in general. It follows, therefore, that focus groups do not provide any basis for claiming that a given stimulus would have the same effect for persons different from those in the focus group.

To be sure, there is a very strong temptation to draw strong causal conclusions from focus groups. When a person tells you, for example, why they support a given candidate, it is natural to take them at their word. But our reactions to people and events are highly complex – even we are not fully aware of all that goes into them. Focus groups can be useful in showing us how people explain their reactions to others, but they cannot show us why people think what they think, nor can they help to predict how others will think or react in the future. It would be wonderful if all those who present and discuss focus groups would learn the difference.