Tag Archives: qualitative research

The essence of qualitative research: “verstehen”

“But how many people did you talk to?” If you’ve ever done qualitative research, you’ve heard that question at least once. And the first time? You were flummoxed. In 3 short minutes, you can be assured that will never happen again.

Folks, qualitative research does not worry about numbers of people; it worries about deep understanding. Weber called this “verstehen.” (Come to think of it, most German people call it that too. Coincidence?). Geertz called it “thick description.” It’s about knowing — really knowing — the phenomenon you’re researching. You’ve lived, breathed, and slept this thing, this social occurrence, this…this…part of everyday life. You know it inside and out.

Courtesy of daniel_blue on Flickr

Courtesy of daniel_blue on Flickr

You know when it’s typical, when it’s unusual, what kinds of people  do this thing, and how. You know why someone would never do this thing, and when they would but just lie about it. In short, you’ve transcended merely noticing this phenomenon. Now, you’re ready to give a 1-hour lecture on it, complete with illustrative examples.

Now if that thing is, say, kitchen use, then stand back! You’re not an Iron Chef, you are a Platinum Chef! You have spent hours inside kitchens of all shapes and sizes. You know how people love them, how they hate them, when they’re ashamed of them and when (very rarely) they destroy them. You can tell casual observers it is “simplistic” to think of how many people have gas stoves. No, you tell them, it’s not about how many people, it’s about WHY they have gas stoves! It’s about what happens when you finally buy a gas stove! It’s about….so much more than how many.

Welcome to the world of verstehen. When you have verstehen, you can perhaps count how many people have gas stoves. Sure, you could determine that more men than women have them. Maybe you could find out that more of them were built between 1970 and 80 than 1990 and 2000. But what good is that number? What does it even mean?

When you’re designing, you must know what the gas stove means. You must know what it means to transform your kitchen into one that can and should host a gas stove. You must know why a person would be “ashamed” to have a gas stove (are they ashamed of their new wealth? do they come from a long line of safety-conscious firefighters?). You must know more than “how many.”

So the next time someone asks you, “how many people did you talk to?”, you can answer them with an hour-long treatise about why that doesn’t matter. You can tell them you are going to blow them away with the thick description of what this thing means to people. You are going to tell them you know more about this thing than anyone who ever lived, and then, dammit, you’re gonna design something so fantastic, so amazing that they too will be screaming in German. You have verstehen!

See my discussion about sampling methods in qual and quant research for more insight into the reasons why “how many” is irrelevant in qualitative research.

Designers are from Venus, Six Sigmas are from Mars

DT has a great post over at Design Sojourn that discusses Six Sigma methodology and how it relates to design. He cites Tim Brown at IDEO who argues that Six Sigma is essentially Newtonian, while design thinking is quantum. In his own design work, DT expressed doubts about using Six Sigma:

After studying the Six Sigma process, I point blank said: “There was no way any of my designers are going to be judged on the quality and success of a design based on how many sketches or iterations we did before we deliver it.”

Both Brown and DT cite Sara Beckman, who recently discussed the topic in the New York Times. Beckman reviews how Six Sigma focuses on incremental improvements, while design and design thinking focuses on big changes. For those of you who aren’t familiar with Six Sigma, it’s a method pioneered by Motorola, which aims to reduce the number of errors to 3 in one million. The “six sigma” refers to six standard deviations. The number of errors should be at the extreme end of the normal curve, or between + or – 3 standard deviations, represented by the Greek symbol sigma.

I argue that design is more complementary to the “interpretivist” paradigm of qualitative research while Six Sigma is positivist. Interpretivists don’t believe the world is a static place. They see reality as being continuously created by you, me and other social actors. There is no such thing as “The Truth” in interpretivist approaches, just different versions of the truth. Typical methods of interpretivists are ethnography, in-depth interviewing and discourse analysis. Positivist research, on the other hand, assumes that reality is static. Positivists believe that “The Truth,” is out there to be discovered. Typical methods would include quantitative surveys.

Designers should focus on interpretivist methods, therefore. They should uncover different versions of the truth using observation and interviewing, as well as deep reflection on symbols and their meanings. Surveys and other quantitative methods are more Six Sigma in that they can measure improvement over time. Designers ought to consider measuring improvement, but starting with qualitative approaches is best.

The evolution of qualitative sociology

The blog Economic Sociology has a great post on the “evolution” of qualitative sociology. They note, quite rightly, that the notion of “evolution” is implicit in much of social science, even if it has no bearing on the subject matter at hand. Many sociologists place quantitative research “on top” of the research “evolutionary ladder,” even when there is no such thing as a ladder when it comes to good research design. Interestingly, the fathers of sociology themselves would be on the “lower rung” of that methodological ladder:

The works of Marx and Weber, like virtually all the classic literature in the field, were based on qualitative, historical methodology (Durkheim’s quantitative study Suicide being a notable exception).

This post just reinforces to me why the design process is so important for social scientists. One must design a research project to solve contextual problems, just as one designs, say, a chair. You cannot “solve” questions of why or how by using quantitative methods. It is simply impossible.

Are why and how somehow “less than” questions than “how many” or “how fast”? I don’t think so. Indeed Economic Sociology points out that even Darwin was not concerned with “how many” but more with “how,” and few accuse him of being “unscientific.”

Design thinking’s big problem

So-called “design thinking” is the new It-Girl of management theory. It purports to provide new ways for managers and companies to provide innovative, creative solutions to old problems. But design thinking alone will not solve these problems because a lack of creativity was never the issue.

The real issue is one of power.

Design is attractive to management because it is a de-politicized version of the well known socio-cultural critique of managerial practices. Design thinking is so popular because it raises only questions of “creativity” or “innovation” without ever questioning the legitimacy of managerial practice. Instead, design thinking aspires only to “better” management technique by investigating “contextual problems” or the truly innocuous “pain points.”

The inconvenient truth is that the science of management fails because it treats people as either mere inputs into the production process or as faceless “consumers” who have no real stake in outcomes. Design thinking allows for these truths to remain unaddressed, thereby avoiding any discussion of power itself. Workers are cast as something to be organized or “incented.” Consumers are to have their “needs met.” And neither group is granted a meaningful stake in the creative process.

Within this frame, design techniques attempt to solve managers’ typically tone-deaf executions of creativity without ever naming the root cause of workers’ and consumers’ dissatisfaction, which is their lack of meaningful participation in the design process. Managers’ ability to control both the organization of work and the availability of consumer goods is the true problem, not an inability to think “creatively.”

Managers have control over the working conditions under which creativity is supposed to happen, as well as the the distribution of the fruits of such labour. One significant reason workers’ creativity does not flow easily from studio or factory to consumers is because of management’s need to control costs and secure profits. Were it not for the profit motive, workers would be free to radically innovate continually and consumers would have unrestricted access to such new and innovative goods. But because profit stands as the pre-eminent benchmark of business success, both workers and consumers are thwarted in their pursuit of supplying and demanding innovative goods.

In other words, there is no shortgage of creative solutions to “unmet needs,” only a shortage of profitable ways to provide them.

Hence the inevitable ineffectiveness of design thinking, if applied in isolation to the problem of creativity. Designers must consider what role power plays in an organization’s inability to create innovative products. But more importantly, designers must be prepared to identify and name power and its sources (e.g., the pursuit of profit at the expense of innovation).They must not simply use ethnographic techniques to uncover “unmet needs”.

This is perhaps where designers will feel most out of their depth. It is a long leap from solving contextual problems to providing an analysis of inequality. All the more reason then, for designers to study the socio-cultural theory that underlies ethnography and other qualitative research methods.

In particular, designers should study feminist writers such as Canadian sociologist Dorothy Smith. Smith founded the method she calls “institutional ethnography,” which takes the standpoint of its participants and not that of the organization. This method frequently yields lived experiences that differ from the “official record” because it assumes that users of a technology, a product or a social policy lack meaningful access to those who record such records.

Ethnographic approaches are a good starting point for designers to cultivate empathy and hone observational skills. But it is in issues of power that rememdies to innovation bottlenecks will be broken.

Mind the gap: qualitative insights and strategy

It’s very common to turn to numbers first when strategizing about new products, policies, or social movements. But nuanced, sideways or “integrative” thinking often requires more than just numbers. This is where qualitative research can help you.

Most people are trained to think of “research” as numbers and “hard facts.” That approach will lead to very specific, numerical questions when crafting new strategies. What are the most popular products consumers want? What are the top five frustrations with our current policy? What are the top Web sites that progressive people visit?

But imagine there was no such thing as numerical “evidence.” Imagine instead that you were trying to figure out how to innovate without the benefit of any kind of counting. What kinds of things would you consider to be insight?

Why do consumers get frustrated with their telecommunications service providers? How and in what ways do citizens react to our policy on childcare? What kinds of digital tools do progressive people use in everyday life?

These second sets of questions are far more likely to yield what qualitative researchers call “thick description.” Thick description fills the gaps between numbers. If I told you Superbowl 36 ended with a score of 20-17, you’d miss all the detail and the drama of the late-in-the-game push by the Rams, and the final Patriot field goal that ultimately won the game. Thick description tells you the entire story, not just the numerical summary.

If policymakers know that 49% of parents are frustrated with no childcare policy, that doesn’t begin to explain a day in the life of a working parent. Spend a day with a working parent and a sick child, and you will begin to understand all the detail and the drama of childcare.

If you spend time with person who is interested in progressive causes, you may learn that they spend more time using their mobile phone than their computer. Or perhaps you learn that for them, computers = work. That may lead you to think that mobile campaigns are better than Web-based campaigns.

Qualitative research intended to fill the gaps that numerical data inherently possess. If you rely too heavily on numerical data, you miss a great deal of nuance that could ultimately result in true innovation.

Personas are “empathy tools,” not stereotypes

Bookmark and Share

We all use personas in everyday social situations. But, like in many design projects, we use to them to typecast instead of to evoke empathy. Personas, like stereotypes, often result in discriminatory behavior. When used in design, personas can create poor design that disempowers and alienate users.

We all like to know how to treat people appropriately. We tend to use what social theorists Berger and Luckman call “typifications” when interacting socially. When we go to the store, to a meeting, to a party — we need to know how to act with people. We genuinely want to make people feel comfortable and we want to feel comfortable ourselves.

But to use a typification often has the unintended consequence of being condescending. Elderly people are spoken to in loud, exagerrated tones. Women are assumed to be physically fragile. Men are considered to be sexually aggressive. These typifications are stereotypes that affect how we, in turn, react. Elderly people may react angrily, for example, at the implied loss of their faculties.

Designers often make the same mistake when making personas. Personas are tools to evoke empathy. But poorly created personas will simply regurgitate stereotypes instead of actually answering real needs. When a site is designed “for women,” it should allow women (and all of its users) to define their experience, according to their needs. Women may have more need to juggle schedules, for example, so interactive experiences should allow them to adopt such features.

An interactive experience should not, however, force me to be treated as a “mom on the go” simply because I’m a woman. And honestly, if there’s one persona phrase that makes me want to vomit/go on a murderous rampage/re-design the design process, it’s the dreaded “mom on the go.” Show me a mom NOT on the go, and I’ll show you a mom who forgets she has children.

Worse, don’t treat me, a childless woman of 38, as a “mom on the go,” simply because YOUR data tell you I should have children. Instead, empathize with me. Allow me to satisfy unmet needs, should I so choose. DO NOT force me to adopt features and functionality that are appropriate for what you think I OUGHT to need.

As a woman, I am frequently “treated” to “gentle” behavior. People will open doors for me, or perhaps allow me to pass first out of a crowded elevator. This is not because I require it, nor because I expect it, but because it is believed that women still are the “gentler sex.”

Defeating the problem of personas as stereotypes is to put yourself in the user’s shoes. In other words, don’t forget that personas are empathy tools. Allow her to choose her experience. Provide her the features and functionality that she MIGHT like, based on your qualitative research. But under no circumstances force her to adopt features or functionality that reproduce what someone “ought” to be.

Forcing people to adopt behaviors is as far from empathy as one can get. Interactive experiences that foist “mom on the go” fantasies onto real people risk alienating their users at best; at worst they perpetuate sexist stereotypes.

Sampling methods in qualitative and quantitative research

Why does sample size not matter in qualitative research? Because of the assumptions that qualitative researchers make, namely, that the social world is not predictable. Qualitative researchers believe that people are not like molecules or other objects; people’s actions are not predictable.

But quantitative researchers DO believe that social activity IS predictable. So when they compare their observations of social activity to what would happen in purely random results, the difference says something. Let’s say you were to research people’s preferences for a particular interactive feature. Say you’re wondering if young people will like a radio button more than older people. First, you model what results you’d expect if you’d just flipped a coin. Then you use a probability (random) sample, and compare those results to purely random results. Is there a difference?

If there is a difference between them, you can infer that indeed, something other than chance (in this case, age) affect people’s preferences.

Qualitative researchers don’t agree that such things can be reliably predicted. That’s why they don’t bother with expensive and involved random sampling. See all these details below from my research design course.