Category Archives: methods

Detecting Social Media Bullshit: A Sociologist’s View

Social media “gurus” abound these days. Which ones are worth listening to and which ones are bullshitters?

Philosopher Harry Frankfurt exposed bullshitters in his famous essay “On Bullshit.” The liar knows what the truth is and cares very much about concealing it. The bullshitter, on the other hand, doesn’t care what the truth is and has no compunction in stretching it.

The same goes for social media “gurus.” Those that care what about rigourous examination of the social may be wrong, but at least they take great pains to analyze the phenomenon. Those that don’t care about systematic, theoretically informed social inquiry are interested only in stretching or shaping their own agendas.

How can you tell the difference?

Here are a few signs you’re dealing with a social media bullshitter.

  1. They skate over the tension between structure and agency: The tension between structure and agency is an age-old sociological debate. Social media bullshitters somehow miss this very important point. They often argue that implementing social media or social business design will somehow evaporate decades or even centuries of organizational structures. If your social media guru tells you that adding social media and stirring will create equality, harmony, and profits, begin to question them. If, on the other hand, they tell you that your organization does not live in a vacuum, and that your social media will be integrated in people’s existing lives with their existing economic, technological, and ethnically grounded experience, then they may be onto something.
  2. They use the same social research methods every time: A classically trained sociologist is trained in both qualitative and quantitative methods. They are designers in the sense that they have expertise, which they draw upon selectively, according to the research question. Social media bullshitters, on the other hand, likely have a common stock of tools that they use repeatedly, regardless of the nuance of the research question. If their answer is always, “do a focus group,” or always, “do a survey,” then question them.
  3. They see no paradoxes. Ever: Sociologists are constantly grappling with paradoxes. Weber’s famous paradoxical finding was that bureaucracies are both efficient and inefficient. They work wonders building and managing railroads, for example, but they result in horrible catastrophes like the Challenger disaster. Weber explained this paradox by arguing that rationality, or the rule of rules, is an “iron cage,” that keeps us safe but enslaved. If your social media guru claims there will be no paradox, nuance, or ambiguity, question them.
  4. They don’t know what social capital really is: Social capital is not something one can measure in terms of bank balances. It was the creation of French sociologist Pierre Bourdieu (come to think of it, the bullshitters wouldn’t know that either). Social capital is something one develops by being in a particular social location. I may go to an exclusive boarding school. My social capital is my network of well-off friends. Social capital is a particularly important concept when thinking about social media. Bourdieu noted that those in lower economic classes explicitly reject items they consider “above their station.” This means that luxury or “top of the line” is not always your best approach.

The bottom line is this: social media bullshitters have no knowledge of social theory or methodology. Trust a person who provides no easy answer, who carefully selects their research method, and who understands complex concepts.

Do you have more signs of being a social media bullshitter? Please share them here!

Customers more satisfied when served by white males

In an interesting study, researchers at UBC have found that customers express higher satisfaction when they’re served by white men than by women or people of colour — even when their behaviour is exactly the same. Marketing professor Karl Aquino expressed surprise at the findings, as he told The Globe and Mail

“We had thought there would be some bias going on in the sense of people who were males or whites would be rated more positively,” Mr. Aquino said

“But we didn’t anticipate that for performing the same behaviours, the women and minorities would actually be rated lower,” he said of the study to be published in the Academy of Management Journal.

This study should not be surprising at all.

What this study demonstrates is what Raymond Breton calls the “symbolic order”; we unconsciously place white men at the top of our social hierarchy. We do this in multiple ways, including placing art, culture and ideas at the top of an invisible ladder. Public Enemy sums it up nicely in “Fight the Power”:

None of my heroes don’t appear on no stamp

We know that people have largely unconscious reactions of sexism and racism, oftentimes without even realizing it. It is likely that these unconscious ideas bleed into marketing research easily, especially when such studies are quantitative in nature, and therefore lack the thick description or deep probing offered by qualitative approaches.

This finding has wide-reaching implications. First, when companies use customer satisfaction surveys, they must be aware of the inherent inaccuracy of these surveys. You may believe you’re accurately measuring actual satisfaction, but this study shows that frequently, we don’t measure any such thing. Secondly, such surveys are often used to award bonuses or even job security. As we know in academia, student evaluations are frequently what stands between a scholar and a full-time position. If we know that customer satisfaction is driven by factors other than actual performance, then we are likely to be unwittingly simply rewarding membership in a dominant group.

Read the entire story on The Globe. It’s worth a think.

Improving participation rates: research recruitment best practices

Those of you out there who’ve tried it know: recruiting research participants is HARD. Here are a few insights from the research to help you with better recuitment.

  1. Personalized contact with respondents, followed by pre-contact and aggressive follow-up phone calls *: Don’t count on a form letter, email or random tweet to do the job. Capitalize on your personal relationship with that person. If you don’t have a personal relationship, ensure that you use the person’s name, and for God’s sake, spell it correctly!

    Once you’ve made initial contact, you are not done. Not by a long shot. Make sure you speak to the person (you can do this through IM or email if you’d like) to give them more information. They’re now interested. Don’t stop! One more step!

    Follow up 1 week after initial contact. Assuage any fears they may have. Answer any questions honestly. And above all, be available for more information.

  2. External researchers with social capital are best**: University-based researchers have been shown to have the best participation rates, but you don’t have to be a professor.  Researcher Sister Marie Augusta Neal of Emmanuel College achieved a near perfect response rate because of her close ties to the respondents and their communities. The lesson here is, if you hire a consultant, make sure they’re trusted. Even better if they personally know the people to be recruited.
  3. Monetary incentives have no effect, unless money is offered “no strings attached”***: Little known fact: the best way to use a monetary incentive is to offer it, up front, with absolutely no strings attached. The “free” money makes people feel more indebted socially. Evidence of this effect can be found in the book Freakonomics. Researchers found that daycare centres that levied late penalties on tardy parents actually had more of a late-pickup problem than those that levied no fine. Why? Because the parents reduced their relationship to the daycare as a mere transaction. Use the “gift economy” approach and ensure a feeling of indebtedness. My personal favourite is a coupon for a single iTunes song at $.99. It is cheap but appears to have great value. Offer it, up front, and then ask for participation

*  Cook, C., F. Heath, and R. Thompson. 2000. “A Meta-analysis of Response Rates in Web or Internet-based Surveys.” Educational and Psychological Measurement 60:821-836.

** Rogelberg, S., A. Luong, M. Sederburg, and D. Cristol. 2000. “Employee Attitude Surveys: Examining the Attitudes of Noncompliant Employees.” Journal of Applied Psychology 85:284-293.

***Hager, M., S. Wilson, T. Pollak, and P. Rooney. 2003. “Response Rates for Mail Surveys of Nonprofit Organizations: A Review and Empirical Test.” Nonprofit and Voluntary Sector Quarterly 32:252-267. Singer, E. (2006) Introduction: Nonresponse Bias in Household Surveys. Public Opinion Quarterly, 70, 637-645

What’s wrong with ethnography?

Ethnography is bandied about frequently in business and design circles these days. And sadly, like many buzz words, its true meaning has been lost in its popularity. Let me start by saying ethnography is hot today because it provides you insight you can’t get from being far away from your target users.

Ethnographic research evolved out of cultural anthropology. Some of you may remember Margaret Mead’s famous ethnography of the Samoans (some of you may also remember the Samoans’ famous joke on her, but more on that later). Mead lived with the Samoans to decipher how their culture affected the sexual maturation of girls. She wrote copious notes on her experiences, and later, when studying elsewhere in the South Pacific, took over 25,000 photographs.

Ethnographic research is first and foremost about observation. Ethnographers are not experimenters. They do not engineer or contrive situations to elicit reactions. They observe “natural” settings, that is, where people are going about their lives. Contrary to popular belief, ethnographers also do count things — quantitative data can serve to summarize a large number of observations (e.g., how many people on the subway are carrying a briefcase?).

Ethnography is NOT simply “in-person interviewing.” Now there is such a thing as “ethnographic interviews,” which melds ethnographic observation of natural settings with in-depth interviewing techniques. I myself have used ethnographic interviews on my dissertation and gleaned great insight.

But true ethnography means months of observation and in-depth analysis of all the “symbols” that your target users use. That means paying attention to their clothes, their manner of speech, their “argot” or local shared dialect, and even the accepted practices around social events like meals, meetings, and saying goodbye.

Ethnography does have a very clear limitation, which becomes clear when you learn about the Samoan joke on Margaret Mead. It was quite common to joke about sex in Samoan culture, so when Mead asked these young girls what they did at night, they jokingly told her they spent the night with boys. Mead later reported that Samoans matured sexually much more quickly than North Americans and had little of the same repressed sexuality. But this was not at all true and Mead had been duped.

Mead’s assumptions that Samoans were sexually more liberated than North Americans affected her research. Ethnographers who do not understand issues of gender and power are condemned to repeat these mistakes. An ethnographer interviewing workers must understand that when they tell her they “like having a laptop,” they have a need to portray themselves as “team players.”

An ethnographer must understand that when he interviews women in their kitchens, they are demonstrating their “proper” roles as women and may have a vested interest in portraying themselves as more “homey” than they really are.

Ethnographers are not objective. They are part of this thing we call society. As such, they have biases, just like everyone else. Good ethnographers understand that designing new laptops or new kitchens is about understanding the target user’s place in society as well.

Further reading: