Improving participation rates: research recruitment best practices

Those of you out there who’ve tried it know: recruiting research participants is HARD. Here are a few insights from the research to help you with better recuitment.

1. Personalized contact with respondents, followed by pre-contact and aggressive follow-up phone calls *: Don’t count on a form letter, email or random tweet to do the job. Capitalize on your personal relationship with that person. If you don’t have a personal relationship, ensure that you use the person’s name, and for God’s sake, spell it correctly!

   Once you’ve made initial contact, you are not done. Not by a long shot. Make sure you speak to the person (you can do this through IM or email if you’d like) to give them more information. They’re now interested. Don’t stop! One more step!

   Follow up 1 week after initial contact. Assuage any fears they may have. Answer any questions honestly. And above all, be available for more information.

2. External researchers with social capital are best**: University-based researchers have been shown to have the best participation rates, but you don’t have to be a professor.  Researcher Sister Marie Augusta Neal of Emmanuel College achieved a near perfect response rate because of her close ties to the respondents and their communities. The lesson here is, if you hire a consultant, make sure they’re trusted. Even better if they personally know the people to be recruited.
3. Monetary incentives have no effect, unless money is offered “no strings attached”***: Little known fact: the best way to use a monetary incentive is to offer it, up front, with absolutely no strings attached. The “free” money makes people feel more indebted socially. Evidence of this effect can be found in the book Freakonomics. Researchers found that daycare centres that levied late penalties on tardy parents actually had more of a late-pickup problem than those that levied no fine. Why? Because the parents reduced their relationship to the daycare as a mere transaction. Use the “gift economy” approach and ensure a feeling of indebtedness. My personal favourite is a coupon for a single iTunes song at $.99. It is cheap but appears to have great value. Offer it, up front, and then ask for participation

*  Cook, C., F. Heath, and R. Thompson. 2000. “A Meta-analysis of Response Rates in Web or Internet-based Surveys.” Educational and Psychological Measurement 60:821-836.

** Rogelberg, S., A. Luong, M. Sederburg, and D. Cristol. 2000. “Employee Attitude Surveys: Examining the Attitudes of Noncompliant Employees.” Journal of Applied Psychology 85:284-293.

***Hager, M., S. Wilson, T. Pollak, and P. Rooney. 2003. “Response Rates for Mail Surveys of Nonprofit Organizations: A Review and Empirical Test.” Nonprofit and Voluntary Sector Quarterly 32:252-267. Singer, E. (2006) Introduction: Nonresponse Bias in Household Surveys. Public Opinion Quarterly, 70, 637-645

What’s wrong with ethnography?

Ethnography is bandied about frequently in business and design circles these days. And sadly, like many buzz words, its true meaning has been lost in its popularity. Let me start by saying ethnography is hot today because it provides you insight you can’t get from being far away from your target users.

Ethnographic research evolved out of cultural anthropology. Some of you may remember Margaret Mead’s famous ethnography of the Samoans (some of you may also remember the Samoans’ famous joke on her, but more on that later). Mead lived with the Samoans to decipher how their culture affected the sexual maturation of girls. She wrote copious notes on her experiences, and later, when studying elsewhere in the South Pacific, took over 25,000 photographs.

Ethnographic research is first and foremost about observation. Ethnographers are not experimenters. They do not engineer or contrive situations to elicit reactions. They observe “natural” settings, that is, where people are going about their lives. Contrary to popular belief, ethnographers also do count things — quantitative data can serve to summarize a large number of observations (e.g., how many people on the subway are carrying a briefcase?).

Ethnography is NOT simply “in-person interviewing.” Now there is such a thing as “ethnographic interviews,” which melds ethnographic observation of natural settings with in-depth interviewing techniques. I myself have used ethnographic interviews on my dissertation and gleaned great insight.

But true ethnography means months of observation and in-depth analysis of all the “symbols” that your target users use. That means paying attention to their clothes, their manner of speech, their “argot” or local shared dialect, and even the accepted practices around social events like meals, meetings, and saying goodbye.

Ethnography does have a very clear limitation, which becomes clear when you learn about the Samoan joke on Margaret Mead. It was quite common to joke about sex in Samoan culture, so when Mead asked these young girls what they did at night, they jokingly told her they spent the night with boys. Mead later reported that Samoans matured sexually much more quickly than North Americans and had little of the same repressed sexuality. But this was not at all true and Mead had been duped.

Mead’s assumptions that Samoans were sexually more liberated than North Americans affected her research. Ethnographers who do not understand issues of gender and power are condemned to repeat these mistakes. An ethnographer interviewing workers must understand that when they tell her they “like having a laptop,” they have a need to portray themselves as “team players.”

An ethnographer must understand that when he interviews women in their kitchens, they are demonstrating their “proper” roles as women and may have a vested interest in portraying themselves as more “homey” than they really are.

Ethnographers are not objective. They are part of this thing we call society. As such, they have biases, just like everyone else. Good ethnographers understand that designing new laptops or new kitchens is about understanding the target user’s place in society as well.

Further reading:

http://onlineqda.hud.ac.uk/resources.php#Ethnography