Archive for the 'qualitative research' Category

Data-driven social interaction: The difference between analogue and digital part III

June 30, 2009

Data-driven social experience is an entirely new manner of social interaction, one that obscures our emotional connections to people. Data makes social relationships visible, knowable, and countable in unprecedented ways. But it does not — and cannot — convey the emotional experience of social interaction. I’ve already discussed how digital technologies transform text and time. Now I want to explore how “data” transforms social experience.

Take the notion of the “social network.” Most people (especially those that read blogs!) think these synonymous with Web sites like Facebook. Truth be told, social network analysis has existed for almost a century. We’ve all heard the term “six degrees of separation,” but most of us don’t know that was coined by none other that Stanley Milgram, of the “shock experiments” fame, when he tracked letters mailed around the world.

Social networks are exceedingly difficult to know from a quantitative perspective. We all live inside social networks but we have a very hard time knowing how these networks are constructed. We may know, for example, that our friend Jeff is friends with another one, Sarah, but we don’t know if Sarah knows Jeff’s partner Sam. Social network analysis is a set of methods designed to learn exactly that.

Now imagine your social network, as it is represented on Facebook (what, you’re not on Facebook?). Below is an image from Visual Complexity that renders a social network visibly but also very easily, simply by mining the data inherent in Facebook’s structure:

from Visual Complexity

from Visual Complexity

Note how we instantly and easily know how institutions are connected, and through which people. Previously, researchers would have to conduct extensive and expensive surveys to get these data. Now these data are easily calculated and visualized by anyone with access to a social network online.

Some people are talking about this visualization as a piece of intellectual property. Alex Iskold on Mashable, for example, asks “Who owns the social map?” I go further and ask, “What does it mean that our social world is mappable?”

Our social world is now infiltrated by masses of data. These data inform us about the structure of our interactions with others in ways that we could not recall correctly if asked. Suddenly we can now see our social world reflected back to us, punctuated by  institutions, and social structures. When we see our social network through the eyes of data, we see the names of organizations, or the institutional affiliation of the people. We do not “see” the emotional experience that created our connections in the first place.

Suddenly, we may think we really are not that close with Jeff, because his partner Sam is really not friends with anyone I know. I can also see that Sarah and I have very few friends in common, which may lead me to think I don’t have much of a future friendship with her.

Those data crowd out the qualitative, embodied experience of the laughs I shared with Jeff and Sam at their cottage last summer. Those data obscure the fact that Sarah and I shared 3 long months as call centre employees together, a time that bonded us forever. A data-filled social world is one that masks the visceral, emotional experiences of face-to-face interaction.

Digital social life is revealed to us in fragmented, mashed up ways. Such ways were impossible before the freely available data on social networks, data that is now so ubiquitous, we don’t even see it.

The Difference Between Analogue And Digital Part II: Time

June 16, 2009

In an earlier post, I examined how text is transformed when it is created and shared in digital form. In this post, I argue that time itself is transformed when it is represented in digital format. To illustrate, consider my experiment with my Filofax.

Yes, I said Filofax. I still have one. I haven’t filled it with inserts in years, even though that was actually one of my favourite end-of-year rituals. I would make a special trip to the stationary store, just to buy the next year’s worth of calendar. In the process, I would review last year’s appointments, marvel at how much I had gotten done and how fast time had passed. I would linger over favourite appointments, which seemed, at the time, inconsequential, as recorded in my scribbled hand.

I bought a 2009 insert for my Filofax and inputted only two weeks’ worth of appointments. It took me 20 minutes.

Analogue time "reckoning"

Analogue time "reckoning"

The time it took me to enter in all these appointments was more than just scribbling. It was reviewing, remembering, considering. I could not physically enter overlapping appointments. There simply wasn’t room!

Now compare this to the same amount of time, as rendered by my iCal:

Digital Time "Reckoning"

Digital Time "Reckoning"

There are overlapping appointments, my husband’s appointments easily inputted into mine, meetings from people I barely know, all dropped into my life automatically. Worse, I carry this around, automatically updating it, second by second, through my iPhone.

Sociologists use the term “time reckoning” to describe how we collectively understand time and make it intelligible to ourselves. There was a great hullabaloo about “clock time,” when clocks came to replace the seasons as our primary way of time reckoning. We forgot we didn’t know how long a minute actually was — we actually now think we can tell how long 23 minutes and 42 seconds is (spoiler: we can’t, especially when we’re enjoying ourselves!).

Now we have “digital” time reckoning, which bears almost no resemblance to how we actually experience time. If you have the misfortune of using time tracking software like TimeControl, then you will likely recognize this fantastical, farcical, FrankenTime:

Screenshot from Microsoft's TimeControl

Screenshot from Microsoft's TimeControl

According to this, a mythical interaction designer named Joseph Gardner spent 8 hours and 20 minutes on Sunday “design interface.” Ignoring the assault on proper grammar for a moment, let’s take a step back and understand what this means. First off, Poor Old Joe was working on Sunday. Notably, TimeControl allowed this kind of time use, despite the fact that it likely broke overtime laws. But secondly, how long is 8 hours and 20 minutes? Did Joe forgo the need for bathroom breaks? Was he glued to the chair for precisely 8 hours and 20 minutes? How long did he actually spend in that chair anyway?

Digital time allows to represent time in impossibly tiny fragments, and to work impossibly long hours. This kind of time would never be recorded in one’s Filofax — there simply isn’t room for all those hours. Moreover, the time it takes to record one’s time in a Filofax also requires one to contemplate the implications of 8 hours and 20 minutes of work on a Sunday.

In short, the difference between analogue and digital time is that digital time is even less like cognitively experienced time than “clock time.” Digital time can be schedule effortlessly, without any thought to the physical need for sleep, food, or relaxation. Digital time is a faster, manifold version of clock time, one that makes it possible for use have multiple, synchronous events.

Designing for conversations: the critical importance of turn taking

June 11, 2009

Hugh Dubberly and Paul Pangaro had a great post on Interactions magazine about designing for conversations. They propose to use how a conversation actually works to make interactions better. They rely heavily on Claude Shannon’s conversation model to help guide the conceptual model of interaction designs.

In Shannon’s model an information source selects a message from a known set of possible messages, for example, a dot or a dash, a letter of the alphabet, or a word or phrase from a list. Human communication often relies on context to limit the expected set of messages.

I applaud Dubberly and Pangaro’s attempts to use rigourous theory to support interaction design. But I’d have to agree with Peter Jones as he wrote in the comment section, that other philosophically informed communication theories are more robust when it comes to designing for conversation. Peter specifically mentions Winograd and Flores’s “conversation for action model” which relies on Habermas’s contention that you are acting when you communicate.

I’ll add to Peter’s critique. Garfinkel’s ethnomethodological approach gave way to “conversation analysis,” which posits that speakers use “indexical expressions” (or phrases that are fraught with meaning but are meaningful to the participants through unspoken means). Where in Dubberly and Pangaro’s article is the discussion of such expressions?

Where also is the notion of turn taking? Turn taking is a very significant component of a conversation. Try to have a trans-atlantic mobile phone conversation and you’ll see how important smooth turn taking is to meaningful conversation.

I would exhort interaction designers to continue to read and integrate theory into their mental models. But I would also discourage them from taking the short route; theories are debated for a reason. Interaction design ought to be a robust digital representation of those debates, and include all aspects.

Improving participation rates: research recruitment best practices

May 23, 2009

Those of you out there who’ve tried it know: recruiting research participants is HARD. Here are a few insights from the research to help you with better recuitment.

  1. Personalized contact with respondents, followed by pre-contact and aggressive follow-up phone calls *: Don’t count on a form letter, email or random tweet to do the job. Capitalize on your personal relationship with that person. If you don’t have a personal relationship, ensure that you use the person’s name, and for God’s sake, spell it correctly!

    Once you’ve made initial contact, you are not done. Not by a long shot. Make sure you speak to the person (you can do this through IM or email if you’d like) to give them more information. They’re now interested. Don’t stop! One more step!

    Follow up 1 week after initial contact. Assuage any fears they may have. Answer any questions honestly. And above all, be available for more information.

  2. External researchers with social capital are best**: University-based researchers have been shown to have the best participation rates, but you don’t have to be a professor.  Researcher Sister Marie Augusta Neal of Emmanuel College achieved a near perfect response rate because of her close ties to the respondents and their communities. The lesson here is, if you hire a consultant, make sure they’re trusted. Even better if they personally know the people to be recruited.
  3. Monetary incentives have no effect, unless money is offered “no strings attached”***: Little known fact: the best way to use a monetary incentive is to offer it, up front, with absolutely no strings attached. The “free” money makes people feel more indebted socially. Evidence of this effect can be found in the book Freakonomics. Researchers found that daycare centres that levied late penalties on tardy parents actually had more of a late-pickup problem than those that levied no fine. Why? Because the parents reduced their relationship to the daycare as a mere transaction. Use the “gift economy” approach and ensure a feeling of indebtedness. My personal favourite is a coupon for a single iTunes song at $.99. It is cheap but appears to have great value. Offer it, up front, and then ask for participation

*  Cook, C., F. Heath, and R. Thompson. 2000. “A Meta-analysis of Response Rates in Web or Internet-based Surveys.” Educational and Psychological Measurement 60:821-836.

** Rogelberg, S., A. Luong, M. Sederburg, and D. Cristol. 2000. “Employee Attitude Surveys: Examining the Attitudes of Noncompliant Employees.” Journal of Applied Psychology 85:284-293.

***Hager, M., S. Wilson, T. Pollak, and P. Rooney. 2003. “Response Rates for Mail Surveys of Nonprofit Organizations: A Review and Empirical Test.” Nonprofit and Voluntary Sector Quarterly 32:252-267. Singer, E. (2006) Introduction: Nonresponse Bias in Household Surveys. Public Opinion Quarterly, 70, 637-645

The evolution of qualitative sociology

May 8, 2009

The blog Economic Sociology has a great post on the “evolution” of qualitative sociology. They note, quite rightly, that the notion of “evolution” is implicit in much of social science, even if it has no bearing on the subject matter at hand. Many sociologists place quantitative research “on top” of the research “evolutionary ladder,” even when there is no such thing as a ladder when it comes to good research design. Interestingly, the fathers of sociology themselves would be on the “lower rung” of that methodological ladder:

The works of Marx and Weber, like virtually all the classic literature in the field, were based on qualitative, historical methodology (Durkheim’s quantitative study Suicide being a notable exception).

This post just reinforces to me why the design process is so important for social scientists. One must design a research project to solve contextual problems, just as one designs, say, a chair. You cannot “solve” questions of why or how by using quantitative methods. It is simply impossible.

Are why and how somehow “less than” questions than “how many” or “how fast”? I don’t think so. Indeed Economic Sociology points out that even Darwin was not concerned with “how many” but more with “how,” and few accuse him of being “unscientific.”

#TOEthno: is Twitter a “place”?

January 4, 2009

I’m currently forming research questions for an ethnography of Toronto-based technology and design workers. I am working through this question: is Twitter a “place”?

In her 2000 book Virtual Ethnography, Christine Hine argues that there are two analytic strategies to see “cyberspace.” First, one can view it as a “place,” where social norms emerge. Or second, one can view it as a cultural artifact. The second view allows us to see the designers behind the technology. Think of it as a hermeneutics of a technology, which allows us to see what assumptions its designers about their users (this is an approach that will make sense to interaction designers).

I believe Twitter to be a place, but one that is heavily influenced by its architects and its users. In other words, its design sets the stage for certain kinds of interactions, just as prisons, malls, and casinos do. The architecture of Twitter, which includes its dozens API-driven applications as well as its simple, Web-based interface, is constantly evolving by its network of users, API application designers, and the company of Twitter itself.

This approach suggests that Twitter has “interpretive flexibility,” which is how technology theorists argue that design is determinant; users decide how a technology will actually be used, within the confines of the material form of that technology.

Do you believe Twitter is a “place”? What kind of place? Or is Twitter a technology or technological artifact?

Personas are “empathy tools,” not stereotypes

November 17, 2008

Bookmark and Share

We all use personas in everyday social situations. But, like in many design projects, we use to them to typecast instead of to evoke empathy. Personas, like stereotypes, often result in discriminatory behavior. When used in design, personas can create poor design that disempowers and alienate users.

We all like to know how to treat people appropriately. We tend to use what social theorists Berger and Luckman call “typifications” when interacting socially. When we go to the store, to a meeting, to a party — we need to know how to act with people. We genuinely want to make people feel comfortable and we want to feel comfortable ourselves.

But to use a typification often has the unintended consequence of being condescending. Elderly people are spoken to in loud, exagerrated tones. Women are assumed to be physically fragile. Men are considered to be sexually aggressive. These typifications are stereotypes that affect how we, in turn, react. Elderly people may react angrily, for example, at the implied loss of their faculties.

Designers often make the same mistake when making personas. Personas are tools to evoke empathy. But poorly created personas will simply regurgitate stereotypes instead of actually answering real needs. When a site is designed “for women,” it should allow women (and all of its users) to define their experience, according to their needs. Women may have more need to juggle schedules, for example, so interactive experiences should allow them to adopt such features.

An interactive experience should not, however, force me to be treated as a “mom on the go” simply because I’m a woman. And honestly, if there’s one persona phrase that makes me want to vomit/go on a murderous rampage/re-design the design process, it’s the dreaded “mom on the go.” Show me a mom NOT on the go, and I’ll show you a mom who forgets she has children.

Worse, don’t treat me, a childless woman of 38, as a “mom on the go,” simply because YOUR data tell you I should have children. Instead, empathize with me. Allow me to satisfy unmet needs, should I so choose. DO NOT force me to adopt features and functionality that are appropriate for what you think I OUGHT to need.

As a woman, I am frequently “treated” to “gentle” behavior. People will open doors for me, or perhaps allow me to pass first out of a crowded elevator. This is not because I require it, nor because I expect it, but because it is believed that women still are the “gentler sex.”

Defeating the problem of personas as stereotypes is to put yourself in the user’s shoes. In other words, don’t forget that personas are empathy tools. Allow her to choose her experience. Provide her the features and functionality that she MIGHT like, based on your qualitative research. But under no circumstances force her to adopt features or functionality that reproduce what someone “ought” to be.

Forcing people to adopt behaviors is as far from empathy as one can get. Interactive experiences that foist “mom on the go” fantasies onto real people risk alienating their users at best; at worst they perpetuate sexist stereotypes.

Sampling methods in qualitative and quantitative research

October 30, 2008

Why does sample size not matter in qualitative research? Because of the assumptions that qualitative researchers make, namely, that the social world is not predictable. Qualitative researchers believe that people are not like molecules or other objects; people’s actions are not predictable.

But quantitative researchers DO believe that social activity IS predictable. So when they compare their observations of social activity to what would happen in purely random results, the difference says something. Let’s say you were to research people’s preferences for a particular interactive feature. Say you’re wondering if young people will like a radio button more than older people. First, you model what results you’d expect if you’d just flipped a coin. Then you use a probability (random) sample, and compare those results to purely random results. Is there a difference?

If there is a difference between them, you can infer that indeed, something other than chance (in this case, age) affect people’s preferences.

Qualitative researchers don’t agree that such things can be reliably predicted. That’s why they don’t bother with expensive and involved random sampling. See all these details below from my research design course.

Qualitative versus quantitative research, Part II

October 8, 2008

Thousands of people arrive at this blog wanting to know what is the difference between qualitative and quantitative research. Qualitative versus quantitative research is by far the most popular post on this blog. In that first post, I explained why sample size doesn’t matter in qualitative research. In this post, I explain why qualitative research is generally a better approach for design research.

Notice how the qualitative process is iterative with the going back and forth from data to sense-making or developing theory. It is flexible and can change direction easily.

Qualitative design process

Double click for a larger image

Double click for a larger image

And the quantitative design process is very linear, and does not include an iterative component:

Double click for larger image

If your design process involves an iterative prototyping phase, for example, then qualitative research is likely the best approach for you. Notice also that qualitative research necessarily involves the researcher putting herself in the shoes of the user. Quantitative research does NOT require the researcher to see through the eyes of the user.

Designers often want to empathize with their users. They want to understand their experiences and pain points. They want to know what their users are thinking. This is why qualitative research is often better suited to design research.

See also this embedded slideshow from my research design class. This should give you the basic differences between the two.

Research Design Course: Follow along on slideshare

September 14, 2008

I am currently teaching a Research Design and Qualitative Methods course at Ryerson University. This is a core course for an interdisciplinary group of students, from social work, to business, to psychology, to sociology to…well you get the picture.

I will be uploading slides from my lectures regularly. See them all at:

http://www.slideshare.net/sladner

I have toyed with adding audio, but so far my students do not appear to be too interested. Are you? If so, let me know and I will add audio to my slide space.