How can you really tell if users are satisfied? If the product is desirable to them? If they mean what they say when they respond to structured survey questions giving you feedback on their experience? We find out by using Microsoft’s product reaction cards.
We love the cards because they give users control over the story of their experience following a usability test. We find them especially useful when combined with a quantitative feedback mechanism such as The System Usability Scale. In this 3-part series, we’ll tell you about the origin of the cards, how they work, and how we use them in our usability studies.
In the beginning, Microsoft created a Desirability Toolkit
In 2002, Two UX researchers at Microsoft, Joey Benedek and Trish Miner, set out to create a feedback mechanism to understand the intangible aspects of a user’s desire for a product. They wanted to know if users found products fun, enjoyable, and desirable enough to purchase.
The original “Desirability Toolkit” had two aspects: (1) a faces questionnaire, in which participants were asked to rate how closely their experience related to each of 6 different faces using a 7-point Likert scale to indicate their response; and (2) a card selection exercise, in which participants were asked to choose descriptive words or phrases from a large set of product reaction cards.
The Faces Questionnaire proved problematic
Here’s an example of a question using a face.
Look at the picture and circle the rating on the scale that most closely reflects how performing the tasks today made you feel.
|Not at all like this||Very much like this|
The faces questionnaire proved problematic for some users, who had difficulty understanding the emotion portrayed in the face and explaining the reason for their rating. So, it was dropped from the toolkit.
The Product Reaction Cards are a winner
The second part of the toolkit, the product reaction cards, consisted of a large set of cards that participants used to select words that matched their experience.
We all know that if we ask users how they feel about a product’s desirability, they’re likely to inflate their responses to be more positive than what we observed as they engaged with the product. And by product, we mean any interaction the user has with the interface. The reason they say nicer things than what we observed is because they want to be nice; they want to please us. It’s human nature. Microsoft kept this tendency toward a positive response in their creation of the cards, with 60% being positive words and 40% being negative words.
The list started out with 75 words but developed over several studies into the following final list of 118 words.
|The complete set of 118 Product Reaction Cards|
|Annoying||Cutting edge||Fragile||Not Secure||Stable|
|Business-like||Disruptive||Gets in the way||Ordinary||Time-consuming|
|Busy||Distracting||Hard to Use||Organized||Time-Saving|
|Clean||Easy to use||High quality||Overwhelming||Trustworthy|
In part 2, we’ll talk about how we use the cards in our usability studies.