UX Firm: Your Atlanta-based solution for usability research and testing.

Microsoft’s Product Reaction Cards Unlock User Satisfaction. Part 1

By on Jun 7, 2016

How can you really tell if users are satisfied? If the product is desirable to them? If they mean what they say when they respond to structured survey questions giving you feedback on their experience? We find out by using Microsoft’s product reaction cards.

We love the cards because they give users control over the story of their experience following a usability test. We find them especially useful when combined with a quantitative feedback mechanism such as The System Usability Scale.  In this 3-part series, we’ll tell you about the origin of the cards, how they work, and how we use them in our usability studies.

In the beginning, Microsoft Created a Desirability Toolkit

In 2002, Two UX researchers at Microsoft, Joey Benedek and Trish Miner, set out to create a feedback mechanism to understand the intangible aspects of a user’s desire for a product. They wanted to know if users found products fun, enjoyable, and desirable enough to purchase.

The original “Desirability Toolkit” [DOC] had two aspects:  (1) a faces questionnaire, in which participants were asked to rate how closely their experience related to each of 6 different faces using a 7-point Likert scale to indicate their response; and (2) a card selection exercise, in which participants were asked to choose descriptive words or phrases from a large set of product reaction cards.

The Faces Questionnaire Proved Problematic

Here’s an example of a question using a face.

Look at the picture and circle the rating on the scale that most closely reflects how performing the tasks today made you feel.

faces question Not at all like this Very much like this
1 2 3 4 5 6 7

The faces questionnaire proved problematic for some users, who had difficulty understanding the emotion portrayed in the face and explaining the reason for their rating.  So, it was dropped from the toolkit.

 

The Product Reaction Cards Are a Winner

The second part of the toolkit, the product reaction cards, consisted of a large set of cards that participants used to select words that matched their experience.

We all know that if we ask users how they feel about a product’s desirability, they’re likely to inflate their responses to be more positive than what we observed as they engaged with the product.  And by product, we mean any interaction the user has with the interface. The reason they say nicer things than what we observed is because they want to be nice; they want to please us. It’s human nature.  Microsoft kept this tendency toward a positive response in their creation of the cards, with 60% being positive words and 40% being negative words.

The list started out with 75 words but developed over several studies into the following final list of 118 words.

The complete set of 118 Product Reaction Cards
Accessible Creative Fast Meaningful Slow
Advanced Customizable Flexible Motivating Sophisticated
Annoying Cutting edge Fragile Not Secure Stable
Appealing Dated Fresh Not Valuable Sterile
Approachable Desirable Friendly Novel Stimulating
Attractive Difficult Frustrating Old Straight Forward
Boring Disconnected Fun Optimistic Stressful
Business-like Disruptive Gets in the way Ordinary Time-consuming
Busy Distracting Hard to Use Organized Time-Saving
Calm Dull Helpful Overbearing Too Technical
Clean Easy to use High quality Overwhelming Trustworthy
Clear Effective Impersonal Patronizing Unapproachable
Collaborative Efficient Impressive Personal Unattractive
Comfortable Effortless Incomprehensible Poor quality Uncontrollable
Compatible Empowering Inconsistent Powerful Unconventional
Compelling Energetic Ineffective Predictable Understandable
Complex Engaging Innovative Professional Undesirable
Comprehensive Entertaining Inspiring Relevant Unpredictable
Confident Enthusiastic Integrated Reliable Unrefined
Confusing Essential Intimidating Responsive Usable
Connected Exceptional Intuitive Rigid Useful
Consistent Exciting Inviting Satisfying Valuable
Controllable Expected Irrelevant Secure
Convenient Familiar Low Maintenance Simplistic

 

In part 2, we’ll talk about how we use the cards in our studies.