Microsoft Desirability Toolkit Product Reaction Words

The faces questionnaire proved problematic for some users, who had difficulty understanding the emotion portrayed in the face and explaining the reason for their rating. So, it was dropped from the toolkit.

Using the product reaction cards in usability tests

Here’s how we use the cards in our usability tests.

For in-person studies, we have made laminated cards for each of the 118 words. We laminate them so that we can reuse them, but it’s not a requirement. You can make a new set for every study, if needed.

We spread the cards along a table so that the cards are touching but not overlapping. When the user completes the last task, we direct them to the table and provide the following instructions:

In addition to showing the results in a word cloud, we often include a table of the positive and negative words chosen with a count of how many times each word was selected by each user. For example, the table below shows a partial list of the words presented in the word cloud, indicating the number of times it was selected and whether it was deemed a positive or negative word.graph

Reaction cards are great in small studies

You don’t need big study numbers to get great results from using the product reaction cards. Here are the cards from a single study with six participants in a day.

The same powerful consistency is clear in the results.

Reaction cards are great in iterative studies

We can also use the reaction words to establish a baseline for user satisfaction in the first study, which can then be used in iterative studies to track improvements in user satisfaction as the product goes through development or redesign.  The example below is from 3 studies of a web-based application for hotel properties worldwide to implement and monitor green initiatives.

User testing of the client’s first version demonstrated that the general idea of the application was motivating; however, the product had significant problems that slowed or stopped users from achieving success. Participants’ repeated positive card choices were low with only "comprehensive," "professional" and "usable" selected twice each; but the themes of quality, appearance, ease-of-use, and motivation emerged from participants’ card choices.

The second study was of the prototype of the redesigned application. The transformation of the users’ experience was night to day. The positive card choices from 12 users now represented 82% (compared to 42% in the first study) with the most often selected card being "useful."

With such a positive and significant measurement of change, the development team focused on the remaining issues, and a small test of the pilot version was conducted with 4 users just before launch. The pilot version results showed that all participants chose only positive words. The theme of speed now predominated and confirmed that the earlier negative issue of slow speed was now a positive feeling of fast speed—the application was fast, time-saving, and efficient.

Using product reaction cards in your UX studies

You can do this. Using the Microsoft reaction cards is easy and the results are powerful. What’s more, Microsoft has given us permission to use them with only the following disclaimer:

“Developed by and © 2002 Microsoft Corporation. All rights reserved.”

You don’t have to do complex diagrams to show the results. You can just count the number of times each word is chosen and present these results.

Or use a free word cloud generator to turn the word counts into a cloud. Then you can do some quick clustering of similar words to present themes emerging from the words users chose.

And, of course, use these results along with your other feedback mechanisms, such as The System Usability Scale, to compare your results from different perspectives. By triangulating your data from different sources, you strengthen your case for understanding what users showed you about their experience.