UX Firm: Your Atlanta-based solution for usability research and testing.

Heuristic evaluations

Here are some questions we get asked frequently about heuristic evaluations.

 

QWhat are heuristic evaluations?
AHeuristic evaluation is a review or “inspection” of an interface or product by experts, using a set of formal or informal heuristics to guide the review and document issues.
QWhat are heuristics?
AHeuristics are guidelines of good practice.The most widely used formal heuristics are developed by Jakob Nielsen and Rolf Molich (1990) and updated by Nielsen (1994).These 10 heuristics were developed for use in evaluating software, but they can be modified to work with websites, apps, and more.
QWhat’s the difference between heuristic evaluation and expert review?
AThere are no clear distinctions between these two ways of characterizing an inspection. Some practitioners use “expert review” to mean a review by a single expert and “heuristic evaluation” to mean a review by several experts. Other practitioners use heuristic evaluation to mean a review using a clearly defined set of heuristics, such as those by Nielsen. In contract, these practitioners would call the inspection an expert review when the heuristics are internalized through familiarity and practice.
We use the terms interchangeably to mean the same thing, and we adopt the methodology best suited to the client’s needs with regard to the choice of heuristics (or not) and the number of reviewers.
QHow many experts make for an effective heuristic evaluation?
ANielsen, who is credited with combining the practices of a small set of evaluators for a heuristic evaluation and a small set of users for a usability test into a method called “discount” usability engineering, found that the maximum cost-benefit ratio for heuristic evaluation is achieved with 3-5 evaluators.
That’s a lot of experts! Most practitioners use fewer experts for reviews, with some doing reviews by a single expert. We advocate using 2-3 reviewers for effective results.
QWhat’s the methodology used in a heuristic evaluation?
AIn formal practice, each evaluator does an independent inspection of the interface, documents violations of the heuristics, writes notes, and then joins a meeting with the other evaluators to compare notes and arrive at consensus findings.
Severity ratings are often assigned to the findings to prioritize the need to address issues uncovered.
Although not part of the original guidelines presented by Nielsen, practitioners today often conduct their review from a typical scenario of use for a specific type of user or persona.
We conduct this type of persona-based, scenario review.
QWhat’s the difference between heuristic evaluation and usability testing?
AUsability testing provides feedback from users while they engage in typical tasks. Heuristic evaluation provides feedback from usability experts (who may or may not be domain experts), based on anticipated issues users will face.
QWhich is better? Heuristic evaluation or usability testing?
ABoth tools are widely used, and both produce useful findings. So, one is not better than the other. However they do produce different kinds of findings:

  • Heuristic evaluations tend to find more of the minor issues that may not emerge in usability studies.
  • Usability testing tends to find some issues that experts miss in heuristic evaluation.

Recent research indicates that both methods tend to find roughly the same number of important findings.

QIf you can only choose one method, which one should it be?
AIf your budget does not allow for using both methods, we strongly recommend choosing usability testing over any other method. Nothing beats learning directly from users.
QIf you can use both methods, which one comes first?

AThe one-two punch of heuristic evaluation and usability testing is the best combination of tools for most situations.
The results tend to overlap in some ways and diverge in others, producing findings that substantiate user experience from two different perspectives: experts and users.
The way we typically use both with clients is we start with a heuristic evaluation, if an interface or product exists, so that we can identify usability issues. Depending on the timeline, we advocate one of the following approaches:

  • If time permits, the issues identified in the evaluation can be addressed before testing to improve user experience during testing and get at some of the larger issues.
  • Expert review (heuristic evaluation) to inspect a product or interface against a set of rules or guidelines
  • If there isn’t time to make changes to the interface, the issues identified can become the basis for tasks and scenarios to be used in testing.
  • Surveys to broaden the reach to your users and gain feedback

Contact UX Firm to start a discussion about usability heuristics for your company’s heuristic evaluations.