Eye Tracking the User Experience. Aga Bojko
research methods (e.g., observation and user interview) can answer the study questions (which is, in fact, often the case), the use of eye tracking may be unjustified.
The third and last question, “Does my study need a buy-in boost?” should resonate with many UX researchers. You can’t make a difference if you don’t have others on your side, and that sometimes requires adding “color” and perceived credibility to your results. Even if the answer to the second question is “no,” a “yes” to this one gives you the green light, as long as the first question is a “yes” (meaning you have made sure that actionable insight will result from eye tracking).
Actionable Eye Tracking
When eye tracking proponents state reasons for using eye tracking, they frequently frame them as “Here is what eye tracking can tell us.” For example, “Eye tracking can tell us about users’ search strategies and decision-making processes” or “Eye tracking can determine what users find interesting.” These statements are not incorrect. However, they are vague and not very helpful to those trying to figure out when to use eye tracking in their studies.
Does it matter that you can identify a visual strategy employed by users to find the correct drug among others (see Figure 2.2)? Or that you can find out how interesting each area of a Web page is? While it is certainly true that some UX research (especially the more academic research) aims to explore how people interact with products and interfaces in general, as UX practitioners, we must often focus on gathering actionable information. We conduct research to inform decisions. “How can we improve this product?” or “Is this product ‘good enough’ to be launched?” are the high-level questions that our studies typically have to answer. Eye tracking can provide copious information, but most of it may never be useful for answering these kinds of practical questions.
FIGURE 2.2 You could collect and analyze eye movement data from pharmacists looking at drug labels when filling prescriptions, but how can you make these findings actionable?
Let’s say that furniture assembly instructions were the object of your study (see Figure 2.3). Your eye tracking data revealed that participants spent 10 seconds looking at the front cover before opening the booklet and reading the instructions. If your study goal were to assess instruction comprehension, this is not a useful finding. The same information could be considered useful, however, if the goal of the study were to evaluate the effectiveness of new advertising placed on the cover.
The starting point is always the high-level study objectives. More specific research questions should be then developed to target each of these objectives. Once those are in place, the next step is to identify the appropriate methodology for the study, which could but does not have to include eye tracking. One unfortunate but common practice is to decide to conduct an eye tracking study before specifying the research questions. Don’t do it. You cannot possibly know which methods to use without knowing what the study is trying to accomplish!
FIGURE 2.3 Eye movements of a person reading instructions can reveal both useful information and information that is of no use to the researcher and stakeholders. It all depends on the study objectives.
Just as bad as having no objectives or clear research questions is the infamous, “We just want to know where users are looking.” Let’s settle this once and for all: This is not a good enough reason to use eye tracking. Doing eye tracking just because you want to see where people are looking is like eating donuts for breakfast, lunch, and dinner just because you can.
So rather than just focusing on what eye tracking can tell you, you should ask yourself, “Why do I need to know this?” or even better, “What type of decisions do I want to be able to make based on the study results?” These questions will help you look for actionable insight and determine techniques that can gather it.
Figure 2.4 shows an overview of the actionable insights that eye tracking can generate and the kinds of decisions it can help you make. This information will help you answer the first question from Figure 2.1, “Will eye tracking generate actionable insight that addresses the study objectives?” If there is no overlap between your research questions and what you see in Figure 2.4, you should consider employing methods other than eye tracking to find your answers. But wait, you don’t have to make a decision just yet! The next two sections of this chapter explain and provide examples of actionable qualitative and quantitative research findings that can result from eye tracking.
FIGURE 2.4 Types of actionable insight that eye tracking can generate.
Qualitative Insight: Detecting and Explaining Usability Issues
One reason to use eye tracking is to obtain qualitative insight into users’ cognitive processes during formative research. This insight sheds light on the user experience that takes place during and in-between easily observable events such as mouse movements and clicks, physical object manipulations, and participant comments. Eye movements help reveal the often-not-fully-conscious processes that led to these outcomes. This information can be used to detect and explain usability issues. This is where the actionable part comes into play—a thorough understanding of usability issues and their sources leads to more specific (and thus more useful) design recommendations.
Qualitative insight can be obtained after data collection, as described in Chapter 12, “Qualitative Data Analysis.” There are also two ways to take advantage of the eye tracking data during testing. First, when possible, the moderator can watch participants’ eye movements live during the study and use them to inform her probes. The second way takes advantage of the gaze-cued retrospective verbal protocol (RVP). Unlike the traditional concurrent verbal protocol (CVP), where participants are asked to describe their thought process while performing tasks, RVP involves participants’ verbal account after the task, which was initially completed in silence. During gaze-cued RVP, participants are shown a replay of their eye movements superimposed on a video of their actions. The replay is meant to serve as a memory aide as the participants walk the moderator through their experience (see Figure 2.5). This method is discussed in detail in Chapter 6, “Verbal Protocols and Eye Tracking.”
FIGURE 2.5 The gaze-cued retrospective verbal protocol involves a participant watching the replay of his gaze post-task while providing commentary.
Detecting Usability Problems
Eye tracking is sometimes able to detect usability problems even when conventional usability testing methods such as behavior observation and participant feedback elicitation don’t indicate that there is a problem. For example, let’s say that a study participant