The key feature of this methodology is the capacity to capture details that otherwise would be lost, without compromising the integrity of the interview.
One of the foundations of qualitative research is the “gut reaction” — that intangible, yet unmistakable sensation immediately following the first glimpse of something new. It strikes instantly as an overwhelming “I must have it” feeling or a permeating sense of “I can’t stand it.”
The gut reaction influences purchase decisions. It colors lasting perceptions. And it serves as the cornerstone upon which marketing researchers have balanced many of their observations for decades. No matter what the context — focus group, IDI, observational setting — the initial reaction of the consumer to the website, product, advertisement or television program is always a critical piece for understanding consumer motivations.
The logic behind traditional qualitative methods of capturing this gut reaction is fairly intuitive: present the test material to people, and then find out what they think. Sometimes, this involves allowing a respondent to “think aloud” while using a website for the first time. Other times, it is as simple as showing a commercial and asking viewers to discuss their impressions alone or with a group.
The methods are diverse, but the ultimate goals are the same. By capturing these snapshots of perception, researchers gain an invaluable glimpse at the budding attitudes and opinions of the potential customer. This information is extremely important in identifying positive and negative aesthetic, usability and design features.
Drawbacks of Traditional Research Methods
Despite the aforementioned strengths of traditional methods, some researchers level criticisms against them based on the potential for creating bias at the moment of the first impression. Social pressure and group dynamics are often cited as factors that can distort perceptions and reactions during qualitative interviews.
Also, the think-aloud methodology has been suggested to create an unrealistic scenario on which to base research conclusions. Indeed, compelling evidence in the literature of cognitive and social psychology suggests that the simple act of asking someone to voice his or her opinion has a tendency to influence the formation of that opinion.
These critics advocate a more passive approach to research, allowing respondents to fully interact with a website, find a product on a crowded shelf or watch a commercial in the context of a television program without interruption. The interview comes only after the respondent has been given ample time to form an opinion.
While this hands-off approach may be useful in creating a realistic interaction, a great deal of important information may be no longer accessible when respondents are finally asked to provide feedback. In the time between the first impression and the beginning of the interview, a lot can happen. People forget. They change their minds. They make up stories without even knowing it. The gut reaction is a fleeting impulse, and once the moment has passed, the sensation is difficult to retrieve.
With these criticisms in mind, qualitative researchers have two choices: (1) jeopardize the accuracy of the gut reaction by posing questions while impressions are being formed or (2) jeopardize the accuracy of the gut reaction by waiting too long to pose questions. Using traditional methods, this is likely to be an enduring dilemma for qualitative researchers.
A Possible Alternative
However, there is hope in technology. New advances in the field of eye-tracking provide a means of getting around this problem. A methodology called an ActionReview Interview™, developed by San Diego-based EyeTracking, Inc., allows a respondent to relive the gut reaction in vivid detail retrospectively.
Small, high-speed cameras are used to record the eye movements of a respondent as he or she interacts with the new stimulus. These cameras record every blink, scan and fixation, gathering up to 250 data points each second. Once the respondent has finished interacting with the test material — navigating the website, choosing products from the grocery store shelf, watching the television segment, etc. — the researcher conducts the interview while showing a video of exactly where the participant was looking, second by second, throughout the testing session.
It may sound somewhat complicated with all this talk of sophisticated equipment and 250 data points per second, but the concept possesses a simple elegance: showing respondents exactly how they visually interacted with a product allows them to remember their gut reaction with far more accuracy than any other method available.
Currently, the most common application of the ActionReview Interview is in the field of webusability research. The method allows an extremely realistic website interaction without risking the richness of real-time respondent observations. The respondent is allowed to interact with the site unaided — completing tasks, stumbling with new features, enjoying content and continually forming and revising impressions. If later asked to recall their impressions without the aid of eyetracking video, respondents tend to generalize (i.e., “I had trouble” or “It was easy touse”). Respondents often cannot recall momentby-moment impressions; thus, information about usability strengths and weaknesses is lost.
However, when the respondent is provided with a visual cue that shows in great detail where he or she was looking, a wide array of specific information becomes available (i.e., “I was searching everywhere, but couldn’t find X,” “I expected Y to be in the top navigation,” “Z was the first thing I noticed,” etc.). As each moment of the session is replayed, the respondent is able to relive that thought process and express it to the researcher to create a clear narrative of the interaction. The researcher is then able to couple this data with his or her own impressions of the actual interview to complete the picture. It is the difference between knowing something is not working and knowing why it is not working. The obvious benefit of knowing why is that you gain the information needed to fix the problems.
Eyetracking in Action
The following is an example of using this method to evaluate a new version of a homepage for a popular website. Respondents with experience using the existing website version were asked to complete a series of tasks to evaluate the new version. These tasks were completed without assistance, and the moderator did not ask questions during the session.
Not surprisingly, respondents experienced significant problems with the new design. When asked about the new version immediately after the interview, most respondents expressed substantial discontent and pleaded for the site to be returned to its previous format. Had the research stopped there, it is likely that we would have had to return to the client with a recommendation to scrap everything.
However, once the respondents were shown the video of their testing session, they were able to see the good and the bad. As they watched their eyes moving around the site during their first exposure, many recalled that they were immediately intrigued by the sleek new design.
While observing their own visual interaction during tasks, they often noticed that, in hindsight, some of the things that they were asked to do were very intuitive. For example, they were able to explain what they were initially thinking when they began to search for a link to local weather. They were able to express how easy it was to find their inbox. They made comments like, “Oh, if I would have seen the navigation feature initially, I could have completed tasks much more easily.”
There was still some dissatisfaction with the new design, but what had started as an exercise in tearing down all that was new and unfamiliar became a constructive discussion of how the new design could be adapted to create a more comfortable transition. Respondents were able to access their thoughts throughout the process by reviewing their eye movements. The gut reaction to each new feature was captured in vivid detail, while maintaining a realistic site interaction.
In addition to web-usability studies, this methodology has been used to evaluate print advertisements, ad copy, package designs and television commercials in realistic settings (i.e., in a magazine, on a shelf, within a televised segment, etc.). In all of these areas, this method has proven useful in helping respondents to discuss what catches their eye, what they tend to ignore and how their opinion is formed.
The level of elaboration possible in reports using the Action Review Interview has been far above the standard for regular IDIs or focus groups. Also, the technology seems to possess a good deal of “Wow!” value with both respondents and clients. This is not to mention the vast amount of additional information regarding usability and visibility that is available through in-depth analysis of the eye data itself.
The key feature of this methodology is the capacity to capture details that otherwise would be lost, without compromising the integrity of the interview. From the first instant that people encounter something new, they are building impressions. By allowing them to relive the process by which these impressions are shaped, we can have it both ways — realistic interactions with rich detailed learnings.
BY MIKE BARTELS
EyeTracking, Inc. •San Diego, CA •firstname.lastname@example.org