Eye tracking banners (Realeyes study)

by Realeyes: Eye tracking banners

Eye-tracking banners

View more OpenOffice presentations from Mihkel Jäätma.

Using eye tracking to enhance sales and usability

by simonday.com

Eye tracking is a tool which tracks your eye movements as you navigate through pages. It has been around for a number of years and the usefulness of this knowledge is incredible. Let me give you a good example.

In this case look how many people looked at the big red “SALE” banner in the top images compared to the green banner below. Not a single person looked at the red sale banner! That alone is interesting but now look at how this one simple banner change affected the top navigation – it’s completly altered the way people navigate the site!

Let me say that last bit again: By changing a BANNER Virgin radically altered user NAVIGATION. I bet no-one saw that coming, I know I didn’t when I first started using this knowledge. This is why testing is so important. You’re sat there tinkering with one part of the page when the change actually alters the way the user navigates your site. By understanding how eye-tracking works and looking at results you can start to understand how a visitor will journey through your site.

Combine this with the other tools and you can actually CONTROL the journey for the majority of people coming to the site.

Don’t believe me? Let’s try this:
I want you think your James Bond and you want to buy some lock picking tools from Devon Locks (who said this wouldn’t be cool!). Visit http://www.devonlocks.com/index2.html right now! Don’t look any further because if you do you will skew your results. Go now!

Ok, if you’re in the 86% of visitors I’ve targeted you pretty much did exactly this:
You skipped the header and yours eyes first saw the logo of the 2 men. You completely ignored all the header and focused straight on that 2 men logo. You still ignored the header and next looked at the big text “I need tools for”. You then looked, read and digested all 4 graphic links because you’re here to buy so that was always going to be where I focused your eyes.

If you were still in your “looking for tools” mode you probably read the H1 header text just below the 4 images but only 65% of you did. Again you probably read the sentence just below the header but you would have speed read it if you did. Now the cool part. Everywhere on the page you probably speed read EXCEPT for the first 3 bullets which you read in FULL. You may have read all 5 in full but in all honesty it was a little too much text and the top 3 are the only ones I really want people to see.

At that point you probably saw the YouTube logo but came back here. Everything below the YouTube logo is fluff to balance the page.

So hopefully you’re suitably impressed and want to know how I did it.
Headers – The only time people look at headers is when they want to know they are in the right place (logo), or they want to contact the site and are looking for a phone number of email address. Although I have text up there it is mainly for search engines. Virtually no-one looks at the header anymore for any length of time because all sites offer the same information so people associate headers with contacting and logos.

So because most of us ignore the header we start to read the page as we would a book – Top left to right. This is often called the “F” shape pattern. If you look at Devon Locks again you should be able to spot the “F” shape pattern in the design.

Content – Your eyes were first drawn to the gold 2 men because it’s pretty big, it’s unusual and because it is human shaped which we tend to respond more to images if they take on a human shape. The reason it’s there is to do exactly that; get your attention because I want your eyes exactly at the point to start your “F pattern” viewing. Immediately to the right of the men is some nice big text “I need tools for”. So now I’ve controlled where you start viewing and I’ve offered an instant solution to the question “where do I go to buy them”. I have nothing left on the page to confuse the visitor (like right hand content) so the only place to go now is down…exactly at the point I want you! The images in the four boxes are so sub-consciously whatever tool you came here for I have the 3 options straight in front of you plus a spare for any help questions you might have.

In case they need a little more help I have those bullets below. Why did you read the bullets but skimmed most other bits? Because you were always going to be 126% more likely to read the bullets because they are short, to the point and sub-consciously we don’t tend to skim bullets… 126% less likely. Even if you do read those bullets you can’t escape those big buttons ready to take you to the next level of the journey.

On this site I have a bounce rate of 14% on the home page and it goes down to 9% on other level 1’s. Now 14% is pretty much as low as I can go but I would be very surprised if many e-commerce sites could manage that kind of bounce rate. That’s what happens when the designer gets FULL control over the design!

So how do you utlise this tool? I’m afraid the answer is; not easily. Unless you work for a large company there is only one way. It isn’t as good as the real thing but it still helps: get as much online research as you possibly can. Get as many images of these results as possible.

As a poor mans eye tracking you could also get friends and family to visit sites whilst you stand beside them and let them point out what they are looking at and why. It isn’t fool-proof but it does help build a mental map in your mind of why users do certain things but more importantly it’s what they’re missing! Over time you can train your mind to view as the user would.

Eye tracking emails: Realeyes usability test

Eye tracking emails - Realeyes usability test

Eye-tracking emails

View more OpenOffice presentations from Mihkel Jäätma.

Eye tracking history by Wikipedia

Eye tracking history by Wikipedia

In the 1800s, studies of eye movements were made using direct observations.

In 1879 in Paris, Louis Émile Javal observed that reading does not involve a smooth sweeping of the eyes along the text, as previously assumed, but a series of short stops (called fixations) and quick saccades.[1] This observation raised important questions about reading, which were explored during the 1900s: On which words do the eyes stop? For how long? When does it regress back to already seen words?

An example of fixations and saccades over text. This is the typical pattern of eye movements during reading. The eyes never move smoothly over still text.

Edmund Huey[2] built an early eye tracker, using a sort of contact lens with a hole for the pupil. The lens was connected to an aluminum pointer that moved in response to the movements of the eye. Huey studied and quantified regressions (only a small proportion of saccades are regressions), and show that some words in a sentence are not fixated.

The first non-intrusive eye trackers were built by Guy Thomas Buswell in Chicago, using beams of light that were reflected on the eye and then recording them on film. Buswell made systematic studies into reading[3] and picture viewing[4].

In the 1950s, Alfred L. Yarbus[5] did important eye tracking research and his 1967 book is one of the most quoted eye tracking publications ever. For example he showed the task given to a subject has a very large influence on the subject's eye movements. He also wrote about the relation between fixations and interest:
"All the records (…) show conclusively that the character of the eye movements is either completely independent of or only very slightly dependent on the material of the picture and how it was made, provided that it is flat or nearly flat." [6] The cyclical pattern in the examination of pictures "is dependent not only on what is shown on the picture, but also on the problem facing the observer and the information that he hopes to gain from the picture." [7]

This study by Yarbus (1967) is often referred to as evidence on how the task given to a person influences his or her eye movements.
"Records of eye movements show that the observer's attention is usually held only by certain elements of the picture. (…) Eye movements reflect the human thought processes; so the observer's thought may be followed to some extent from records of eye movements (the thought accompanying the examination of the particular object). It is easy to determine from these records which elements attract the observer's eye (and, consequently, his thought), in what order, and how often." [8]

"The observer's attention is frequently drawn to elements which do not give important information but which, in his opinion, may do so. Often an observer will focus his attention on elements that are unusual in the particular circumstances, unfamiliar, incomprehensible, and so on." [9]

"(…) when changing its points of fixation, the observer's eye repeatedly returns to the same elements of the picture. Additional time spent on perception is not used to examine the secondary elements, but to reexamine the most important elements." [10]

This study by Hunziker (1970)[11]on eye tracking in problem solving used simple 8 mm film to track eye movements by filming the subject through a glass plate on which the visual problem was displayed. To view a slow motion movie of the eye tracking in problem solving click: http://www.learning-systems.ch/multimedia/eye%20movements%20problem%20solving.swf for details of the study: http://www.learning-systems.ch/multimedia/forsch1e.htm

In the 1970s, eye tracking research expanded rapidly, particularly reading research. A good overview of the research in this period is given by Rayner.[12].

In 1980, Just and Carpenter [13] formulated the influential Strong eye-mind Hypothesis, the hypothesis that "there is no appreciable lag between what is fixated and what is processed". If this hypothesis is correct, then when a subject looks at a word or object, he or she also thinks about (process cognitively), and for exactly as long as the recorded fixation. The hypothesis is too often today taken for granted by beginning eye tracker researchers.

During the 1980s, the eye-mind hypothesis was often questioned in light of covert attention,[14] [15] the attention to something that one is not looking at, which people often do. If covert attention is common during eye tracking recordings, the resulting scan path and fixation patterns would often show not where our attention has been, but only where the eye has been looking, and so eye tracking would not indicate cognitive processing.
According to Hoffman, [16] current consensus is that visual attention is always slightly (100 to 250 ms) ahead of the eye. But as soon as attention moves to a new position, the eyes will want to follow.[17]

We still cannot infer specific cognitive processes directly from a fixation on a particular object in a scene.[18] For instance, a fixation on a face in a picture may indicate recognition, liking, dislike, puzzlement etc. Therefore eye tracking is often coupled with other methodologies, such as introspective verbal protocols.

Eye tracking studies: Realeyes eye tracking videos

Eye tracking studies: Realeyes eye tracking videos

Eye tracking video of IKEA website



Eye tracking video on User on Samsung website



Eye Tracking Video on User on the Sony Ericsson website



Eye Tracking video of a user on the American Express Website

Eye tracking study: Twitter eye tracked by an expert and a beginner

Eye tracking study: Twitter eye tracked by an expert and a beginner

Expert



Beginner

Interesting: Baby Eye Tracking :)

Baby Eye Tracking Gaze Replay at ECVP 2007

Interesting: EEG brain map with Eye tracking

EEG Brain map, GSR and Eye Tracking during Gaming

Eye tracking conference at EyetrackUX 2009 - Realeyes

Eye tracking conference at EyetrackUX 2009
Realeyes

Eye tracking - Soldat

Eye tracking study: Soldat

Eye tracking analysis: WordCraft III, House of the Dead

Eye tracking analysis: House of Death

Eye tracking analysis: WordCraft III

Eye tracking Realeyes - Why do you need 50 people in eye tracking studies?

Why you need 50 people in eye tracking studies

View more OpenOffice presentations from James Breeze.

Testing Web Sites with Eye-Tracking

By Will Schroeder

Thanks to some recent usability studies we conducted using an eye-tracking system, we now have real evidence of where users actually look when they view a web page. It’s clear that users quickly learn to look where they expect to find content. They also quickly learn to avoid areas where they don’t see—or expect—what they’re looking for, including banner ads and parts of the page outside the central area.

Where Did Users Look?

Our client wanted to know how much attention users would pay to several areas of a prototype web page — and thus to the content of each area (see figure). With the eye tracker, we monitored how much time users looked at each area.

To determine where users looked on a prototype page, we tracked their eye movements within these areas. Users typically looked first in the center, then to the left, then to the right. When they encountered ads in the Center Area, they quickly learned to stop at the borders—and then look away.

The figure shows the design grid for the client’s index and navigation pages. We asked users to look for specific information on the site. When deciding which link to click, users typically looked first in the center area, then in the left panel, then in the right column. Users spent an average of 11 seconds on each of the pages we tested.

Our users were more likely to investigate areas outside the Center Area when they spent more time searching for the correct link, or when they visited the page for the second or third time in a task.

Scan Patterns Were Similar

All the users spent the same proportion of time looking at each area. This similarity of behavior surprised us. We think it means that they all applied similar criteria in deciding on the relative importance of the tested areas—the likelihood each area would contain the link or information they wanted.

New and experienced web users scanned essentially the same way. At first, the new user scanned pages from left to right, as if reading a book. But he quickly changed to the center–left–right sequence.

He needed only two or three page visits to learn where to find the “good stuff”—and how to avoid less-interesting (to him) material. He looked at the browser controls more than the experienced users did, but was otherwise unremarkable. We had only a single new-to-the-web user, but this is an interesting pattern to watch for.
Because these users so quickly adopted the center-left-right strategy, we believe the experienced users—who used it immediately—had learned the behavior through their earlier visits to other web sites.

This suggests that designers may not need to design pages differently for new web users, but we’d need more tests to confirm this.

Users Don’t Go to the Bottom
Users rarely looked at what we called the Study Area just above the browser’s status line. Users often found what they wanted before getting to this area. Interestingly, if they wanted to see information from this area, they scrolled to bring it higher on the screen rather than looking at the bottom.

Some users failed to find content that began within this area; they apparently assumed that anything important would begin in the center area. For these users, the “fold” was 2/3 of the way down the first screen, not at the bottom. Users may not look where they don’t expect useful material.

New Scans for Some Changes

We tested several variations of the site’s prototype page layout to see if they’d alter user behavior. When the designers changed the proportions and content of the three main areas only slightly, users kept the same scan pattern. Horizontal and vertical grid changes of 6–to–12 pixels (1/8 inch at the resolution we tested) had no apparent effect on users.
On the other hand, all users immediately detected a change that narrowed the left column about 30 pixels (1/3 inch) and used a heavier font. Most users scanned this changed area as soon as it appeared and read the content.

Users apparently will reevaluate their scan strategy when they detect a design change of this magnitude.

This argues against the design strategy of using a consistent grid on all pages—it may cause users to miss content of interest.
Users seem to notice changes somewhere between 12 and 30 pixels, but we didn’t test for this, so we don’t know how big the change must be before users notice.

What Attracted Users
Using direct observations as well as videotapes of user behavior, we found that:
Bright colors and animation attract users’ eyes. This is not a new finding, but it is interesting to actually see it happen. The gaze cursor flicks to the animation, dwells for an instant, and then returns to content.
Eye gaze often stopped at the borders around ads (the bottom of ads at the top of the page and the top of those near the bottom). To users, these lines apparently indicate content that is unimportant to their task.

The Role of Peripheral Vision

The eye tracker tells us where the users direct their central vision, the part of the visual field that can discriminate fine detail. But peripheral vision clearly plays a role. We can’t measure how much, but we must account for peripheral vision in interpreting what users see. For example:
Users rarely looked directly at the scrollbar; their gaze peaked considerably to the left of it. This shows that they don’t look at the scrollbar—even when using it. Peripheral vision obviously helped.

Ads apparently attracted users only when they related to the current task—even if the content interested users (but was irrelevant to the task). For instance, a lover of old automobiles was not attracted to an ad featuring a picture of an antique car. These observations are further evidence that peripheral vision plays a role: users somehow determined if an ad interested them before they looked directly at it.

(originally published at 1998)

Eyetracking research and forms design

Formulate: Eye tracking research and forms design

In recent years, a number of form design related eye tracking studies seem to have captured popular attention amongst the web community. These include a study by Matteo Penzo on label placement and another more recent study.

Not only have these studies been widely circulated on the Internet but, in the case of Penzo's eye tracking study, they have formed the basis of some parts of Luke Wroblewski's popular book “Web Form Design: Filling in the Blanks”.

We think it's great that the unique design challenge forms represent is getting more attention from the web and user experience community. However, we are a little unsettled about the increasing use of these articles as the basis for best practice. Our concern stems from what we see as fairly major flaws in the methodology that these and similar eye tracking research studies contain.

The two methodological problems lie behind much past forms-related eye tracking research
In many cases, eye tracking studies that have examined different options for the design of forms have suffered from two main shortcomings: insufficient recognition that seeing is not equivalent to attention; and drawing inferences from an inadequate sample. Seeing does not equate to attention. It is one thing to know that someone has directed their gaze in a particular place. It is another thing entirely to know what they were attending to—or thinking—at that time.
Zimmerman estimates that at any one time, the eyes take in 10,000,000 bits per second of information, yet we pay conscious attention to only 40 bits per second¹. That's 40 bits out of 10 million, or attention going to only 0.0004% of what we see.

If you're not convinced about this phenomenon, try to remember what colour shirt the person you share an office with was wearing yesterday, or even the colour of their eyes. You probably look at both things many times in a working day, but you don't necessarily attend to them.
The implications for eye tracking research is that such studies give us only part of the picture of what's going on when someone interacts with a form. In order to truly make informed conclusions, we need to supplement this picture with information from other sources. This might include error and task analysis of the completed forms and/or probing the participant, using protocols such as concurrent “think aloud” or retrospective discussion.

An adequate sample is a prerequisite for drawing inferences Our second and equally significant concern relates to the design of the samples used to conduct these many eyetracking studies.
As an example, the cxPartners' study involved only 8 participants: 6 female and 2 male, all of which were in their 20s or 30s and reasonably web savvy. Without considering any other aspects of the study's design, this is enough to make a statistician break out into a cold sweat.
The statistician's reaction is because the sample used by cxPartners is highly likely to have been skewed. By skewed we mean that the sample probably doesn't accurately reflect the greater web-form-filling population. At the very least, it would have been preferable to have included both younger and older participants, not to mention more males.

Furthermore, the sample size—8 people—is so small that it is likely to be highly influenced by the nature of the particular 8 participants that were involved. Pick a different 8 people and there is a good chance that the findings from the research would be very different.

This is why 30 is the recommended minimum sample size for any study from which inferences for a general population are to be drawn². While there's a lot more to designing a good sample than having a minimum of 30 participants, this will at least get you into the space where you might be able to calculate statistical significance.

Statistical significance is about knowing which differences are likely to be due to just the particular sample that was selected as opposed to reflecting a true difference in the underlying population. With a small sample size, we cannot calculate statistical significance and thus have no real indication of the reliability of our findings. (For more on statistical significance and user research, see Caroline Jarrett's recent article on Usability News titled "Statistically significant usability testing".)

Being transparent about sample design is important One thing cxPartners did well in their article is describe the sample that formed the basis of their research. Providing this information empowers the reader to make their own judgement about how to use the findings presented therein.

Conversely, Matteo Penzo's article doesn't give many specifics about the design of his sample. He says that the sample included both expert users—primarily designers and programmers, but also some usability experts—and novice users. But we are not given any more detail nor told how many participants there were. One hopes, given the immense popularity of his article, that Penzo's sample was both representative and large.
Better not to report at all?

To be fair to the team at cxPartners, their eyetracking forms article did begin with note about the potentially invalidity of the study. Isn't it enough that readers were duly warned? Unfortunately, we think not.

It is our impression that web designers and developers are hungry for guidelines based on research. This hunger is a great thing: it means we all want to know more and create the best sites we can. However, it also means that readers are likely to latch on to the findings of a study and pay little regard to the caveats regarding methodology that are placed around it. This is just human nature. We can work with a guideline; we need a guideline. The perhaps-flimsy basis behind the guideline is just all too often seen as the spoil-sport at the party and pushed to one side.

So what should researchers do with findings based on an inadequate sample? Perhaps controversially, we suggest that rather than report findings with caveats around them, it may be better to not report such findings at all. That way widespread inappropriate use can be prevented.

This is a hard position for many people to accept. Surely it is better to have some findings than nofindings?

The problem is that the “some” findings may be pointing in completely the wrong direction. If we have no data, there's nothing to suggest one course of action is better than another. But if we have bad data, it can lead us astray, all the while with a false sense of confidence in our decision because, after all, it is based on research findings.
We raise these issues to help progress the field
We did not write this article to embarrass or shame anyone, nor to discourage people from doing forms research. We know from direct experience how unbelievably hard it is to design a sound research study.

Moreover, we think both Matteo Penzo and cxPartners should be congratulated for actually taking the (not insignificant) time and effort to actually do some research and share their findings with the community. A lot of people make demands of such individuals—“Why didn't you do X?”, “What would have happened if you had tested Y?” etc—but very few people actually take up the gauntlet and run such studies themselves.

Having said that, what we would like to see in the future is for the web community to have a higher awareness of what makes for quality research, and approach published studies with a more critical eye. Formulate has and will be on the receiving end of such critique—see, for example, the comments to our recent research article on A List Apart—but as long as it is informed and considered, we believe this can only help to advance the field.

In the end we hope the web industry will recognise the importance of the sort of rigour that has been commonplace for decades in other fields such as psychology and social research. Not only will this lead to better design decisions, but we believe it will help the industry mature, in turn generating respect for the web as a serious vehicle for communication, transaction and information.
References

1 Zimmerman, M. (1989) "The Nervous System in the Context of Information Theory". In Zimmerman, M. Schmidt, R. F. & Thews, G. (eds) Human physiology pp. 166-173.
2 This minimum of 30 can be found in almost any statistics or sampling textbook, e.g. Howell, D.C. (1982) Statistical Methods for Psychology p. 149. The number comes from the fact that given a large population, the greater the sample size, the closer the distribution of means from samples of that size comes to approximating the normal distribution. This in turn makes various sample estimates—including statistical significance—valid (provided some other conditions also hold, but we won't go into that here!).

Results of Eye Tracking Study: Google Versus Bing

Bill Hartzer: User Centric Releases Results of Eye Tracking Study: Google Versus Bing

User Centric has released the results of an eye tracking study that compares data between Google and the new Microsoft search engine Bing. What is interesting to note is that sponsored links are attracting more attention than they are on Google. Google users appear to be more aware of the sponsored links.

User Centric used eye tracking technology to capture 21 participants’ eye movements as they completed two informational and two transactional search tasks, each in Google.com and Bing.com.

The two search phrases that were used during this eye tracking study:Learn about eating healthyBook a last minute vacation

User Centric used eye tracking technology to capture 21 participants’ eye movements as they completed two informational (e.g., “Learn about eating healthy”) and two transactional (e.g., “Book a last minute vacation”) search tasks in each engine.

According to User Centric, “Preliminary findings revealed comparable amount of visual attention on organic search results and top sponsored links across both search engines. Sponsored links on the right, however, attracted more attention on Bing than they did on Google. On average, across all four tasks, 42% of participants looked at Bing’s sponsored links on the right; by contrast, only 25% of participants looked at Google’s right rail links.”

When it came to the amount of attention paid to the organic search results, Bing and Google did not differ: users spent an average of seven seconds looking at the organic search results. They were both about the same amount of time spent.

During a search, 90 percent of users tested looked at the “sponsored links”. User Centric also reports that during “transactional searches” (searches that involved someone buying something or searches related to completing a transaction), “participants would spend more time looking at the sponsored results on top (~2.5 seconds) than they did on informational searches (~1.5 seconds).” But, on the Bing.com search engine, participants of the study spent more time looking at the paid links (sponsored links). About 42 percent paid more attention to the sponsored results on Bing. About 25 percent on Google.

Eye-tracking 2.0: it's about users, not science

Mihkel Jäätma is co-founder of eye-tracking company Realeyes

Eye-tracking has been used in web design for many years. However, the widespread preconception is that it takes PhD skilled technicians - plus long consulting hours - to make any sense or use of people’s eye gaze data.

The value from eye-tracking has been directly related to consultancy skills, but shouldn’t it be more about real users?

Eye-tracking has been used mainly as a qualitative tool because of technical reasons. The hardware was difficult to operate and only few consulting houses had the access and capability to run the tests in their lab conditions. Operating labs and recruiting people from consumer panels is expensive and forced consultancies to stick with small sample sizes to fit into client’s budgets. Small sample sizes have held back the wider acceptance of eye-tracking analysis in web design. In 'Eye-tracking 2.0' devices will be taken to users, not users to devices. This fundamentally changes the speed and sample size of users who can be eye-tracked for analysis purposes.

Quantitative eye-tracking speaks the voice of the customer, not of the consultant. It takes to test about 50 people (why 50? about eye tracking) to achieve reliability in the visual analysis for any piece of media. Statistical significance of this sample size allows conveying real user design preferences in neutral and objective manner. Visual metrics and animations of user interaction gain their own stand-alone value as models of real user behavior on designers and marketers desktops. Consultant expertise and qualitative insights are extremely valuable, quantitative eye-tracking complements them with sound numbers. Technological progress has made data collection procedures and packaging 'a standard issue'. Professionals can now spend their time on more demanding and value-adding activities than running user tests or analyzing gigabytes of eye-tracking data.

Reliable quantitative visual analysis is now available as a given tool and consultants can focus more on fundamentals. Much like fund managers do when using outsourced data feeds, but making their own investment decisions.

Eye-tracking was often oversold in the past, creating well-deserved skepticism towards the technology. The 'before-and-after' case studies of websites being redesigned based on only heatmaps of 10 people rightly upset many industry professionals.

As eye-tracking hardware improves and operational models for analysis develop there will be less ‘magic’ and more of the real stuff: identifying user preferences and employing that knowledge to achieve better web design.