What’s wrong with (Click) Heat Maps?

Just take a look at this click heat map:

A heat map example from UseItBetter Analytics

The heat map above shows where people clicked on a login page. You have been to similar pages thousands of times on hundreds of websites. You’ve used such a page yourself. You clicked where those people clicked. Whether you are a UX designer or a CEO, you instantly understand this heat map, don’t you?

What Do Your Eyes Tell You?

Take a long look at the heat map, before you read further. What do you see?

There were lots of clicks in “Sign In Now” (on the left) and “Create Account” (on the right) buttons which is good. Both buttons get the same share of clicks, which is also good as the business retains existing customers but also acquires new ones. There are some clicks in “Forgot the Password?” link. It would be better if everybody remembered their passwords, but obviously, that’s not possible. That’s why the link is there, so people who don’t can click it. So clicks in “Forgot Your Password” are good too. There are practically no clicks in navigation or any other elements which are great because this means that people did what they were supposed to do — they signed in or registered.

Therefore, everything looks FINE on this heat map.

But does it?

Don’t Trust Your Eyes

What if users put wrong passwords when they tried to sign in? They had to correct the password and click the “Sign In Now” button again. What if they did this a couple of times? All those clicks would sum up and make a huge hot spot over the “Sign In Now”.

The same problem applies to the hot spot over the “Forgot the Password?” link. On this page, the link doesn’t even work. If visitors clicked it multiple times in frustration, we get another hot spot.

What could have those visitors done next if the link didn’t work? Probably, they clicked the “Create Account” button. Perhaps, there were not that many new customers as we first thought, and in reality, existing customers who couldn’t sign in decided to register. That wouldn’t be that bad, after all, they would remain customers.

But does the click on the “Create Account” button means that visitors registered? What if they went to the registration form, entered their email and got an error that the email is already registered?

Unless a visitor has another email, this the end for them. They can’t sign in, and they can’t create a new account.

So no, “FINE” is not what this heat map shows us — in fact, it shows mostly behaviours of users who neither signed in nor registered.

Are Heat Maps Any Good?

It’s easy to get fooled by a heat map, but that doesn’t mean heat maps are evil. Heat mapping is as good as any other method of visualising data. Would a pie chart showing the same data be any better? Not really.

The same data visualised in two different ways.

The problem is usually the data, rarely the visualisation. “To see is to believe” doesn’t work well with data. We should always question what we see, whether we look at numbers or hot spots.

How could we get better data for heat maps?

There are five issues we need to account for if we want to read heat maps correctly, or even better, create heat maps that actually have value.

1. 1) A click does not mean a success. A click on “Sign In Now” button does not mean user signed in.
2. 2) A click doesn’t mean a failure, either. If a user clicks something that is not clickable, she usually corrects her mistake a moment later.
3. 3) An individual visitor can click the same element multiple times. Clicking “Sign In Now” button three times is very different to clicking “add to basket” button three times. The former signals problems and the latter may just as well mean more profit. Both look the same on a heat map.
4. 4) An individual visitor can click multiple elements, even if a purpose of those elements is contradictory, for example, “Sign In” v “Create Account” buttons.
5. 5) Order of clicks matter. If users click the password form field after they submitted the Sign In form, it‘s likely they tried to correct the entered password.

You can’t address all* those issues in one heat map. What you need are multiple heat maps of the same page showing different data.

* There are, of course, many other issues with heat maps caused by interactive elements, responsive design rules, browser/devices incompatibilities, or differences in page content (the basket page with one item in a basket, five items, or no items at all will look very different). All this can lead to incorrectly aligned hot spots and misinterpretation of data.

Doing Heat Maps, the Better Way

The examples included below are specific to UseItBetter, but the ideas are not. In fact, the ideas are not even limited to heat mapping. If you are doing heat maps in-house or using a “sophisticated” heat mapping tool, perhaps you will be able to apply some of them. If you are using a tool that gives you no segmentation options, I hope what you have read so far will make you look for something better.

Segment Heat Maps by Outcome

This is probably the most important tip — always generate separate heat maps for visits in which users failed and succeed to complete a particular process or part of it.

Successful visits are those visits in which users went from account-login.html page to account.html.

For the flow pictured above, we would need two heat maps:

1. a heat map of visits in which users successfully signed in or registered,
2. a heat map of visits in which users failed to sign in or register.

In UseItBetter, you can generate a heat map for any segment you specify. Start with a segment of visits in which visitors went to the login page and then add two sub-segments: the first one with a page view of the account.html page, the second one without it.

The selected segment contains visits in which users did NOT go to account.html page but went to account-login.html because segments inherit criteria of their parents.

It would be tempting to save some time and look only at a heat map of visits in which users failed, but comparing heat maps of both outcomes can reveal critical insights. Just take a look.

Two heat maps presenting behaviour with two different outcomes

On the left, are clicks of visitors who successfully reached the account.html page. On the right, clicks of visitors who failed to do that.

The heat map of visitors who failed shows that the visitors failed regardless whether they tried to sign in or register. But can you see what’s wrong with heat map on the left? Nobody who succeeded used the Sign In form. Nobody! Apparently, the form doesn’t work at all.

Segment Heat Maps Based on Key Behaviours

Users who land on the login page can either sign in or register. Both processes are, in theory, mutually exclusive. In practice, they may overlap, and if you wanted to detect that, you would need to generate heat maps separately for visits in which users started each of those processes. UseItBetter allows you to segment data based on autotracked interactions with a website. In this case, we could generate heat maps for visits, in which visitors failed to sign in or register and clicked “Sign In Now”, “Create Account”, or “Forgot the Password” link.

Two heat maps presenting behaviour with two different outcomes

Any interactions with the Sign In form on a heat map based on the segment of visits in which all visitors clicked the “Create Account” button, would mean that users either went to the registration page and then came back to sign in, or they tried to sign in and then went to the registration.

Segment Heat Maps Based on Order of Events

This is a different type of segmentation. We called this feature “trimming” or “sequencing” because it allows you to include in your analysis only data that happened before or after a certain event. Or, in-between two events.

This heat map will include only behaviours that happened in the account-login.html page but only before the first click in the “Sign In Now” button.

Thanks to trimming, we can look what visitors were doing on the login page before they clicked the “Sign In Now” button:

There are way more clicks on the email than on the password field — do visitors switch fields using a tab or is the password field autocompleted?

There are no clicks on “Create Account” button, just — as expected — clicks on the form fields.

If we would look what visitors did after they submitted the login form, we would see which form fields they corrected and also what they did when they gave up on submitting the form — they clicked the “Forgot the Password?” and/or the “Create Account” button:

Unsurprisingly, visitors corrected passwords way more often than email.

Of course I could generate another heat map to see what user clicked after they clicked the “Forgot the Password?” link and so on.

I could even check if they tried to sign in again after they went to registration and got the “existing account error”.

So, There’s Nothing Wrong with Heat Maps?

Heat Maps are fine, as long as you can control what’s displayed on them. But then again, that’s true for any visualisation method.

As for myself, I love using Heat Maps during the exploratory phase. They show me things that I didn’t expect or asked for explicitly. Inspire me what questions should I ask. But once I create all the segments and verify what’s true and what’s false on a heat map, I move on. I throw the heat map to a trash and continue my analysis using other, qualitative and quantitative techniques, looking for other clues and some business metrics that would tell me how critical my findings are.