Thursday 2 December 2010

On-line Surveys: Best Practices and Sky.com example

I'm a BIG, BIG fan of on-line surveys. To make a really great website, you need to find out why people come to your site and if they find what they're looking for. The easiest way to find out what somebody wants is to ask him: say hello to on-line surveys.
If you launch a survey, you'll find that a lot of people are very willing to give their opinion and will give you very valuable and actionable feedback.

To give you a quick list some things you can expect using on-line surveys:
  • You can find out what the priorities of your visitors are and take action.
  • You can find out what people can do on your site and what they can't and take action.
  • You can find out what people like on your site or what they don't like and take action.
  • You'll find out how many visitors already purchased on your site and take action.
Let's look at a practical example.

Recently I visited the site of Sky UK, the provider of satellite TV and related telecommunications services. When visiting the section on High Definition, a little window popped up on my screen asking if I want to help with a little survey.



I really like this way of doing surveys: you ask a real-time visitor to help filling out the survey at the end of their visit.

How does this work?

When you accept to do the survey, a new window opens underneath your browser and you're asked to answer the questions at the end of your visit.
This is double-clever: first you invite a visitor in real time while he's busy visiting your site. This is: your visitor is having a real-time and real-life experience with your site. There is no better audience then this one to give you feedback.
And second, you'll ask the opinion of the visitor right after he visits. Not an hour after, not a day after: right on the spot, seconds after visiting your site.

If you accept to participate in the survey, you'll see something like this:


Good Practise 1: Be brief

During the survey, you're asking somebody's opinion which means that you're asking time and effort. So be a good boy or girl, and don't abuse of the good intentions of your visitors: ask what's necessary, be brief. And in case you want to ask a lot: launch various surveys to different people. Better to ask 10 questions in 3 different surveys then 30 questions in 1 survey.

Good Practise 2: Be transparent

A good idea for being transparent is using a bar indicating the progress of the survey. Your visitors will be grateful to know how much time they'll need to complete the survey.

Looking back at second screenshot, you'll see that the first question of the survey was: are you a client or not? This brings us to:

Good Practise 3: Segmentation

You're going to ask a number of different visitors what they think of your site, and you better be sure not all of your visitors are the same. Socio-demographic segmentation is very important, the question on being client or not is something else. You try to found different types of audiences on your site. This is a very important difference for any type of site. For an e-commerce site like Sky.com, the difference between client or not is very relevent: you'll want to give different services and content to these different groups.
For a site like Amazon, you might want to ask if the user already purchased before on the site. For a site like BMW.com, maybe you want to ask if the visitor will be purchasing a new car in the next 3 to 6 months. How to best segment, depends on your business and it might take some brainstorming to see how best to implement this question.

Next question on the Sky Survey was this one:



Good Practise 4: Task Management

This is a good one: why did you come to visit this site? This is the purest form of Task Management: you can consider that every visitors comes to your site with a purpose, and you just ask what this purpose is. The free on-line survey 4Q is based on this idea: it asks why you came to visit the site and if you were able to complete the task.
That's so great: in that way you get to know why people come to visit your site (look at video, find offer, compare with competitor, download form,...) and what tasks are doing good (=visitors gets to download the for he's looking for) or not (=visitor doesn't find prices to compare with your competitor). This is called Task Completion Ratio: what percentage of visitors interested in a specific task get to complete that specific task.

Unfortunately, Sky didn't ask the Task Completion Ratio. It just asks what the main reason is for you to visit your site and moved on to the next question.



Good Practise 5: Ask about site-goals.

This 'Did you purchase'-question is not a bad one: a site like Sky has a clear goal of selling subscriptions or upgrades to, for instance, their High Definition services. So it's pretty straightforward to ask if the visitor purchased that day.
But I think it's not very subtle. First of all, I suppose there are not so many people purchasing Sky HD the exact same day as visiting a site. It's not buying a car, but it's still a bigger buy then, say, a loaf of bread. Second, I suppose that Sky's web analytics or call center will already tell the ratio of purchases. Third, it sounds a bit intrusive: people know that Sky sells stuff, but it feels a bit premature and a bit pushy to ask directly if the visitor purchased.
But above all, I think it shows the interest of the company (to sell) and doesn't take in account that people could be in various stages of a purchase process. The survey doesn't take advantage of braking down the process in different steps and finding out what the satisfaction-rate of each step is. It just takes purchasing as final goal and that's it. If you break down a purcase process you can and defining goals for every stage of the process. These are called micro-goals, and every micro-goal adds to the main goal.

Good Practise 5 revisited: Ask about micro-goals

Next question:



The next question is again a good one. I would have asked for micro-goals (every step in the purchase process), but it's still good to ask why people didn't buy at this stage. This will give you an idea of what you can improve. But: if asking wether you purchased or not might seem a bit pushy, asking why not might even be worse.

Good Practise 6: Be action-orientated

The next question is a typical one, and I must say that I'm not a fan. The question is: what would you change on our site?



In my experience, people don't normally know what needs to be changed on a site and you'll probably end up with some answers which aren't actionable.
For instance: if somebody answers on the survey that he wants more info on HD – how would you change your site? 'More information' is something very general and  you don't really have a clue of where to start. When a lot of people ask for more information, your manager might ask you to publish whatever information you can get your hands on. So in no time, you'll get a site impossible to manage or to navigate, probably full of useless information. It might be a lot more useful to know that a visitor was trying to find out if he could watch the World Cup in High Definition and that he couldn't find the information necesary. Now that's actionable information.

If the last question wasn't actionable, the one following is really company-focused and not at all customer focused. Before you read the question, take a deep breath:

Does the Sky HD section on the site make you think of Sky as a follower or a leader?

What? Can you repeat that question? Or maybe better: don't repeat the question and let me just close this survey-window and I'll continue with my own business. Bye!

Serious. I know companies are interested in knowing what people think of them, and what the 'core brand values' are. But is on quick on-line survey really the place to ask such questions? You can't waste your visitor's time. If you do, they disconnect and leave the survey unanswered.


Best Practice 7: Be customer centered

I'm sure you agree that the language in the question above is very company-centered. And it's not just one question: it's a whole list.
There is also another problem with that last question: When you ask people to answer on a scale from 1 to 7, people tend to put their score at the center, which makes the answers to your question pretty much not relevant. Gerry McGovern explains this in this white paper on surveys, arguing that scaling the answers are bound to give you incorrect feedback.
Another issue with scaled answers is that, on a scale from 1 to 7 some people will give a 7 for very good and a 1 for very bad, and others a 5 for very good and a 3 for very bad, even if they experience the same value of 'very good' or 'very bad'. You could say that some people answer more 'extreme' then others. Luckily there are techniques to overcome this, but make sure you use them.

The following question also uses scaling, but at least has a pretty relevant question: how satisfied are you with your visit and how likely you are to tell a friend about it.


Next up are some socio-demographic questions. These are very relevant for the segmentation talked about above.



Good Practise 8: Be consistent

Last is an invitation to subscribe on a emailing list for further surveys.


This is OK to do, but it's a bit dodgy: a lot of people are willing to give feedback for free, but they don't want you to take advantage of them. So if you promise that a survey will be anonymous, maybe it's better to be consistent with this promise and do not to ask personal data, not even e-mail.
The same remark holds for giving incentives to people for answering a survey: if you promise somebody a gift for helping with a survey, you'll need his address and name to send him his gift. So even if you don't cross this personal data with the answers of the survey, your visitors won't believe you when you say it's an anonymous survey.
In general, it's not necessary to give the participants of the survey a gift. People are very generous and are very happy to give you feedback and help you to improve, just for free. That's a very nice things so keep Best Practice 7 in mind: bo customer centered.

Good practice 9: Be polite
Say thanks at the end of the survey.