On-line Surveys: Best Practices and Sky.com example

I'm a BIG, BIG fan of on-line surveys. To make a really great website, you need to find out why people come to your site and if they find what they're looking for. The easiest way to find out what somebody wants is to ask him: say hello to on-line surveys.
If you launch a survey, you'll find that a lot of people are very willing to give their opinion and will give you very valuable and actionable feedback.

To give you a quick list some things you can expect using on-line surveys:

• You can find out what the priorities of your visitors are and take action.
• You can find out what people can do on your site and what they can't and take action.
• You can find out what people like on your site or what they don't like and take action.
• You'll find out how many visitors already purchased on your site and take action.

Let's look at a practical example.

Recently I visited the site of Sky UK, the provider of satellite TV and related telecommunications services. When visiting the section on High Definition, a little window popped up on my screen asking if I want to help with a little survey.

I really like this way of doing surveys: you ask a real-time visitor to help filling out the survey at the end of their visit.

How does this work?

When you accept to do the survey, a new window opens underneath your browser and you're asked to answer the questions at the end of your visit.
This is double-clever: first you invite a visitor in real time while he's busy visiting your site. This is: your visitor is having a real-time and real-life experience with your site. There is no better audience then this one to give you feedback.
And second, you'll ask the opinion of the visitor right after he visits. Not an hour after, not a day after: right on the spot, seconds after visiting your site.

If you accept to participate in the survey, you'll see something like this:

Good Practise 1: Be brief

During the survey, you're asking somebody's opinion which means that you're asking time and effort. So be a good boy or girl, and don't abuse of the good intentions of your visitors: ask what's necessary, be brief. And in case you want to ask a lot: launch various surveys to different people. Better to ask 10 questions in 3 different surveys then 30 questions in 1 survey.

Good Practise 2: Be transparent

A good idea for being transparent is using a bar indicating the progress of the survey. Your visitors will be grateful to know how much time they'll need to complete the survey.

Looking back at second screenshot, you'll see that the first question of the survey was: are you a client or not? This brings us to:

Good Practise 3: Segmentation

You're going to ask a number of different visitors what they think of your site, and you better be sure not all of your visitors are the same. Socio-demographic segmentation is very important, the question on being client or not is something else. You try to found different types of audiences on your site. This is a very important difference for any type of site. For an e-commerce site like Sky.com, the difference between client or not is very relevent: you'll want to give different services and content to these different groups.
For a site like Amazon, you might want to ask if the user already purchased before on the site. For a site like BMW.com, maybe you want to ask if the visitor will be purchasing a new car in the next 3 to 6 months. How to best segment, depends on your business and it might take some brainstorming to see how best to implement this question.

Next question on the Sky Survey was this one:

Good Practise 4: Task Management

This is a good one: why did you come to visit this site? This is the purest form of Task Management: you can consider that every visitors comes to your site with a purpose, and you just ask what this purpose is. The free on-line survey 4Q is based on this idea: it asks why you came to visit the site and if you were able to complete the task.
That's so great: in that way you get to know why people come to visit your site (look at video, find offer, compare with competitor, download form,...) and what tasks are doing good (=visitors gets to download the for he's looking for) or not (=visitor doesn't find prices to compare with your competitor). This is called Task Completion Ratio: what percentage of visitors interested in a specific task get to complete that specific task.

Unfortunately, Sky didn't ask the Task Completion Ratio. It just asks what the main reason is for you to visit your site and moved on to the next question.

Good Practise 5: Ask about site-goals.

This 'Did you purchase'-question is not a bad one: a site like Sky has a clear goal of selling subscriptions or upgrades to, for instance, their High Definition services. So it's pretty straightforward to ask if the visitor purchased that day.
But I think it's not very subtle. First of all, I suppose there are not so many people purchasing Sky HD the exact same day as visiting a site. It's not buying a car, but it's still a bigger buy then, say, a loaf of bread. Second, I suppose that Sky's web analytics or call center will already tell the ratio of purchases. Third, it sounds a bit intrusive: people know that Sky sells stuff, but it feels a bit premature and a bit pushy to ask directly if the visitor purchased.
But above all, I think it shows the interest of the company (to sell) and doesn't take in account that people could be in various stages of a purchase process. The survey doesn't take advantage of braking down the process in different steps and finding out what the satisfaction-rate of each step is. It just takes purchasing as final goal and that's it. If you break down a purcase process you can and defining goals for every stage of the process. These are called micro-goals, and every micro-goal adds to the main goal.

Good Practise 5 revisited: Ask about micro-goals

Next question:

The next question is again a good one. I would have asked for micro-goals (every step in the purchase process), but it's still good to ask why people didn't buy at this stage. This will give you an idea of what you can improve. But: if asking wether you purchased or not might seem a bit pushy, asking why not might even be worse.

Good Practise 6: Be action-orientated

The next question is a typical one, and I must say that I'm not a fan. The question is: what would you change on our site?

In my experience, people don't normally know what needs to be changed on a site and you'll probably end up with some answers which aren't actionable.
For instance: if somebody answers on the survey that he wants more info on HD – how would you change your site? 'More information' is something very general and  you don't really have a clue of where to start. When a lot of people ask for more information, your manager might ask you to publish whatever information you can get your hands on. So in no time, you'll get a site impossible to manage or to navigate, probably full of useless information. It might be a lot more useful to know that a visitor was trying to find out if he could watch the World Cup in High Definition and that he couldn't find the information necesary. Now that's actionable information.

If the last question wasn't actionable, the one following is really company-focused and not at all customer focused. Before you read the question, take a deep breath:

Does the Sky HD section on the site make you think of Sky as a follower or a leader?

What? Can you repeat that question? Or maybe better: don't repeat the question and let me just close this survey-window and I'll continue with my own business. Bye!

Serious. I know companies are interested in knowing what people think of them, and what the 'core brand values' are. But is on quick on-line survey really the place to ask such questions? You can't waste your visitor's time. If you do, they disconnect and leave the survey unanswered.

Best Practice 7: Be customer centered

I'm sure you agree that the language in the question above is very company-centered. And it's not just one question: it's a whole list.
There is also another problem with that last question: When you ask people to answer on a scale from 1 to 7, people tend to put their score at the center, which makes the answers to your question pretty much not relevant. Gerry McGovern explains this in this white paper on surveys, arguing that scaling the answers are bound to give you incorrect feedback.
Another issue with scaled answers is that, on a scale from 1 to 7 some people will give a 7 for very good and a 1 for very bad, and others a 5 for very good and a 3 for very bad, even if they experience the same value of 'very good' or 'very bad'. You could say that some people answer more 'extreme' then others. Luckily there are techniques to overcome this, but make sure you use them.

The following question also uses scaling, but at least has a pretty relevant question: how satisfied are you with your visit and how likely you are to tell a friend about it.

Next up are some socio-demographic questions. These are very relevant for the segmentation talked about above.

Good Practise 8: Be consistent

Last is an invitation to subscribe on a emailing list for further surveys.

This is OK to do, but it's a bit dodgy: a lot of people are willing to give feedback for free, but they don't want you to take advantage of them. So if you promise that a survey will be anonymous, maybe it's better to be consistent with this promise and do not to ask personal data, not even e-mail.
The same remark holds for giving incentives to people for answering a survey: if you promise somebody a gift for helping with a survey, you'll need his address and name to send him his gift. So even if you don't cross this personal data with the answers of the survey, your visitors won't believe you when you say it's an anonymous survey.
In general, it's not necessary to give the participants of the survey a gift. People are very generous and are very happy to give you feedback and help you to improve, just for free. That's a very nice things so keep Best Practice 7 in mind: bo customer centered.

Good practice 9: Be polite
Say thanks at the end of the survey.

Our job in internet marketing: To listen carefully.

Our customer is constantly talking to us on the internet. Our job in Internet Marketing is to know how to listen to him. Are you a good listener?

Let's see if you recognize this: you've got a new product launch or a campaign coming up. Or you have an idea for a new tool or a new section on your web. This is the process of how you could start working this launch or this new tool:

• You start with defining your target. Most companies have decent studies on who they want to sell their products to.
• This target definition outlines what other products this target tends to buy, the media they get their information from, what their family looks like, where they live and how they take their coffee.
• Your product department or the department you want to make a new section for gives you information on the new product or of the requirements of this new section or tool.
• You don't forget to have a look at competitors - you make a benchmark. This competitor sells this and they do it like this and like that. And that competitor already has this new feature on their web, and they do it this and that.
• You define a budget and a timing and a sales forecast.
  
• You take all the above, and make a nice briefing for your agencies.
• Your agencies come up with an idea, a timing and the production agenda.
• There are various discussion back and forth. And of course lots of changes and updates and meetings. This goes on for a while, until you have something that both agency and client consider as a good interpretation of the brief, and there is belief that things will work.
• You ask feedback to your target in a focus group or you perform some tests to see if everything looks OK.
• You launch the new product with a nice big-bang campaign: rich-media, brand-days, all gunz a-blazin'. You put the new feature or tool live.
• You generate some reports: weekly report, monthly report, KPI report. You have a close look at the reports.
• Some changes are suggested for future launches or features.
• You move on to a new launch, or start working on a new briefing for your agency.

Do you recognize this process? Is this how you work?

Let's have a look and see where the customer is in this whole process. When do we actually listen to our customer?

We listen twice to our customer: first when we define target and see who this target is and how he behaves. The second time we listen is when we are ready to launch, and we take a focus group of 7-9 people to see if all the work has been done correctly and if the launch-campaign or the tool or feature will work.

I must say I think it's a bit thin, those 2 times we listen to our customer. The targeting is a pretty difficult and expensive study, so they're done maybe once a year, and if you're unlucky they're done worldwide or on a large scale, so the targeting can differ in your local market. Or you have to use the study of the big-selling country which is next to your smaller country.

The focus groups or the testing are very useful, but they're always performed in artificial circumstance and with a pretty limited audience.

What's missing in this whole process is all the talk we're missing from our customers after we've launched our campaign or our tool. From the moment we've launched a campaign or put a tool live, we'll have hundreds, maybe thousands of visitors visiting your site and using your applications. All these people are talking to you and we really need to start listening to them.

Once we launch a tool, we can see almost instantly see our visitor's digital footprints in our clickstream data: where did our clients click, what page did they abandon, which options did they select most. This is really valuable information, and it's already there, you just have to look at it. This is maybe the most basic way of listening to your customers. It gives you an idea if the t

he tool serves it's purpose, if people can finish to do what you want them to do.

Besides the clickstream, we can launch an on-line survey to ask directly what the clients think. Within a couple of days, you'll have a lot of feedback, not from some target-study performed months ago for a pan-European audience. No, it's feedback from the people who have been using your tool, right there on the spot. Clients tend to be generous with sharing their opinion: we only have to ask and listen to them. With the information we have, we can act immediately.

When it comes to the launches, there are even more ways of listening. Every launch will cause posts in blogs or discussions in forums. With the internet, your client has become pretty ha

ndy with sharing his views, opinions and issues he might have with your new product on-line. You won't have to ask for his opinion, especially when his opinion is not that positive:

he's already expressed them.

Today with the widespread use of social media this is getting very pertinent. Your client is talking, and you want to be the first to know what he's saying. Start listening.

With social media, this gives also an bonus: you can start a dialogue. Clickstream and surveys aren't really fit for answering your clients, but in social media you can start a conversation, and this will give you very valuable insights: how can my product be improved, how do my clients persieve my campaign, is my new tool useful for my clients.

On the internet, your client is talking constantly to you. You only have to learn how to listen, and to listen carefully.

To give you an (real-life) example: this week we had an agency presenting a couple of

designs of some landing pages (excellent agency, btw). They brought 2 different designs and the main difference was that the colours were different. They asked us for an OK for 1 of the designs and which one we preferred: the one with the red button or the one with the black button.

The point is that it can't be decided in a meeting between client and agency which colour would be best for a web design. The only valid opinion on this is our customer's opinion.

The 2 designs looked perfect for a A/B test: you serve to 50% of the audience the red option, and to the other 50% the black option. After a while you'll have enough impressions and feedback to see which of both pages had the best conversion. So by listening to your customer you g

et the best option.