How NOT to Execute a Website User Survey

As I'm writing this, it's been less than 48 hours since the election of Donald Trump as the next President of the United States. Given the widespread response of shock and surprise, it's clear this wasn't the expected result. Polls had consistently indicated a different outcome, which many people and media outlets took at face value.

While my thoughts on the election could probably fill a manuscript (or maybe a trilogy), that's not my area of expertise and, let's be honest, you didn't come here to read my political musings. However, I do believe there's a valuable lesson to be learned from this, which is directly applicable to the world of marketing and web presence management.

The Danger of Taking Survey Responses at Face Value

How many times have you been asked a question where you thought one thing and said another? For example, your significant other asks, "Where do you want to eat?" Your response: "I don't care." But, inside, you're thinking "Burger, I want a burger, please say you want a burger..."

See the problem?

This type of behavior extends beyond just restaurant selection. A business might ask its employees whether they would prefer more time at home with their family, or more money. Often, people will give what they feel is the "right" response, saying they want more time with their family, even if the latter is true. It's up to the person analyzing the results to determine true intention.

To some degree, this is where political pollsters failed in this election.

The Trouble with the Website User Survey

We've all been there: It's time to redesign our website. How can we be sure we're giving our audience what they want? Easy. We'll ask them, by including a user survey on the site. Seems simple enough, right?

I think you know where I'm going with this.

If you want good data, it's not only important to interpret the true meaning of responses, but also to ensure that you're asking the right questions, in the right ways.

The 7 Deadly Survey Sins

The following are common pitfalls that plague many organizations and businesses who survey their customers.

1. Leading Language

Wording a question in a way that pushes respondents towards a certain answer.

Example Question: Having a four-wheel drive vehicle can provide better safety for your family in adverse weather conditions. Would you prefer a two- or four-wheel drive car?

2. Limiting Responses

Providing the respondent with a set of responses that doesn't fully align with their intrinsic response.

Example Question: What do you think of bananas? Choose: (A) They're my all-time favorite food, or (B) The thought of a banana makes me sick.

3. Double-Barreled Questions

Asking questions that touch on multiple issues, but only allow a single response.

Example Question: We recently redesigned our app and also redesigned our website. Are you happy with the changes? Check YES or NO.

4. Scratching the Surface

Taking answers at face value, assuming no mistakes, lies or hidden intent.

Example: Doing a survey without considering the other things mentioned in this list.

5. Ignoring Non-Respondents

Failing to account for non-respondents can lead to a skewed interpretation of the market, as data has only been collected by a vocal portion of the population.

Example: Running a survey about what percentage of the population owns a dog, by asking people who are walking into PetSmart.

6. Stakeholder Bias

Asking respondents who have an incentive to drive a certain response.

Example: Your team gets a bonus based on the number of new clients they help bring into the company. You poll your employees to ask if website leads should count towards bonus.

7. Compensated Surveys

Providing other forms incentive to respond to a survey increases the likelihood of false responses.

Example: You send out an offer for an Amazon gift card to anyone who responds to your survey. Hundreds select the first option for every answer as they don't care about the response, just the gift card. The responses of those who do care are subsequently outweighed. 

Use Your Survey Right

A website survey can be a fantastic way to garner insight and opinion you may never have otherwise considered, but if you don't execute correctly, or rely too heavily on your results, it can do more harm than good.

In the digital marketing industry, we have access to a plethora of inexpensive and readily-accessible tools, such as analytics and behavior tracking utilities, which allow us to review data that users reveal through their actions. When you combine this with survey responses, you end up with extremely insightful data that can be used to make business decisions.

Bad: Guessing what your audience wants.

Better: Asking them with a survey.

Best: Combining survey results with analytics data, to form a complete and accurate picture.

Feeling lost?

If you're considering a website survey and you need a partner to execute, collect and review the results, look no further.