Is Surveying Obsolete?

Harvard and Wharton seem to think so, as do a growing number of leading consumer brands.

Harvard Business Review recently published a ‘Management Tip’ entitled ‘Don’t Listen to Customers – Observe Them.’ In the ‘Tip’ they state:

“There’s a fundamental problem with asking people what will persuade them to change: Most of the time they won’t know the answer. It’s not that they won’t give one. They’ll give you plenty. But those answers are likely wrong.”

As Jerry Wind, The Lauder Professor, Marketing, from The Wharton School of The University of Pennsylvania explains, “The bias of surveys causes a tremendous challenges for businesses and often points to the wrong answer. To get to the right answer – the truth – you need observation. But having the ability to observe millions at once changes the game. You can gain understanding on a wide and deep level like never before.”

One of largest and fastest growing business waves is brands observing their customers’ natural, unsolicited discussions, activities and opinions instead of traditional, manufactured responses by asking them what they would intend to do via traditional surveys. In fact, some of the largest consumer brands today are relying on the ‘big data’ of open social media to observe millions upon millions of consumers “all at once,” free from distraction or bias.

Recommended for YouWebcast: Sales and Marketing Alignment: 7 Steps To Implement Effective Sales Enablement

Questionable Questions

For decades, businesses have relied on surveys to generate understanding of consumer behaviors and attitudes, however, given the manufactured nature of the information and the inherent bias related to question structure, survey approach and selected respondents, among other factors, many brands are leery when it comes to trusting the resulting information. As Steve Martin of Harvard Business Review wrote earlier this year in his piece entitled ‘Stop Listening to Your Customers’:

The great majority of the decisions we make in our information-overloaded, distraction-heavy lives are made outside our conscious awareness, driven more by contexts than cognitions. As a result, asking someone to pinpoint what will influence them in the future is a bit like saying, “tell me how you will behave in the future when you are not thinking about what I have just asked you about?”

A Social Shift

When it comes to measuring consumer reactions, many large consumer brands are discovering the value of the immediate insight social business intelligence provides. Traditionally, consumer brands have relied on shopper surveys which can take upwards of six months to receive results on.  As one senior market research with a leading food company explained, “You’ve missed the market and any opportunity to adapt waiting (upwards of) 26 weeks for consumer reactions.”

With social business intelligence, genuine, unbiased responses from shoppers, consumers and influencers can be delivered in a fraction of that time, often a mere few days. Brands are able to take action on this deep intelligence related to the likes, dislikes, attitudes, behaviors, decisions and actions of these individuals.

From here, robust, multidimensional views of shoppers, consumers and influencers can be constructed related to personas, decision drivers, demand moments, path-to-purchase, and a wide array of other actionable insights.

A Deep Impact

Social Business Intelligence is the solution more and more brands are relying on to gain genuine consumer insight, free from the bias found in manufactured surveys. This basically delivers ‘market truth’ to companies based on digital ethnography.

Brands turning to unsolicited social business intelligence as a foundation of their understanding of shoppers, consumers, influencers and competitors are gaining actionable insight into their own businesses, but also understanding the competition like never before.

So, what say you, are traditional surveys a valuable business tool today?

  Discuss This Article

Comments: 13

  • Most of our surveys are very short and very general. We use the NPS in our surveys, but have found that low responses can skew that score. Finding other options can be a difficult transition. What are your feelings on NPS? How does a company implement something new?

    • Hi Andy,

      I actually introduced a version of NPS at one of Citi’s corporate payment divisions back in 2005 and managed that process for Fortune 500 clients through 2012. It can be a powerful program to drive client insights and innovation. The NPS approach is much different in my opinion than consumer surveys given the scope and level of depth you can get with a client due to the relationship. Social business intelligence would be a supplement to innovation assuming your product serves a consumer market, but it would be unlikely it would be a replacement to NPS. A client’s ability to skew the aggregate score is certainly a challenge, which is why we incorporated a qualitative aspect to our program to give it “perspective.”

  • It is my experience that focus groups and real observation should precede ANY kind of survey. Some respondents have knowledge below the level of articulation. Some words are defined differently than management understands. So, no matter what technique you are using — survey or otherwise — it is wise to begin with focus groups.

    • Hi Steve,

      Focus groups allow for depth of conversation, but the challenge is getting a broad view of the market with them. I was involved in many focus groups during my time with eBay and the concern was always around the representation we were getting for eBay’s massive customer base. I do agree that observation is a valuable aspect to help set up a survey. This summarizes a lot of the value proposition of an advanced social intelligence solution; allowing the brand to gain deep insights across millions of consumers to “see” a majority of their entire base.

  • Hi Mark:
    Just so I can get my head wrapped around this Social Business Intelligence option…..does it also take into account the fact that upwards of 80% of purchase decisions are made at the store shelf the day of the shop or is that the next step following obtaining the likes, dislikes, attitudes, behaviors, etc.?

    • That number largely depends on the product. Consumers are mobile and are broadcasting from practically everywhere at all times. With some of the pharmaceutical brands we work with we’ll see upwards of 45% of patients discuss the drug their doctor prescribes moments after the office visit or at the pharmacy. It’s a mobile world allowing for these discussions to take place in real-time.

  • 26 weeks? Seriously? We turn field work via mobile surveys around for customers in a matter of days. Obviously there’s some survey prep work and data crunching that happens before and after. With agile partners this is still matter of two weeks tops.

    That is not to say that surveys definitely are not suitable for every occasion. Consumers are changing, behaviours are changing, channels are changing but for organisations who keep up there’s a whole lot of value in timely, well targeted surveys.

  • Surveys continue to be useful, but like observation and any other method, they have advantages and disadvantages. Surveys are a tool, and like any tool should be one of many sources of customer and market information.

  • This article is both biased and self-serving, plus out-of-touch (sorry to be so frank) with what is available out there. Social listening has been around for quite a while, and most CPG companies know both the pluses (there are some) and limitations – there are plenty. The tools out there that are really moving the needle are MROCs and mobile self-ethnography, IMHO. Oh and there are some disruptive survey tools that have 2 day turnaround and costing under 10% of what a major league provider would offer.

  • Painting the entire insight gathering and voice of the customer industry with a such a broad paint brush based on a very targeted quote from HBR against doing behavioral research in surveys is unfair.

    As with any research project, writing a survey is both an art and a science. Asking consumers behavioral questions in a survey is one way to gather the information, but it should be triangulated if you’re making big decisions using predictive models from other business intelligence data you have access to.

    Involving the direct voice of the customer in your decision making is still the best way to ensure you don’t make million dollar mistakes – whether you’re involving them at the exploratory/discovery phase; doing co-creation; or testing, prioritizing and validating your ideas.

    In fact, you can work with your customers directly to validate the hypotheses and insights you’re seeing in your big business intel data too.

    Don’t lose site of the big picture – keeping a pulse on the direct voice of the customer is critical to stay ahead of the competition.

  • Fantasitc article and equally interesting comments. I dont know where to start :). I agree that active web listening is going to up the game of brands. Private online communities is a also a great way for the brands to keet their ear to the ground and test some of the comments that are filtered through the social media big data. To Mike Wenning I would like to make a point about the ZERO MOMENT OF TRUTH; yes the customers do make the final decision in fron of the shelf (FIRST MOMENT OF TRUTH) but they are now heavily infuemced by what is going on in social media. I agree with Steve Bottfeld about qualitative research preceeding surveys and want to add that there are now richer online tools such as video diaries and asynchronous bulletin boards with rich media that improved the quality of insights a lot. To Andy Savage I would like to say that NPS is not sufficient to measure customer loyalty or satisfaction because it does not tell us why and it soes not tell us how to improve if our NPS is low. We now have an equivalent metric for social media which we call Net Sentiment Score. I look forward to discussng more with everyone.

  • Interesting point of view Mark. Pleasure to meet you virtually. Survey research certainly has limitations in its traditional form. I would only raise an equal number of concerns about the inherent biases in social listening data as well. The aspirational nature of social-sharing is well-documented by Dr. Alessandro Acuisti and Carnegie Mellon and others. People measure their comments based on how they believe they will be perceived, given that their identities are ascribed to those comments. Some form of purely discrete research (whether we call these surveys or not) is critical to painting the picture of a consumer’s unabridged and honest opinion.

Add a New Comment

Thank you for adding to the conversation!

Our comments are moderated. Your comment may not appear immediately.