When it comes to research that you question (or that you’re about to make business-changing decisions on), there are two questions you should ask to avoid making costly mistakes. First, does the data make sense from a statistical perspective, and second, does the research apply to you? Let’s look at these two critical questions using last week’s Gallup poll data as an example.

The Gallup poll asked the following question: “How much does social media typically influence your purchasing decisions?” While the poll was statistically significant and a representative sample, it was also very old in Internet time. Have things changed in the last year and a half since Gallup’s survey? We decided to test this for ourselves with a consumer survey to the US adult Internet population. What did we find?

The Gallup poll was right on the first two categories of people who believed that social media had a great deal of influence to them or some influence, but the last two categories people who believed it had no influence at all and people didn’t know, the results were wildly different.

In our survey, the number of people who were not sure whether social media influenced their purchasing decisions was significantly higher than in Gallup’s poll, whereas the people who said it had no influence at all declined by nearly the same amount. How do you interpret this data? Fundamentally, that “Don’t know” category means a lot of people aren’t sure how social media influences their purchase decisions – but it also means they believe that it may have at least a little influence, else they would have answered “No influence at all”. What’s striking about the differences in polls was the large margin of people who don’t know.

Let’s talk about numbers and percentages. Both the Gallup poll and our own poll did representative sample surveys to the US adult Internet population. The last full census of this population put that number of people at 228 million (2010 US Census, series 1157). Even if only 5.4% of people believe that social media has a great deal of influence, that’s still 12.3 million people. That’s not a small number of people or a small number of potential customers. Combining “A great deal of influence” with “Some influence”, you’re talking about 34.9% of the US population, or 79.5 million people. That’s an entire market, a third of the country. To draw the conclusion that social media isn’t influential would be the height of innumeracy.

That tackles question 1, “does the data make sense from a statistical perspective?”. The answer is that yes, it does – social media matters. Let’s address the second and arguably more important question: does this research apply to you?

The answer might surprise you: no. No, the research doesn’t apply to you as a brand or a business, in the sense that the opinion of a consumer about how they feel social media influences their decisions ultimately doesn’t matter. It’s largely irrelevant because what you should care about is not their opinion but what actions they take. This is doubly true when it comes to people’s subjective opinions about themselves.

What shows up in your business and marketing analytics, what they do – that’s what matters.

Here’s how you find whether social media is working for you and generating real results. Using Google Analytics, we’ll assume that you have it properly configured and that you have goals with dollar-amount goal values set up. Go to Conversions. Choose Multi-Channel Funnels. Select Assisted Conversions. Now look for social media:

If social media is in fact working for you, then the social media line should not be zero. There should be some goal conversions for social media. There may not necessarily be any direct conversions – people who buy right from a Twitter link or a Facebook post etc. If, however, social media influences purchasing decisions, then there should be some assisted conversions from social media, indicating that social media was part of the customer journey.

Ultimately, this is what really matters: people did what you wanted them to do, in part because of your efforts on social media.

Does social media influence purchase decisions? Only you can determine that from looking inside your own analytics. For some brands, the answer will be no. For other brands, the answer is a resounding yes.

The big picture conclusion here is that while the Gallup and SHIFT polls showed that social media has influence in the minds of the consumer, the data you should be paying attention to most is your own. Pay attention to the statistical and methodological validity of data you see in the news, absolutely, but pay even closer attention to the things that influence your business first and foremost.

Survey Disclosure Statement

SHIFT commissioned Google Consumer Surveys to collect responses from a representative sample of 1,000 respondents who answered the following screening question: “How much does social media typically influence your purchasing decisions?” This survey collected 2,135 impressions and 1,002 responses for a 46.9% response rate with an RMSE score of 4.7%. Possible answers were “A great deal of influence”, “Some influence”, “No influence at all”, and “Don’t know”. SHIFT Communications was the sole investor in the study. The survey population was the adult Internet user population of the United States. The date of the survey period was 6/25/14-7/2/14.