Today, I’d like to continue our discussion on digital analytics — Are Social Media Analytics lying to you?
In exploring around the topic some more, I found other issues with big data across multiple platforms, not just your social ones.
Here are some of the problems I uncovered with big data across all types of digital analytics:
Problem #1. Response bias
Sure, we used that term more when we talked about survey responses, but I think we can use the same term to represent the problem in digital anlaytics.
Response bias (or bias of any type) refers to the problem when you data is systematically different from your population of interest. Using data from your website or email list results in response bias because data comes from a select group from your target audience — those who visited your website or who subscribed to your email list.
In a client project, I built an algorithm to score email list subscribers and discovered that a very significant number of subscribers weren’t even IN the company’s target audience but were competitors, students, and others who weren’t even valuable to the organization. Without this algorithm, the company wasted resources trying to convert folks who weren’t even prospects and likely made content and other types of decisions based on feedback from readers who didn’t matter.
In Friday’s edition of Analytics in Action, we discussed systematic bias among Facebook posts — where most of the conversation comes from a small number of posters who don’t represent the consumer population at large. That problem happens with your website, where visits from different devices by the same individual further obscure the true customer journey by fragmenting it across various devices.
Problem #2: Digital analytics are plain wrong
That’s right. You heard me.
In addition to being biased, your big data might be wrong — full of errors, duplicate counting, and just plain wrong!
In a recent PPC campaign for a client, I discovered just how inaccurate data is — even data coming from Google who makes their living by providing accurate data. As a data-driven agency, we monitor analytics on a daily basis — sometimes and hourly basis. I noticed something funny (or not so funny) in my data — the PPC campaign charged my client for a higher number of clicks than were recorded by Google Analytics (combined with Webmaster Tools).
Which number was accurate?
I never got a satisfactory response from the Google Ads Team.
I noticed a similar problem with Sprout Social. I routinely unfollow accounts that remain silent — why follow someone who never Tweets. I noticed a friend of mine appeared among my silent accounts, but I thought that didn’t seem right. So, I switched over to Twitter and found Tweets as recently as 1 hour ago. Now, I’m afraid to delete supposedly silent accounts.
Which leaves me pondering:
- What other data is inaccurate?
- How big is the difference in my digital analytics from reality?
- Are my decisions based on real data or just some fiction?
- Is there some way to fix data problems or even understand the extent of these problems?
Problem #3: Digital analytics don’t speak
Your digital data doesn’t speak — you have to construct queries to answer questions. Construct the wrong query or misinterpret what results mean and you’re making bad decisions.
Here’s what Scott Liewhehr told TechCrunch:
Everybody can use data to tell whatever story you want to tell and it’s a big challenge for marketers. If they don’t know how to run studies, they can make a lot of bad decisions.
Ask the wrong question, you get the wrong answer and make the wrong decision.
This means data scientists need an understanding of the firm — its business model, customers, strategies, etc. Which means pairing up data scientists with marketers within the firm or, better yet, training marketers to be data scientists.
The same goes for tools. Tools don’t provide answers, they provide a means to ask questions. Buying another tool isn’t going to magically solve your digital analytics problems.
Problem #4: Correlation isn’t causation
By extension, it bears repeating that correlation isn’t causation — no matter how big the coefficient of correlation is. But, big data offers the tantalizing option of seeing correlations and using them to inform decisions.
As an example, let’s say you pick up a series of click about my search for a new car. You pick up similar signals about me using cookies or other data sources that indicates my lifestyle and predicts income. You then start sharing information encouraging me to buy your high-end car.
But, whoops. You guessed wrong.
I don’t have the kind of income necessary and I’m not a prospect for your expensive car.
Now, you’ve wasted resources and maybe even damaged a potential relationship with me in the future. This happened to a friend of mine just last week.
Problem #5: Behavior isn’t understanding
Watching what people do, even in digital and mobile space, isn’t the same is knowing them or why the customer journey evolved the way it did.
- I may let someone use my mobile device to find information.
- I may research a product for someone else or as a gift — something I have no interest in.
- Maybe I’m researching a term paper, job prospect, or blog post.
Using behavior to infer WHY I chose a particular path along my customer journey is dangerous, but, when combined with big data about other behaviors, the danger is compounded.
Even when we ask consumers why they do something, we might get inaccurate data, but certainly inferring attitudes based on behaviors is wildly inaccurate.
For instance, my daughter is getting married this summer. I’ve been bombarded with emails for wedding albums, honeymoon trips, and other related wedding paraphernalia because I attended a wedding expo with her.
But, I’m not getting married — been there, done that, have the bruises.
I’m not making the decisions. In fact, I have little time or expertise in such matters so she’s got other folks helping her put the wedding together. All a company does by sending me this deluge of unsolicited email is incur my wrath — especially when I ask multiple times to unsubscribe from their list.
Problem #6: Reaching the top
Even when digital analtyics are working well, it’s tough getting top management to make decisions based on insights.
You make your report. Make recommendations. You move on to the next puzzle.
Management hears your findings. Ohs and ahs over your colorful infographics and visualizations. Nods heads appropriately.
Then.
Nothing.
Maybe it’s inertia or maybe fear of the unknown, but using customer insights to guide future plans is challenging for even the most data-driven organizations.
Many advocate for a C-level information officer, such as a Chief Analytics Officer or Chief Data Officer as a champion for digital anlaytics in the C-suite and as an advocate for using information as a tool in strategic decision-making.