Welcome back, buzzword barons!

We’ve reached the third installment on my series about buzzwords and the real, important trends behind them.

In this edition, we’re going to tackle a buzzword with an incredible amount of misunderstanding swirling around it: Big data.

That’s right – you’re about to get the goods on what big data really is, why you should care and how it can benefit your brand.

What does “big data” mean?

For starters, “big data” means more than “a whole lot of data.”

While there is a literal interpretation based entirely on volume (“Wow! So much data!”), big data goes beyond the notion of a really, really big spreadsheet.

As explained by Lisa Arthur,

“Big data is a collection of data from traditional and digital sources both inside and outside of a company that represents a source for ongoing discovery and analysis.”

Another definition:

“Big data is a broad term for data sets so large or complex that traditional data processing applications are inadequate.”

In other words, you might think of big data as referring to a large volume of data or data collection and analysis on a gargantuan scale.

Breaking down big data

Big data sets often include structured, unstructured and multi-structured data that is challenging to capture, curate, process, analyze and manage through traditional data processing methods. Here’s what that means in layman’s terms:

  • Structured data is data that fits easily into traditional “row and column” formats. Think numeric values that are easy to quantify.
  • Unstructured data is data that’s not easily organized or interpreted by traditional databases and models. This includes things like text-heavy data found in social media posts, emails, videos and so on. 
  • Multi-structured data refers to a combination of both structured and unstructured data, i.e., the kind of thing you derive from interactions between people and machines.

To truly understand what makes big data “big,” there are certain characteristics you can analyze and identify:

  • Volume: Big datastems from the fact that we’re creating more data than ever – about 2.5 quintillion bytes of data per day. It’s not so surprising when you think about the growing number of data sources and collectors we have today: Sensors, machines, social media, RFID tags … you name it; it’s collecting data.
  • Variety: Today’s data comes in a huge variety of formats, from structured numeric data to unstructured text documents like email.
  • Velocity: In our interconnected world (and the emerging “internet of things”), data is streaming in to be collected, scrubbed and analyzed in real time. Take a company like FedEx: If they analyze GPS signals to locate their trucks, cross-reference the location with RFID tags and cross-reference that with traffic data, they can optimize shipment deliveries based on time, fuel consumption and more. In other words, big data requires quick analysis and rapid reaction. 
  • Variability: Variability refers to inconsistencies in the way that data flows, with peak periods and valleys. This accounts for things like social media trends and seasonality. 
  • Veracity: This refers to the quality and reliability of the data being captured, which can vary greatly depending on the collection methods being used. 

How is big data being used?

From oil and gas to consumer retail, big data is being used for everything from predictive analysis to customer experience research. This piece outlines 10 big data case studies, some of which include:

  • Macy’s used big data to implement real-time pricing adjustments to nearly 73 million items.
  • American Express uses big data to predict customer loyalty or when an account will close.
  • Researchers are using big data to decode entire DNA strings in a matter of minutes.
  • Sports teams are using big data to crunch numbers on player performance and test different equipment.
  • Retailers are using big data to generate personalized coupons and customer loyalty programs.

As you can see, there is a myriad of ways to use big data, especially when it comes to understanding consumer behavior in real-time.

What are some of the problems and challenges with big data?

For all of its merit, big data can be difficult for businesses to embrace. There are even some who argue that businesses are making worse decisions based on false positives and misplaced confidence.

A few challenges to be aware of:

1. Talent is hard to find.

Data scientists – the people adept at dealing with big data – are exceedingly hard to find and hugely in demand.

2. Leadership needs to adapt.

Even in 2015, leadership has a propensity to make decisions based on hunches, gut feelings and personal bias. Organizations who want to embrace big data need leaders who are willing to set goals, define success, deal with data, analyze outcomes and ultimately make a data-driven decision.

3. Company cultures need to adapt.

As eloquently written by Andrew McAfee in the Harvard Business Review,

“The first question a data-driven organization asks itself is not ‘What do we think?’ but ‘What do we know?’”

He claims that many company cultures pretend to be more data driven than they actually are, only doing research after a decision has already been made (usually to try and justify what’s already been decided).

4. Context is still critical.

Just because you’re analyzing a great deal of data does not mean your conclusions are more accurate. In fact, it can be far more difficult to draw any meaningful conclusions from a larger data set.

As described by Shvetank Shah, Andrew Horne and Jaime Capella, good data won’t guarantee good decisions. In fact, fewer than 40 percent of employees have the skill sets and mature processes required to derive accurate insight.

Couple that with larger data sets and more substantial time constraints, and you’ve got more false positives, distractions and dangerous decisions about what to measure, what to discard and how much weight to give every data point.

5. Human decision-making is as important as ever.

Big data requires big judgment, i.e., the ability to carefully weigh conclusions and make intelligent decisions. With big data, information is often transferred between people and departments of wildly different skill sets, biases and focuses, so a collaborative decision-making process is a must.

As McAfee states, “People who understand the problems need to be brought together with the right data, but also with the people who have problem-solving techniques that can effectively exploit them.”

Big data can have a big impact!

It’s hard to deny the power of collecting, storing, measuring and analyzing data at an enormous scale. From improving customer engagement to optimizing sales operations, virtually every organization can benefit from big data.

Though big data challenges are difficult and immense, it’s the brands brave enough to swim the big data ocean that will someday lead the pack.

Visual header: Casey Partenio