There’s a lot of new terminology floating around business and startups these days. As I’m working with a technology company that specializes in big data access & analysis, I’m learning so much more about what’s going on that impacts SMBs and Enterprise level businesses.
In my #LeanCoffeeTO meetups over the last 18 months, we’ve talked about Big Data (I’ve listened to them talking) a number of times, and now it is becoming a main focus the community I’m working to build and the conversations I’m charged with participating in and influencing.
WHAT IS BIG DATA?
Big Data is one of the biggest trends in technology as enterprises increasingly need to manage the explosion of data caused by trends like cloud computing, the rise of mobility, globalization, and social media. This proliferation of data has caused enterprises to need new tools and processes to collect data (both structured and unstructured), store data, manage data, manipulate data, analyze data, and aggregate, combine, and integrate data. There is no set definition for Big Data although many third-party firms have provided their perspective. We define Big Data as data sets of extreme volume and extreme variety.
Big data technologies describe a new generation of technologies and architectures,designed to economically extract value from very large volumes of a wide variety of data, by enabling high-velocity capture, discovery, and/or analysis.
Big data: techniques and technologies that make handling data at extreme scale economical.
Big data is a term applied to data sets that are large, complex or dynamic (or a combination thereof) and for which there is a requirement to capture, manage and process the data set in its entirety, such that it is not possible to process the data using traditional software tools and analytic techniques within tolerable time frames.
McKinsey Global Institute
Big data” refers to datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze. This definition is intentionally subjective and incorporates a moving definition of how big a dataset needs to be in order to be considered big data (i.e., we don’t define big data in terms of being larger than a certain number of terabytes (thousands of gigabytes). We assume that, as technology advances over time,the size of datasets that qualify as big data will also increase. Also note that the definition can vary by sector, depending on what kinds of software tools are commonly available and what sizes of datasets are common in a particular industry. With those caveats, big data in many sectors today will range from a few dozen terabytes to multiple petabytes (thousands of terabytes).
Source: McKinsey Global Institute (MGI) – Big data: The next frontier for innovation, competition, and productivity
When business leaders or data management professionals talk about big data, they often emphasize volume, with minimal consideration of velocity, variety and complexity – the other aspects of quantification: Velocity involves streams of data, structured record creation, and availability for access and delivery. Velocity means both how fast data is being produced and how fast the data must be processed to meet demand. Variety includes tabular data(databases), hierarchical data, documents, e-mail, metering data, video, image, audio, stock ticker data, financial transactions and more. Complexity means that different standards,domain rules and even storage formats can exist with each asset type. An information management system for media cannot have only one video solution.
How does big data apply to marketing professionals and business students?
I went to iTunes University and searched for Big Data, and found a fascinating keynote at Pace University from Bart Flaherty, CEO GroupM Business Science > download & watch the video.
There is also a storify of tweets from his presentation, you can read Lubin School of Business at Pace University storify Bart Flaherty keynote.
Very interesting stuff. Most notably, Bart feels that being able to work intelligently with large datasets in our industry of BIG-money-media-investments can do two main things (my own takeaway, yours may be completely different):
1. Agency CIOs and brand CMOs can protect their (client) allocation of marketing dollars during tight economic times with data and the ability to analyze the information on-the-fly, so that CMO’s have the information to make the decisions or share the information internally to protect their position/budget
2. When you understand and accept that the data and research we’ve relied on for so many years is not perfect, and you can understand, analyze and interpret information that is a mix of “reliable” and constantly changing, you’ll be MUCH better positioned as an organization and individual professional to compete.
Of course, since he was speaking to a group at university, he was strongly positioning Data Scientists and data analysis roles as key positions for students to consider. He even had a slide that suggested if you want to be “rich and famous” in the new world of overwhelming information sources, consider if you would be a good fit for a career in data management and analysis. It certainly applies in media, buying and selling.