Perhaps one of the best overviews of Big Data was published last year by Harvard Business Review: Big Data: The Management Revolution, by authors Andrew McAfee and Erik Brynjolfsson. The authors explain that using big data will cause a “management revolution”, in that it will enable managers to measure and know a lot more about their business and customers, and lead to better decisions and improved performance.
Whilst not a substitute for reading the article itself, the three main questions addressed are summarised here.
What makes Big Data different?
The things that make big data different from analytics are:
- Volume: quite simply, there’s a lot more data made now than ever before, specifically “as of 2012, about 2.5 exabytes… each day”, which equates to roughly 50,000 million “filing cabinets’ worth of text”.
- Velocity: all this data is being created really fast, and notably in nearer to “real-time”.
- Variety: all this data comes in many forms, including social data – i.e. information generated and held in social networks, such as Facebook and Twitter. In addition, much of it is unstructured, i.e. “not organized in a database”, which presents the problem of analysis. However, analysis equipment and approaches are also ever evolving, and becoming increasingly cheaper.
Are data-driven decisions better decisions?
McAfee and Brynjolfsson assert that data-driven companies do indeed perform better in relation to typical financial and operational measures than less data-driven companies. For case studies, they cite a major US airline which used big data to better predict when planes would actually land, and thereby potentially saved “several million dollars a year at each airport”. They also cite Sears Holdings, which was able to analyse its large data sets much faster using Hadoop cluster stores, and could generate more pertinent and personalized promotions in closer to real-time (1 week instead of the usual 8).
They then move on to how it’s typically an organization’s HiPPOs (the Highest-Paid-Person’s Opinion) who make the important decisions, with many relying “too much on experience and intuition and not enough on data”. Data should be used more, and organisations should work with people who are able to ask the right questions of and around the data. In addition, you don’t need to spend huge amounts of money on IT and technology in order to use big data; it’s possible to “build… a capability from the ground up” (see the section on “Getting Started”).
What are the management challenges?
The final section of the article outlines five management challenges connected to making the best use of big data:
- Leadership: the real power of big data will be in combining it with human “insight”, “vision”, market knowledge, and the ability to take others on this journey too.
- Talent Management: data scientists are the people who will make sense of the big data; an often rare type of person in possession of both hard and soft skills. They are able to manipulate big data sets, while also making sense of them in business, management, and human terms.
- Technology: is required to deal with the data (with Hadoop being the most typically used at present), with these new technologies consequently requiring IT professionals to master new skills.
- Decision Making: the insight generated from the data will need to be in the same place as the people making the decisions, and must be capable of being understood by these decision makers. This will require organisations to be flexible and effective working across functional boundaries.
- Company Culture: moving away from intuition to be genuinely more data-driven will need to be embedded in organisational culture – along with, presumably, being willing to change and adapt in the wake of the new insights in order to really be able to capitalise on them.
I hope you enjoy reading the full article, and this post may encourage you to do so.
Read more: