What is Big Data?
It is the sum of all the information about everything and anything that is being captured every second of every day.
Today the amount of data being generated is greater than ever before. Be it information about the general population, education, health and medicine, travel, geographic locations, shopping, financial transactions, jobs — you name it; there is data about it. Social networks have only added even more information to the world’s data by contributing information down to a personal level.
Big data is the most talked about topic in IT today. Every organization wants to be able to harness this data in order to make better decisions. Software companies are focused on developing solutions and services that allow businesses to turn big data into real insight.
Here are a few fascinating facts about this hot topic:
- In the third century BC, the Library of Alexandria was believed to house the sum of human knowledge. Today, there is enough information in the world to give every person alive 320 times as much of it as historians think was stored in Alexandria’s entire collection — an estimated 1,200 exabytes’ worth. If all this information were placed on CDs and they were stacked up, the CDs would form five separate piles that would all reach to the moon.
- This explosion of data is relatively new. As recently as the year 2000, only one-quarter of all the world’s stored information was digital. The rest was preserved on paper, film, and other analog media. But because the amount of digital data expands so quickly — doubling around every three years — that situation was swiftly inverted. Today, less than two percent of all stored information is non-digital. Source: www.foreignaffairs.com – The rise of big data
- Obama, The Big Data president. The “big data” presidency transcends intelligence-gathering and surveillance, encompassing the White House’s approach on matters from health care to reelection. A big-data fact sheet the White House put out in March 2012 — upon the launch of its $200 million Big Data Research and Development Initiative — listed more than 85 examples of such efforts across a number of agencies. They include the Cyber Infrastructure for Billions of Electronic Records (CI-BER), led in part by the National Archives and the National Science Foundation, and NASA’s Global Earth Observation System of Systems (GEOSS), which the fact sheet described as a “collaborative, international effort to share and integrate Earth observation data.” And the Defense Department is putting about $250 million a year into the research and development of such projects — “a big bet on big data,” as the White House called it. Source: Washington Post- Obama the big data president
- The business of information management—helping organizations to make sense of their proliferating data—is growing by leaps and bounds. In recent years Oracle, IBM, Microsoft and SAP between them have spent more than $15 billion on buying software firms specializing in data management and analytics. This industry is estimated to be worth more than $100 billion and growing at almost 10% a year, roughly twice as fast as the software business as a whole. Source: Economist.com
- According to IBM, 90% of the data in the world today was created in the last 2 years alone. This data springs from such diverse sources as weather, social media posts, digital pictures and videos, purchase transaction records and cell phone GPS signals, to name a few. Not surprisingly, businesses continue to grapple with how to ingest, understand, and operationalize Big Data. A recent study of 75 North American retail executives found that 46% of retailers considered the volume of information with which they must contend to be the biggest challenge in dealing with Big Data. 34% say managing data variety – both structured and unstructured – is the top challenge; 20% say handling data velocity, or the frequency at which data are generated and captured, is hardest. Source: Forbes.com- big data get ready for the 2013 big bang
- Data never sleeps. Every minute of the day Apple receives around 47,000 App downloads, Brands and Organizations receive 34,722 Likes, 571 new websites are created, Facebook users share 684,478 pieces of content, Google receives over 2,00,000 search queries, Twitter users send over 1,00,000 tweets, Consumers spend $2,72,000 on web shopping. Source: How much data is created every minute
- With SAP’s In-Memory technology, businesses have the ability to run big data analytics on 80 terabytes of data, integrate with Hadoop*, search text content, harness the power of real-time predictive analytics, and more. They can analyze hundreds of billions of rows in seconds, not days, and exploit unstructured data such as text, documents, web, and social media content. From June 2011 to Dec 2012, in 18 short months, SAP AG’s implementation of in-memory database technology had over a 1000+ customers and EUR 550M+ in revenue to become the fastest growing product in SAP history. Source: SAP Big Data Analytics and SAP HANA
*Hadoop is an open source software project that enables the distributed processing of large data (Wikipedia)
- 140,000 to 190,000 people with deep analytic skills as well as 1.5 million managers and analysts will be needed by 2018 to fill jobs in Big Data.
40%-60% annual growth increase is occurring in the volume of data available every year; in media intensive sectors and financial services, the increase is 120%. Source: McKinsey and 38 big facts big data companies
- Aadhar* an ambitious Indian government Big Data project aimed at becoming the world’s largest biometric database by 2014, with a goal of capturing about 600 million Indian identities. This could help India’s government and businesses deliver more efficient public services and facilitate direct cash transfers to some of the world’s poorest people — while saving billions of dollars each year. Source: Forbes.com 5-ways-big-data-will-change-lives-in-2013
*Aadhar is a 12-digit unique number which the Unique Identification Authority of India (UIDAI) will issue for all residents in India.
- The big data market, according to Gartner, drove $28 billion in worldwide IT spending in 2012. This year, that number is expected to grow to $34 billion, as more organizations invest in new technologies and services that are purpose-built for big data. That spending, Gartner estimates, will be scattered across a number of IT landscapes. Demand for new social network analysis and content analytics tools will drive much of it — nearly 45 percent, Gartner says — while investments in application infrastructure, middleware and data integration technologies also are expected to soar. Source: CRN.com are-you-up-to-the-challenge-seizing-big-opportunities-in-big-data
And the above facts are just the tip of the iceberg. Its crucial that we use Big Data to create opportunities that add value in today’s world. And it’s equally important that we appropriately manage the obligations to individual privacy and customer rights. The actual fact is big data has evolved for a reason: we need it.