Big data or large sets of complex data stored across different systems, when pulled together in the right context, may offer hidden insights to trends and even patterns that can help us navigate through life. This unflattering concept may help determine the future of healthcare, businesses, academics, and even government. However, emerging technologies offer varied types of data that offer a potential for new insights, which can answer questions that may have seemed unreachable before. But how does this new capability impact us on a day-to-day basis? And what opportunities do big data offer?
In order to identify opportunities created from using big data we must first understand how it can be processed. As explained in this IBM post, big data spans four dimensions – Volume, Velocity, Variety and Veracity.
- Volume – Big data requires handling and processing large quantities of data. For example some companies may have to process 12 terabytes of tweets to create sentiment analysis
- Velocity – Companies may have to process data in real-time and is often time sensitive. For example, a government agency that has to review 5 million trades in a day to identify any fraudulent activity
- Variety – Big data can be a combination of data from multiple sources including sensors, text, audio and video. This variety of data has to be structured and analyzed properly in order to identify insights
- Veracity – Data analytics is a highly sensitive field and establishing trust is a vital component of working with big data. However, establishing trust is a challenge as you increase the variety and number of sources
The ability to process a large volume of varying data from multiple sources is an integral part in big data. This ability to harness big data in a timely fashion allows developers to identify trends, which can help create applications that improve efficiency and create insights.
Big data has evolved considerably and today supports various initiatives ranging from mobile apps, mapping software and even advancements in medical technology. A great example of such evolution can be found on Reddit where users post a variety of charts and analysis that capture trends generated using big data capabilities. The ability to translate large amounts of data into simple yet intricate analysis helps users to leverage from the insights identified using big data capability. Whether you’re a Data Analyst or simply a business user such analysis is now possible due to the use of big data.
An example application of big data can be found in this article, which highlights software such as the Transit Time NYC. This interactive map developed by WNYC allows New Yorkers to click on a spot in any of the city’s boroughs for an estimate of subway or train travel times. The map was created from a combination of Wikipedia mapping software and the public MTA train schedule. The use of open source applications and direct collaborations allows software developers to maximize the potential created from big data.
Yet another example is an interactive map supported by big data is Jack Dangermond’s Urban Observatory. The mapping software allows users to compare cities and urban settings in relation to each other. This relational mapping will provide an opportunity for identifying trends within a particular region or even comparing the differences in two urban environments. The use of this software can help city officials adopt successful strategies from other urban designs such as enhancing road patterns to improve traffic flow.
Big data is also starting to generate a larger presence in the medical community. At the Alzheimer’s Association International Conference a big data project was announced, which could boost diagnosis and treatment of the disease. The project provides data on entire genomic sequences of 800 participants that will be available via a cloud-based system. This system will allow researchers to download and process data that may offer insights on Alzheimer’s. The availability of big data is very important in brain research, which requires large amounts of analysis to identify the smallest changes and all of this information to be processed within a few seconds or minutes. The project will help generate and identify trends that may aid diagnosis and treatment of the disease.
We see an increased use and application of big data to support real-life business situations. From urban mapping software to brain mapping technology, big data offers a wide variety of opportunities to analyze data drawn from multiple systems. The ability to identify insights and trends allows researchers in a multitude of fields to determine specific actions. Needless to say big data is starting to grow and the opportunities are endless.
We will look at some specific examples of how big data is helping various organizations in a separate post.