The Data Dam Has Burst

The Data Dam Has Burst image broken damIn the old days, when science and technology produced exciting innovations every year, it was easy to keep up with new developments. People picked up the newspaper and read daily about what was going on in their lives. Nightly news broadcasts from such treasured anchors as Walter Cronkite, Tom Brokaw and Frank Reynolds spotlighted the big stories on half-hour newscasts.

In the 21st century, that notion seems so quaint. The advent of the Internet and new technology has released the floodgates. The dam has burst. What spews forth now from most websites is a steady stream of new data, and the problem is today’s colleges, institutions and places of business can’t process all of that information at a proper rate of speed.

This phenomenon is called data deluge. It looked to threaten most people who feared they might drown in the waves of information coming over them. That is no longer the case, as new technologies and innovations have come up to fight the deluge and slow the roar of water to a small stream.

A recent paper titled “From Data Deluge to Data Curation”, written by Philip Lord, Alison Macdonald, Liz Lyon and David Giaretta, suggests that companies and institutions are finding ways to sift through the content coming in on a daily basis and prioritize it based on how important it is to that institution. Archiving and preserving this data is the most important piece of the puzzle. Think of the curation process as a news ticker generating the day’s stories one at a time. In today’s rapid-fire presentation, the stories are coming at you fast and furious, and it is up to your team to decide which ones are important and need to be archived online and in newspapers.

Businesses are handling data deluge in pretty much the same way. As data flows into companies’ computers, people determine the most important items to take care of and which ones can be delayed. Prioritizing tasks and culling the important stuff out of the data fields saves a lot of time and money in some cases, and it can also avoid plenty of headaches.

Recommended for YouWebcast: The Art of Growth Hacking: Gaining Early Traction by Doing Things that Don't Scale

Companies are prepared to handle this data deluge by creating servers with storage capability. Companies such as LSI handle this information. The founder of LSI, Abhi Talwalkar  has served as CEO for nearly eight years, and he comes from a background of servers, storage and communications. Storage companies design software that speed up the storage and networking process to create easily-flowable amounts of data.

Massive amounts of data are being swallowed by customers with smartphones. No doubt if you own a phone you have heard the phrase “Now there’s an app for that?” Applications of all kinds are separating data into easy compartments. You may have an app for news on your phone. There may be one or two others devoted to sports. Want a peek at how the day’s traffic looks on your commute home? Punch a button, and there you are.

There are no signs the data deluge is slowing. You learn something new now every minute, as opposed to every day. Rest assured companies are ahead of the curve and ready to compartment your information and prevent the deluge from turning into a thousand-year flood.

Discuss This Article

Comments: 0

Add a New Comment

Thank you for adding to the conversation!

Our comments are moderated. Your comment may not appear immediately.