Great applied technology typically needs enabling partner technology, and it will struggle to make headway until that partner appears. For decades, Intrusion Detection System (IDS) technology struggled to deliver efficient, high quality intrusion monitoring, and is only now experiencing success with the arrival of an unintentional enabling partner technology – cloud computing.
Let’s take a look through the history of IDS.
The IDS journey started thirty years ago when increasing enterprise network access spawned a new challenge: the need for user access and user monitoring. As day-to-day operations grew increasingly dependent upon shared use of information systems, levels of access to these systems and clear visibility into user activity was required to operate safely and securely.
Much of the initial headway on IDS was made within the U.S. Air Force. In 1980, James P. Anderson, a pioneer in information security and member of the Defense Science Board Task Force on Computer Security at the U.S. Air Force, produced “Computer Security Threat Monitoring and Surveillance,” a report that is often credited with introducing automated IDS. Soon after this report was released, the first model was built, born out of the same methods used by anti-virus applications: rule-based systems that constantly scanned and compared network traffic against a list of known threats.
During the late 1980’s, with a growing number of shared networks, enterprise system administrators all over the world began adopting intrusion detection systems. However, IDS presented a couple problems. First, it could only alert on known issues that had been categorized as threats on a signature list; zero-day attacks could compromise a network’s security. Second, the constant scanning and updating of a signature list was cumbersome and significant resource drain.
In the 1990’s, IDS technology improved to address the increasing number and sophistication of network attacks. This new method, named anomaly detection, relied on identifying unusual behavioral patterns on the network, and provided alerts for any identified abnormality.
Unfortunately, the inconsistent nature of networks through the 1990’s and early 2000’s resulted in a high number of false positives, and many administrators thought IDS to be unreliable, and headed for a slow death.
The advent of cloud computing, however, has brought new relevancy to IDS systems, resulting in a surge in the IDS market. An essential component of today’s security best practices, IDS systems are designed to detect attacks that may occur, despite preventative measures. In fact, IDS is now one of the top selling security technologies, and predicted to continue to gain momentum. After all, security — cloud security in particular – is far too complex to be monitored manually.
The logic and tactics IDS uses are more relevant today than ever before. With cloud computing, IDS has truly found an environment where it can thrive and be most effective. With cloud computing, the infrastructure has caught up with the IDS technology.
The consistent nature of servers in the cloud lends itself perfectly to IDS technology. As such, IDS is able to build stronger and more accurate baselines than were possible on the erratic on-premise network infrastructures of the past.
Big data also plays an important role in the growth and importance of intrusion detection today. The world’s data doubles every 20 months, and as cloud-hosted databases expand exponentially, it’s no wonder IDS is more important than ever.
At Threat Stack, we’re honored to play an important role in this evolution and to support the IDS community — both as providers and users — in continuing this positive progression. Most importantly, we’re focused on advancing IDS in the cloud and developing the best techniques for customers to collect and analyze their data to handle attacks across large, distributed environments.
Stay tuned for Part 2, in which we will highlight the importance of IDS for cloud environments, with a specific focus on the response side of security.