The Evolution from SIEM to AI Driven Automation

This blog was originally published by LogicHub here.

Written by Kumar Saurabh, LogicHub.

Kumar Saurabh, LogicHub CEO and Co-founder, discusses his experience in the SIEM space and what he sees as the natural evolution from SIEM technology to AI and automation driven detection and response.

The Early Days of SIEM

I started in the security operations space 20 years ago at ArcSight, which was arguably one of the first Security Information Event Management (SIEM) products on the market. This was 2001, before SIEM was mainstream, and one of the challenges that security teams were facing is that the data that people needed for effective threat detection, triage, and incident response was massively siloed.

SIEM was a great tool to gather all the data in one place to let security engineers, SOC analysts, incident responders, and so forth – to perform security operations quickly and efficiently. For a number of years, it worked quite well! But around 2007 and 2008, we began to see that the amount of data was growing exponentially, and SIEMs could not scale to the volume of the data.

As a result, I co-founded another company called Sumo Logic, a cloud-based log management solution. At first, we were just trying to solve the big data problem – how do we go from a monolithic database to a distributed data architecture so that you can analyze terabytes of data every day? We solved that problem, and with the advent of the cloud, organizations could leverage hundreds of servers to crunch data. And now came the next problem – you need very intelligent people to extract insights from all that data.

Actionable Insights

The term “actionable insights” describes every security team’s daily lifeblood. How do you accurately collate, contextualize, and analyze the data you collect? How do you then turn that insight into immediate and efficient action? Is something a real threat or not? Is it high risk or low risk? How do you determine if it is a false positive? If it is a real incident, what is the right response that needs to be taken? To do this, you need skilled security personnel.

It becomes evident that the bottleneck is no longer the data or the compute – the bottleneck is now the people. There simply aren’t enough highly skilled and sophisticated people to go around. The answer to the ongoing shortage is that people need virtual assistance and automation.

There simply aren’t enough highly skilled and sophisticated people to go around. The answer to the ongoing shortage is that people need virtual assistance.

Virtual Assistance and Automation

People need machine intelligence to do a lot of work for them. Just like we use any sort of productivity tools, AI and automation can boost productivity and augment security teams that are strapped for time and resources – which, let’s face it – is almost every team.

Alert Overload

With a SIEM, not only is the detection lacking, but of the alerts generated, 90% to 95% of those alerts are not actionable. And yet every single one of them takes 15 to 30 minutes for an experienced analyst to go through it and validate why this is not a real incident. And as a result, you can have your entire team doing nothing but going through 3000 to 4000 different alerts a day just trying to stay on top of it.

This may be possible for a short period of time at some of the large companies that can afford that kind of a robust security team. But 80% to 90% of the security teams we talk to have less than 10 people. They cannot keep up. Forget 3000 alerts a day – they can barely go through 300 alerts a day! Even 30 alerts a day might overburden a team of five security engineers.

Using SOAR to Clean Up SIEM

In the last four or five years, we have seen security teams trying to use Security Orchestration, Automation, and Response (SOAR) products to “clean up” some of the noise that the sensors generate – combining SOAR with SIEM.

Perhaps some of the robust enterprises can get by with this approach, but unless you have 30 people to dedicate to alert triage, your SIEM is going to be relatively useless because it will generate a lot a noise, it won't find critical threats, and your people will have to do a lot of work figure out what is really happening. This is not an efficient way to run security operations.

But I Already Have A SIEM … Now What?

Many large companies have invested a lot of money in SIEM, and it’s not easy to pull the plug on it. But this is detection technology that has been around for 20 years, and it lacks learning, integration, and effective response. As simple rule-based systems, SIEMs end up generating a lot of noise and missing many of the real threats. A simple thing like a malicious PowerShell execution evades the system, and it ends up being utterly ineffective.

If you already have a SIEM and want to hold onto it, you can complement it with an AI and automation driven MDR service that coexists alongside your current system. Or you can ditch the SIEM entirely and upgrade to a SOAR platform. Whether you choose an MDR service or a SOAR platform for detection and response depends on your unique business needs – but either way, you have better options than staying with a SIEM.

Automating security operations ensures that machines do the “heavy lifting” of tedious, monotonous, or burdensome tasks. They can process massive amounts of data at machine speeds and machine scale all day, every day – because machines never sleep! Your “human team” is then free to devote their singular reasoning and expertise to more pressing and proactive security activities, and your business will be better for it.

About the Author

Kumar has 15 years of experience in the enterprise security and log management space leading product development efforts at ArcSight and SumoLogic. He has a passion for helping organizations improve the efficacy of their security operations, and personally witnessed the limitations of existing solutions in helping SOC analysts detect threats buried deep within mountains of alerts and events. This frustration led him to co-found LogicHub to empower cyber analysts by building intelligence automation, not just analytics.

Most recently Kumar was Co-founder and Vice President of Engineering at Sumo Logic. Previously, he was the data architect at Mint.com which was acquired by Intuit. Kumar was also one of the early engineering leads for the analytics and solutions team at ArcSight, and saw the company grow from zero revenue to IPO. Kumar earned his M.S. in Computer Science from Columbia University and B.S. in Computer Science from IIT Kharagpur.