How-To: Combating Security Alert Fatigue

Eric Adams understands the effect of security alert fatigue from many vantage points. As a longtime security professional, he has experienced it many times himself. And as chief information security officer of Kyriba, a tech company whose products help businesses protect against fraud and financial risk, he has seen the effect it has on good security people who have become completely frustrated, numb or less effective because of security alert fatigue.

"We have seen that if there are more than about 75 events over an hour, it's just too many alerts," he said. "It's like being an air traffic controller. When you have a certain threshold of events per hour, you run the risk of an analyst not running the full playbook or analysis of an event."

In fact, alert overload is a huge problem, and it can sabotage security operations. According to one recent report, the vast majority of security analysts say it takes more than 10 minutes to investigate each alert. It's just a matter of doing the math.

In addition to alert overload, other challenges contributing to security alert fatigue include false positives and security analyst churn. One study found that more than two-fifths of organizations experience false positive alerts in more than 20% of cases, while 15% reported that more than half of their security alerts are false positives.

Here, experts weigh in on ways to reduce security alert fatigue.

Upgrade and modernize if you can. Ideally, this will include as much automation as possible. Jason Mical, cybersecurity evangelist at Devo Technology, says the best way to do that is by replacing your legacy security information and event management (SIEM) system with a newer version that is a more automated, rule-based system and relies more on artificial intelligence and machine learning. Newer SIEMs have access to all of the data in an environment instead of just security data. That gives more context to every alert and helps with prioritization. More modern SIEMs also tend to provide more visibility. These capabilities can help winnow down the number of alerts that are actually actionable, helping reduce alert fatigue.

For Kyriba, the solution was an automated security operations center (SOC) based on software from Respond Software with an integrated SIEM. "In our case, the Respond software covers Levels 1 and 2 alerts and can take actions based on a playbook, and escalate only those that need personal attention," Adams said. "So our personnel only look at those qualified alerts, determine whether they are valid or a false positive, and provide feedback into the Respond tooling." The automated nature of this solution helps reduce alert fatigue and frees analysts up to work on other tasks.

If you can't upgrade, you're not completely out of luck; focus on fine-tuning what you have. That includes:

Be careful how many point solutions you add. While it may be tempting to add more tools to your legacy SIEM, choose carefully or you will add more complexity. Sometimes, the more data and products you apply to your SIEM, the less responsive it can become.

"Can your people handle all of those tools? Probably not," Adams said. "So focus on the most effective, efficient and automated tools. It's a real balance to be able to make it work for you. You need a basic set of tools and good set of telemetry, but you have to be able to handle that data in an effective way." Make sure your personnel and processes can handle the load, he added. "We don't bring on any more tooling that makes any more load for us. We look at tools that allow us to reduce the load on the human personnel."

No matter which route you go, testing is critical. Make sure you are doing red team/blue team exercises to validate your rules, and do penetration testing to ensure that there is no way to circumvent them, Michal said.