Into the Wild: Using Public Data for Cyber Risk Hunting

By Vaidotas Sedys, Head of Risk at Oxylabs

Cybercrimes cost companies more and more every year. With a 21% year-over-year increase in the U.S., 2023 witnessed an all-time high of 12.5 billion U.S. dollars lost in damage caused by cybercrimes. This comes as no surprise, considering that the number of reported cyberattacks in the U.S. has been growing steadily from around 250,000 in 2016 to 480,000 in 2022.

No company is secure from threat actors, as the next cyberattack can lurk behind a corner or even hide inside an organization's systems. Reacting to cybersecurity threats once they have started is too late. Companies need to adopt a proactive approach. As cyber risks become increasingly advanced, the best way to approach these risks is by going into the wild and hunting for threats.

Threat hunting is the active search, identification, and isolation of threats. When combined with threat intelligence—the knowledge and understanding of risks, their motivation, and techniques—threat hunting builds a mighty net that can catch even the sneakiest threats before they cause any damage.

To fully understand how your business can build a bulletproof security strategy, it's important to understand what threat hunting and threat intelligence are and how they can support your company's efforts to identify patterns and predict potential threats before they occur.

Related:Linux Ransomware Threats: How Attackers Target Linux Systems

Threat Hunting: What It Is and Why It's Important

An upward trend in the number of security risks emerging every year calls for changes in the response to these threats. A reactive approach, which involves dealing with threats only when they're at the doorstep—or even worse, when they've already attacked—is expensive in many ways. It might bring financial and reputational damage and harm clients if their data is affected. 

Threat hunting, on the contrary, is a proactive approach. It means that cyber teams go out into the wild and proactively identify potential risks and threat patterns, isolating them before they can cause any harm.

A threat-hunting team requires specific knowledge and skills. Therefore, it usually consists of various professionals, such as threat analysts, who analyze available data to understand and predict the attacker's behavior; incident responders, who are ready to reduce the impact of a security incident; and cybersecurity engineers, responsible for building a secure network solution capable of protecting the network from advanced threats. 

These teams are trained to understand their company's IT environment, gather and analyze relevant data, and identify potential threats. Moreover, they have a clear risk escalation and communication process, which helps effectively react to threats and mitigate risks. 

Related:AI Chatbots Ditch Guardrails After 'Deceptive Delight' Cocktail

Specialists often use a combination of tools that help in threat hunting. For example, they employ security information and event management (SIEM) systems that collect event log data from various sources and analyze it in real time to identify deviations. Intrusion detection systems (IDS) enable network monitoring for suspicious activity. Endpoint detection and response (EDR) systems combine continuous real-time monitoring and collection of end-point data with a rule-based automated response.

While many companies still struggle to integrate threat hunting into their business operations, looking in retrospect, it becomes clear that employing a proactive approach is vital. For example, a year ago, terabytes of private data were accidentally exposed by Microsoft AI researchers, an incident that could have been prevented by robust Open-Source Intelligence (OSINT) efforts.

Threat hunting is one of the most effective ways of anticipating cybersecurity risks and deactivating them before they cause harm. But it wouldn't be possible without having the relevant knowledge and publicly available web data, which is where threat intelligence steps in.

Related:We’re Worried About Deepfake Voice Scams. How Do We Protect Employees?

Threat Intelligence as the Key Component

Threat intelligence is the knowledge and information (or data) that allows you to prevent or mitigate cyber risks. It can help understand a threat actor's motives, targets, and behavior. This knowledge comes from skills, experience, and various data sources that help create a comprehensive picture of potential threats.

Threat intelligence uses various sources to gather relevant data points. It can encompass technical data, Social Media Intelligence (SOCMINT), Human Intelligence (HUMINT), and OSINT. The latter refers to publicly available web data that can be gathered from the Internet using web scraping tools.

OSINT contains information from public websites, open forum chats, dark web marketplaces, and many other sources. Monitoring these spaces can help companies identify their vulnerabilities. For example, a company can find its data lurking online, suggesting there might have been an unintentional data leak or a criminal data breach. Moreover, OSINT mostly uses publicly available information, which means companies don't have to invest resources into gaining access to classified or restricted data to collect criminal evidence.

Companies can acquire threat intelligence, including OSINT, using web intelligence solutions. Modern data scraping tools, powered with advanced artificial intelligence (AI) and machine learning (ML) features, improve the threat intelligence collection process as they enable the pulling and analyzing of raw data in real time.

Combining Threat Hunting and Threat Intelligence

Threat hunting and threat intelligence go hand in hand and should be employed together for the best result. How does this combination work?

Risk hunting can be done proactively or reactively, and both approaches can be combined with threat intelligence. Active threat hunting defines a proactive search of potential threats. Threat intelligence can guide the process by helping focus on the most vulnerable system areas. 

Reactive threat hunting means responding to incidents or alerts. Threat intelligence can provide helpful insights into methods and tactics used by the threat actors, helping to identify and eliminate threats quickly.

For example, threat hunters receive an alert about a suspicious login attempt into their systems. They have intelligence from public forums about an increase in such login attempts and know its specific characteristics. Other companies also reported how they successfully managed to neutralize this risk. Here, combining the reactive threat-hunting approach and threat intelligence helps resolve the case effectively.

Attack modeling is another example of how risk hunting and threat intelligence can complement each other. Information about industry-wide threats can be used to model potential attacks, which helps anticipate and prepare for threats. For example, threat intelligence suggests that a certain threat actor group initiates their attack by sending phishing emails. With this information, threat hunters can focus on monitoring incoming emails or spreading awareness of phishing attacks within the company to make colleagues more vigilant.

Moreover, combining the two approaches enables decision-making based on internal and external data. Threat intelligence platforms gather data from various sources, which helps paint a wholesome picture of the threat landscape. Meanwhile, threat hunters additionally analyze internal company data. Combining these two data sets can help make informed decisions for risk mitigation within an organization.

Employing AI for Cyber Risk Hunting and OSINT

Quickly advancing AI technologies have the potential to play an essential role in changing the way businesses approach and combat cyber risks. Already today, cyber specialists are experimenting with various AI solutions to enhance threat hunting and threat intelligence efforts. 

When it comes to threat hunting, ML and AI can help cyber teams identify complex patterns hidden in large datasets and predict potential threats before they occur. This is often called automated threat detection. 

AI algorithms can analyze massive amounts of information, such as network traffic, systems logs, and user behavior data. Specific patterns and deviations that might be unnoticeable to the human eye can suggest a potential threat. AI-powered threat detection also includes historical threat data analysis, the basis for predictive model development. Speed is the main advantage of using AI-driven technologies. These systems can monitor and detect anomalies in nearly real time, which is impossible for cyber teams that rely solely on human intelligence.

AI also advances threat intelligence and OSINT efforts. Using natural language processing (NLP) techniques, such as sentiment analysis or named entity recognition, AI systems can analyze even unstructured public data sources, such as news articles or social media feeds.

Modern web scrapers also employ AI for more effective public web data collection. This can employ dynamic fingerprinting, ML-driven proxy management, and response recognition techniques that help navigate complex websites using strong anti-scraping measures that can significantly impede the efforts of cybersecurity professionals.

Conclusion

Threat hunting and threat intelligence are essential for building a strong cybersecurity strategy. Proactively searching for cyber threats while using internal and external data can help detect potential risks and mitigate them before they cause any real damage. AI-based systems help to automate risk identification processes, including real-time web data collection and analysis.

Considering the current cybersecurity landscape and the rapid advancement of cyber threats that are also augmented by AI technologies, proactively hunting risks and neutralizing them before they cause any harm should be the aim for all organizations that care about the safety of their systems, their clients, employees, and reputation. 

About the Author

Vaidotas Sedys is Head of Risk at Oxylabs.