Threat Hunting: How It Works and 4 Tips for Success

What is Threat Hunting?

Threat hunting uses the skill of human security analysts, augmented by advanced technologies and threat intelligence, to identify and block malicious activities. It helps organizations take on a proactive approach to cybersecurity to stop attacks in their early stages and minimize the damage to the organization.

The starting point of threat hunting is assuming a breach has already occurred and threat actors are inside the network, covertly monitoring the environment and moving laterally. Threat actors can remain in the network covertly for weeks or months, to prepare for data exfiltration or a large-scale attack.

This is especially true for advanced persistent threats (APTs) — highly sophisticated threat actors who can avoid detection by automated security systems. Threat hunting helps find and stop these attacks by actively looking for indicators of compromise (IoCs). Based on these IoCs, they can identify and mitigate the threat before threat actors achieve their objectives.

How Cyber Threat Hunting Works

For threat hunting activity to succeed, it must have access to rich security data from the IT environment. This means that organizations must first deploy monitoring and security systems to collect data. This data can provide valuable clues to threat hunters.

Cyber threat hunters introduce a human factor into corporate security to complement automated systems. They are experienced IT security professionals who can detect, log, monitor and neutralize threats before they cause serious problems. Threat hunters might either be in-house security analysts, or they can be outsourced from an external provider.

Threat hunting techniques look for the unknown in the environment. They go beyond traditional detection technologies such as alerts from a Security Information and Event Management (SIEM) or Endpoint Detection and Response (EDR). Threat hunters deeply explore security data to look for hidden malware, signs of attack, or any suspicious patterns of activity.

When threat hunters discover a threat, they can provide guidance on remediating IT systems to eradicate the threat and prevent it from recurring.

Common Threat Hunting Techniques

Here are some of the most common techniques used by threat hunters to discover threats in an environment.

Searching

This includes querying evidence for specific artifacts using specific search criteria. The evidence could include full packet network data, logs, alerts, flow records, system events, digital system images, or memory dumps.

The challenge of this technique is that when starting a threat search, an analyst doesn’t know exactly what to look for, so the search could be too broad. It is important to find a balance between making the search specific enough so it is possible to review the results, but not too narrow so it can miss important artifacts.

Clustering

This technique relies on machine learning and artificial intelligence algorithms. It involves isolating clusters of similar data points in a larger data set, based on certain characteristics. In this way, analysts can gain a broader understanding of the data they care about most, uncover similarities and correlations, and derive insights into what is happening in an organization's network.

Based on the data obtained by clustering analysis, analysts can discover patterns and decide on the next steps of the investigation.

Grouping

This technique searches for multiple unique artifacts and uses predefined search criteria to determine when they appear together. This is similar to clustering, but involves retrieving only an explicit set of items found to be suspicious. It is typically done using a specific search query and not a machine learning algorithm.

While clustering is used by analysts to sift through very large datasets, grouping can be used to focus on items of interest in a smaller dataset or a specific area of the data.

Stack Counting

This method counts the number of occurrences of a value for a particular type of data, and analyzes outliers in the results. Stack counting works best when analyzing a dataset that contains a relatively small number of results for a certain query, and provided that the query is carefully designed.

The ability to organize, filter, and manipulate problematic data is key to finding anomalies in large data sets. To use this technique, analysts should leverage data exploration skills and use data analysis tools—from a basic tool like Excel to full-featured BI tools.

4 Tips for Threat Hunting Success

The following best practices can help you practice threat hunting more effectively in your organization.

Set Aside Dedicated Time for Threat Hunting

The security team has several responsibilities, including protecting infrastructure, investigating alerts, and other activities. If the same team is responsible both for security and IT, there are even more responsibilities for each team member. The danger is that if threat hunting is assigned to team members alongside other responsibilities, there will always be something more urgent to do.

Threat hunting is usually treated as less important because it is a proactive activity and does not involve responding to immediate, known threats. However, these early investigations are critical to detecting more advanced and unknown threats. To enable a significant time investment in threat hunting, define a dedicated threat hunting role, or at least define a minimum amount of time a team member should spend per week on threat hunting, regardless of other priorities.

Use Automated Tools

Efficient threat hunting requires the ability to quickly prove or disprove assumptions about threats to an organization. This includes the ability to collect and analyze data from a variety of sources inside and outside the organization.

Threat hunters can gather this information manually, but it is time consuming and requires a lot of knowledge and expertise. Investing in specialized security solutions, such as SIEM and dark web monitoring solutions, can significantly speed up the threat tracking process.

Prioritize Based on Risk

Threat hunters can investigate a variety of potential threats to an organization. There will always be more testable hypotheses than a threat hunter can actually investigate. Therefore, when planning an investigation, threat hunters should prioritize based on the potential risk to the organization. Another factor affecting priority is the probability of exploitation. Focusing on high-risk, high-probability threats will help maximize the value of threat hunting activity.

Leverage Outsourced Experts and XDR

There are important benefits to outsourcing threat hunting activity to an external security service provider. Security providers have teams dedicated to threat hunting, with individuals who have specialized expertise in this discipline. They typically also provide state of the art technology that can help identify threats more effectively.

Some security service providers provide extended detection and response (XDR) as part of their service. XDR solutions can collect data across networks, cloud workloads, identity management systems, and email protection systems, identify anomalies, and automatically assemble the kill chain for suspected attacks. This can be a game changer for threat hunters, reducing the time to investigate and find a threat from hours or days to minutes.