Have You Heard of P-IDS? Intrusion Detection Technology That Tracks Attack Flows Beyond Logs

Have you heard of P-IDS?

P-IDS stands for Provenance-based Intrusion Detection System. It refers to a system that represents security events occurring within a system as a provenance graph, or in other words, as a graph, and performs detection based on that structure. Over the past decade, provenance-based security research has steadily increased, with related studies continuously appearing at major security conferences such as USENIX Security, ACM CCS, NDSS, and IEEE S&P.

This naturally leads to the following question:

“Okay, I understand that it analyzes attack flows using graphs, but is it actually better than existing approaches?”

The 2023 ACM CCS paper, Are We There Yet? An Industrial Viewpoint on Provenance-based Endpoint Detection and Response Tools, investigated provenance-based EDR, also known as P-EDR, by surveying security professionals in the industry. Notably, all ten technical managers in the security field who participated in the interviews evaluated P-EDR as more effective than conventional EDR, suggesting that its effectiveness has already been recognized.

A provenance graph is a data structure that represents the origin and causal relationships of system activities in graph form. In this graph, nodes represent system objects such as processes, files, network sockets, and user accounts, while edges represent activities between them, such as execution, reading, writing, and communication.

For example, suppose a user executes a malicious binary, and that binary communicates with an external server to download a file.

In this case, explorer.exe, malware.exe, the external server, and the downloaded file each become nodes. The actions that occur between these nodes, such as execution, writing, and communication, become edges.

In this way, a provenance graph does not merely record “what event occurred.” Instead, it represents “where the event originated, which objects it affected, and what actions followed afterward.” Therefore, when a specific threat event occurs, a security analyst can trace the attack’s starting point, propagation path, and scope of impact.

This is especially important for APT attacks, which often remain dormant for a long period after initial compromise and later perform external communication or additional file downloads. Existing EDR solutions can also connect multiple events, but because detection systems cannot wait indefinitely for events to occur, it becomes difficult to determine that events separated by a long time interval belong to the same attack flow.

In contrast, P-IDS maintains causal relationships between processes, files, and network objects as a graph. Therefore, even if an action occurs much later, it can still be traced as part of the same attack flow if it originated from the same causal object.

## How Is the Research Evolving?

Early P-IDS research focused on representing system events as graphs and tracing attack paths using rule-based approaches. Since then, the field has moved beyond simple rule-based analysis toward learning complex attack patterns using deep learning and graph neural networks.

Studies such as ThreatTrace, FLASH, and KAIROS have attempted to detect attacks by learning both the structural characteristics of provenance graphs and semantic information such as process names, file paths, and command-line arguments. In particular, KAIROS aimed to reconstruct detected attack activities into a compact summary graph, making the results easier for analysts to understand.

Recent research also emphasizes practicality, not just detection performance. ORTHRUS highlights the importance of Quality of Attribution, meaning how easily analysts can identify the cause and impact scope of an attack. This is because some previous detection approaches identified broad areas as malicious and then claimed them as true positives, even though the results were not sufficiently useful for actual investigation.

CAPTAIN presents a direction for automatically adjusting detection parameters using gradient descent while preserving the lightweight nature, real-time performance, and interpretability of rule-based P-IDS.

More recently, the paper Sometimes Simpler Is Better compared eight state-of-the-art P-IDS systems and pointed out that complex GNN-based models are not always the best choice. In real-world security operations, not only detection performance but also real-time capability, low overhead, reproducible evaluation, and operational cost are important. VELOX shows that even a simpler neural network structure can achieve sufficiently high performance and may be more practical from a deployment perspective.

There are also emerging studies on synthesizing provenance graphs to address the lack of APT datasets, as well as research on robust P-IDS systems that can respond to situations where attackers manipulate graph structures to evade detection.

Ultimately, recent P-IDS research can be summarized into three major directions.

First, improving detection performance using GNNs and deep learning.

Second, reconstructing attack paths in a way that is easier for analysts to understand.

Third, achieving lightweight and real-time operation suitable for real-world environments.

Challenges That Still Need to Be Solved

Of course, P-IDS is not a complete technology that solves every problem. In fact, there is a clear reason why the industry recognizes the effectiveness of P-IDS but still hesitates to adopt it: operational cost.

Because provenance graphs represent relationships between system events in detail, the amount of data can become extremely large. If all events such as process creation, file access, network connections, and privilege changes are collected and stored as graphs, storage usage, memory consumption, and analysis time all increase. In real-time security monitoring environments, these costs become a significant issue.

The 2023 CCS paper also pointed out major bottlenecks in adopting P-EDR, including client-side overhead, an imbalance between alert classification cost and interpretation cost, and server-side memory usage. This shows that future P-IDS research must go beyond simply improving detection accuracy.

For P-IDS to become practical, reducing the graph is just as important as constructing it well. If unnecessary events are not removed from detection results, duplicate events are not merged, and only important nodes and edges for attack analysis are not selected, analyst fatigue will inevitably increase.

In the end, the core challenge of P-IDS is balancing accurate detection with practical operation. No matter how sophisticated a graph analysis model is, it will be difficult to use in real environments if it is too slow, consumes too much memory, or generates too many alerts for analysts.

Conclusion

P-IDS is an important approach that addresses the limitations of traditional intrusion detection methods. While conventional methods mainly evaluate threats based on individual events, P-IDS connects events through causal relationships and analyzes the flow of an attack.

This difference may seem simple, but it is highly significant. Real attacks do not occur as a single event. They occur as chains of connected actions. Therefore, future security systems must go beyond simply collecting large volumes of logs and begin understanding how those logs are connected.

The fact that all industry experts in the 2023 CCS paper evaluated P-EDR as more effective than conventional EDR shows that this direction is meaningful even in real-world security operations. At the same time, practical challenges such as client-side overhead, server-side memory usage, and analysis cost clearly remain.

Recent P-IDS research is rapidly evolving by combining GNNs, deep learning, graph representation learning, attack path reconstruction, rule optimization, and even LLM-based data synthesis. This indicates that P-IDS is not merely an academic idea, but a technology with strong potential to become a core component of next-generation EDR and security monitoring systems.