Virtual Knowledge Graphs for Federated Log Analysis

Security professionals rely extensively on log data to monitor IT infrastructures and investigate potentially malicious activities. Existing systems support these tasks by collecting log messages in a database, from where log events can be queried and correlated. Such centralized approaches are typically based on a relational model and store log messages as plain text, which offers limited flexibility for the representation of heterogeneous log events and the connections between them. A knowledge graph representation can overcome such limitations and enable graph pattern-based log analysis, leveraging semantic relationships between objects that appear in heterogeneous log streams. In this paper, we present a method to dynamically construct such log knowledge graphs at query time, i.e., without a priori parsing, aggregation, processing, and materialization of log data. Specifically, we propose a method that – for a given query formulated in SPARQL – dynamically constructs a virtual log knowledge graph directly from heterogeneous raw log files across multiple hosts and contextualizes the result with internal and external background knowledge. We evaluate the approach across multiple heterogeneous log sources and machines and see encouraging results that indicate that the approach is viable and facilitates ad-hoc graph-analytic queries in federated settings.