Practical Multi-pattern Matching Approach for Fast and Scalable Log Abstraction
| Authors | |
|---|---|
| Year of publication | 2016 |
| Type | Article in Proceedings |
| Conference | ICSOFT-EA 2016 - Proceedings of the 11th International Joint Conference on Software Technologies |
| MU Faculty or unit | |
| Citation | |
| Web | http://www.scitepress.org/DigitalLibrary/PublicationsDetail.aspx?ID=SFHlGUrQnPo=&t=1 |
| Doi | http://dx.doi.org/10.5220/0006006603190329 |
| Field | Informatics |
| Keywords | Log Processing; Pattern Matching; Log Abstraction; Big Data |
| Description | Log abstraction, i.e. the separation of static and dynamic part of log message, is becoming an indispensable task when processing logs generated by large enterprise systems and networks. In practice, the log message types are described via regex matching patterns that are in turn used to actually facilitate the abstraction process. Although the area of multi-regex matching is well studied, there is a lack of suitable practical implementations available for common programming languages. In this paper we present an alternative approach to multi-pattern matching for the purposes of log abstraction that is based on a trie-like data structure we refer to as regex trie. REtrie is easy to implement and the real world experiments show its scalability and good performance even for thousands of matching patterns. |