A Study on Log Anomaly Detection using Deep Learning Techniques

A Study on Log Anomaly Detection using Deep Learning Techniques

Kamiya Pithode

Abstract—Anomaly detection is critical in the administration of current large-scale networked systems. Logs created by security systems, servers, network tools, and different software applications are some methods of recording the operational behavior of the equipment or software. These logs are useful for obtaining useful data about system activity. Deep Learning (DL) has lately established the lead of performance in detecting intrusions, denial-of-service attacks, malware, hardware & software system failures. A research background is discussed about feature extraction, machine learning, and deep learning. It also discussed the challenges in log analysis such as unstructured data, instability, log burst, and availability of public datasets. Recurrent Neural Network (RNN) language standards reinforced with awareness to anomaly detection in system logs. The goal of the research is to make an overview of the current study on log anomaly detection utilizing Deep Neural Networks. The research is also providing a summary of log parsing algorithms, datasets utilized for log analysis, and different ideas offered for log anomaly recognition. However, there is comprehensive comparison of representative log-based anomaly detectors that apply DL.

Keywords: Log Anomaly, Feature Extraction, Deep Learning, Recurrent Neural Network, Log burst.