An Efficient Log Management System

http://repository.vnu.edu.vn/handle/VNU_123/56824

Server monitoring is really necessary because this can help administrators to track users’ activities in order to improve user management ability, load balancing as well as to detect DDoS attacks. 

Usually, server monitoring is based on logging.

However, logging system is always considered to be expensive in term of storage, data collection, data searching and analysis.

In this paper, we introduce eLMS, an efficient and scalable log management system.

In our system, the log files can be collected from multiple servers, stored in a scalable manner, appropriately indexed and fast analysed. eLMS acts both in online and off-line modes and provides a practical web-based monitoring interface. eLMS is based on a famous open-source core called ELK including three main modules: ElasticSearch, LogStash and Kibana.

However, by using queue and streaming technique instead of uploading technique used in LogStash, eLMS is tested to be at least ten times faster than existing ELK

Title:	An Efficient Log Management System
Authors:	Nguyen, Van Nam Tran, Van Cuong
Keywords:	Log Files;Big Data;Streaming;Index and Search.
Issue Date:	2016
Publisher:	ĐHQGHN
Series/Report no.:	Vol. 32, No. 2 (2016);
Abstract:	Server monitoring is really necessary because this can help administrators to track users’ activities in order to improve user management ability, load balancing as well as to detect DDoS attacks. Usually, server monitoring is based on logging. However, logging system is always considered to be expensive in term of storage, data collection, data searching and analysis. In this paper, we introduce eLMS, an efficient and scalable log management system. In our system, the log files can be collected from multiple servers, stored in a scalable manner, appropriately indexed and fast analysed. eLMS acts both in online and off-line modes and provides a practical web-based monitoring interface. eLMS is based on a famous open-source core called ELK including three main modules: ElasticSearch, LogStash and Kibana. However, by using queue and streaming technique instead of uploading technique used in LogStash, eLMS is tested to be at least ten times faster than existing ELK
Description:	p. 43-48
URI:	http://repository.vnu.edu.vn/handle/VNU_123/56824
ISSN:	0866-8612
Appears in Collections:	Computer Science and Communication Engineering