Analyze ALB Logs using Grafana and InfluxDb
At any point if you feel the article is bit lengthy then jump to ‘How to Analyze ALB Logs?’ section.
What is ALB (Skip this if you already know all about alb)?
The ALB (Application Load Balancer) is a part of the system called AWS (Amazon Web Services) provided by Amazon.com and is a load balancing service that distributes the load generated by web services.
In recent years, due to the spread of social networking services and other factors, there has been an increase in the sudden concentration of access to web applications.
A sudden spike in access will cause the web service to slow down and cause errors.
A load balancer like ALB distributes the load on such web services and improves their stability and high availability.
With the many features of the ALB, you will be able to run your web services continuously and effectively.
There are many benefits to AWS services, but the following are particularly beneficial for ALB.
・ Supports high availability
・ Security, such as certificate management and user authentication.
・ Flexible response to application load at various levels
・ Close monitoring and auditing of applications
These benefits can lead to increased efficiency and practicality in the operation of complex web services.
The benefits will be even greater in the future, as every business can be improved and new work can be taken forward.
How To Enable ALB Logs (Skip this if you already enabled logs)?
Elastic Load Balancing provides access logs that capture detailed information about requests sent to your load balancer. Each log contains information such as the time the request was received, the client’s IP address, latencies, request paths, and server responses. You can use these access logs to analyze traffic patterns and troubleshoot issues.
Access logging is an optional feature of Elastic Load Balancing that is disabled by default. After you enable access logging for your load balancer, Elastic Load Balancing captures the logs and stores them in the Amazon S3 bucket that you specify as compressed files. You can disable access logging at any time.
There is no additional charge for access logs. You are charged storage costs for Amazon S3, but not charged for the bandwidth used by Elastic Load Balancing to send log files to Amazon S3.
To enable access logs follow steps mentioned here: https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html#enable-access-logging
How to Analyze ALB Logs?
The access log files are compressed. If you open the files using the Amazon S3 console, they are uncompressed and the information is displayed. If you download the files, you must uncompress them to view the information.
AWS Athena (Skip this if you are not interested in Athena)
AWS suggests using AWS Athena to process ALB logs. Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL.
https://docs.aws.amazon.com/athena/latest/ug/application-load-balancer-logs.html
Using Athena it is possible find data points such as most requested path, max response time, user agent and request count.
Athena’s SQL can help filter data based on request time but it is difficult to visualize data based on request time. For an example if you want to know a request pattern for a specific path at different time frame in day, then it becomes a very difficult task.
InfluxDB and Grafana.
InfluxDb: InfluxDB is an open-source time series database developed by the company InfluxData. It is written in the Go programming language for storage and retrieval of time series data in fields such as operations monitoring, application metrics, Internet of Things sensor data, and real-time analytics.
Grafana: Grafana is a multi-platform open source analytics and interactive visualization web application. It provides charts, graphs, and alerts for the web when connected to supported data sources.
Athena can be used to analyze very specific data point while the other option could be to download alb logs, parse each log files and push log data to Influxdb and plot required charts.
For an example:
If you want to know the mean response time or request count for different path during the day at different intervals.
Checkout this git repository. It allows to parse logs directory and push logs to Influxdb. A sample grafana dashboard is also included in the repository to import and get preconfigured dashboard for ALB.
This can be modified to work as a lambda function, which will get triggered on S3 write event for logs S3 bucket and can provide a near realtime alb data analysis. Grafana can also be used to trigger alerts on slack or email for real time notification.
Implementation:
- Clone git repository
- Install required python lib using pip
- Download ALB logs using AWS cli.
- Process logs
python main.py -d ./logs/ - Import Grafana dashboard using json.
- Create you own dashboard for the required files.
