The concept of logging is nothing new. It is commonly used for cross-cutting concerns in an application and has been in use for a long time. It is an essential component of any application – be it a monolithic or a microservice-based one.
This article presents a discussion on the challenges faced when implementing logging in a microservices-based application and the solutions. It discusses why logging is important in a microservices-based application, log searches, and how you can store useful, meaningful data and work with centralized logs in a microservices program.
What is Microservices Architecture?
Microservices architecture can be described as a variant of service-oriented architecture that has been embraced by many organizations worldwide in recent times. It is an architectural paradigm for building autonomous, flexible, extensible, independently deployable, and fine-grained services.
Read: An Introduction to Microservices.
Why do we need Logging in Microservices?
A microservices-based application might have several services running and communicating amongst themselves. Things get complicated when one or more services fail, and you need to know which one failed and why. It’s also essential that you’re able to comprehend the whole request flow — which services were invoked, how many times, and in what order.
To answer these questions, you should have meaningful logs available and a unique Id to correlate requests. You should have a proper way to track errors that might span several microservices. Moreover, the complexities of monitoring and logging grow exponentially when the business logic of your applications spans across several microservices.
Logging in a monolith is as easy as publishing data to a single log file and retrieving it later. When working with microservices-based applications that have logging capabilities, you should be able to address some unique challenges.
Read: Microservices Monitoring and Observability.
Structure your Logs Appropriately
A typical microservices-based application might include several technology stacks. This might impact the structure of your log since the structure used to log data might differ. You might have one service using a pipe “|” symbol as a delimiter for the fields while another service might be using a comma “,” as a delimiter. You should structure your log data to make parsing simple and log only the information that is relevant. Structured logging will help you to make your logs simple, meaningful, and flexible. A dashboard would be a nice to have feature. You may want to use the dashboard to visualize the logs graphically.
Use a Centralized Log Storage
When working with microservices-based applications, you might have several different services spread across different servers. It would be a daunting task if you were to implement logging in each microservice (each of the microservices would need to implement their own logging mechanism) and then to read the log data when an event has occurred. The downside to this approach is that it is redundant and adds a lot of complexity.
Moreover, when the number of microservices grows, the problem would become even worse. Hence, it is imperative that the logs are sent to a centralized location so that the logs can be accessed in one place. Once you have the logs in one place, it is easier to correlate problems.
Read: Security Challenges and Solutions for Microservices Architecture.
Your Logs Should be Queryable
Your logging technology should support querying the logs, i.e., your logs should be searchable. Using the correlation Id, you should be able to get the complete request flow. You may want to query the log data to determine the percentage of requests that have been unsuccessful, the time it takes to complete each request, the frequency with which each service call is made, and so forth. You can take advantage of a tool to aggregate your log data as well. As an example, you may use ELK Stack.
Your Logs Should Provide Sufficient Information
When an error occurs, you should be able to record as much information as possible about the problem in your logs. This would help you to ascertain what went wrong quickly. Hence troubleshooting problems will be easy. You shouldn’t use local time in your log entries – always use UTC time. The information you capture in your logs should, at a minimum, comprise the following data:
- Name of the Service
- Name of the logged-in User
- IP Address
- Correlation Id
- Time at which the message arrived
- Time Taken
- Name of the method
- Call Stack
- HTTP Code
Use a Correlation ID
Assume that there are three services X, Y, and Z where service X calls service Y which in turn calls service Z. Now suppose there is an error in service Y. How do we know which service failed and during which calls? You have the logs but what if your log contains millions of log messages? A correlation ID comes to the rescue precisely at this point.
A correlation ID can be defined as a unique identification number used to correlate a set of operations and monitor or track all individual requests. It doesn’t matter how you would like to generate the correlation ID if it is unique. All you need to do is generate a correlation ID, pass it to the downstream services and log the correlation Id along with other meaningful data across the service calls.
When you have a correlation ID, you can search for it in the logs to track the service calls. You can take advantage of correlation IDs to know the number of times a service was called and much other information.
Read: Scaling Microservices Architecture Using Caching.
Handle Failures
You should also have an automatic alert system in place, which can analyze logs and send out notifications whenever anything goes wrong with one or more services in your application. You should also consider failures – your logging component might be down. You should have a fallback service that is adept at handling logging failures and restoring log data if need be.
Logging Microservices Checklist
Let’s take a quick tour of the points we discussed thus far. Here’s the list of the best practices in logging microservices-based applications:
- Log meaningful, structured data
- Provide enough information in your logs
- Use a correlation Id
- Use centralized logging
- Make sure that your logs can be queried
- Use an automated alert system
- Consider failures
Microservices Architecture Logging Tutorial
Depending on your application you might want to implement logging in each service or logging in the API Gateway. Both these approaches have their pros and cons. There are several log aggregation and management tools such as Fluentd, Loggly, Rsyslog, and Logstash that you can leverage to provide real-time insights into your application’s performance.
In this article, we’ve had a discussion on why logging in microservices-based applications is important, the challenges involved, and their solutions.
Logging is important in a microservices-based application, but you should be aware of the challenges, the solutions, and the best practices. A good practice is to take advantage of asynchronous programming to write the logs – by adding asynchronous log appenders so that the request thread is not blocked.
