A typical enterprise system usually runs on multiple servers behind a load balancer. The simplest logging option could be logging locally to a location on the server where the app runs. However, this setup makes triaging an issue using the logs difficult. The problem is more prominent if the system consists of multiple applications running on different servers, and the sessions are not sticky. When an error occurs, the developer has to go to the different servers and applications to gather the logs. Even when the developer has gathered all the logs, it may still be challenging to piece together the log events that belong to a same session in a chronological order to troubleshoot the issue.
In the past, I faced this problem with the applications I worked on. When troubleshooting an issue, I had to download the logs from multiple servers. Then, I used grep to search for keywords relating to the error. The process is time consuming, and I often got confused when I could not tell whether a log statement related to the error because of asynchronous events.
Things got much better after I integrated the apps with application insights which has powerful features for monitoring and logging.
In application insight, I can search for telemetry events including logs by entering keyword in the search bar. I can also filter by event types. For example, below screenshot shows all the exceptions that occurred in the last 24 hours.
