

Source

One of the task during my work on AppSensor .NET library was to create a simple implementation of log provider. It was my inspiration to make some research about rules how to create a good logging system.

Think about the purpose

The first thing that is necessary before creating or even configuring is to think what is the purpose of logging. When we might need this information and how it can be useful for future application use. The main reasons to set up logging are as below:

errors – we should be aware of all errors appearing in our system, in front-end and back-end side,

– we should be aware of all errors appearing in our system, in front-end and back-end side, security – diagnostic information related to access control or authorization can led us to detection of potential attacks,

– diagnostic information related to access control or authorization can led us to detection of potential attacks, info about problems/unusual conditions – when the system detect some unusual action or it goes to state that might need administration action (i.e. lack of disk space) log can be the source of more detailed information,

control process flow – it also can provide an information about a process progress and how its behave.

Stores

Next decision to take is where we want to store log data. We have several possibilities to choose:

files on disk – very fast and easy, but not suitable for distributed systems. Also hard to query information within it,

– very fast and easy, but not suitable for distributed systems. Also hard to query information within it, database – very useful, easy to query, good for distributed or load balanced systems, little more difficult for configure and need a DB server,

– very useful, easy to query, good for distributed or load balanced systems, little more difficult for configure and need a DB server, external providers (Loggly, Splunk …) – easy to set up, very convenient for querying and reporting, slow because sent over the Internet.

Logging levels

In typical application we want to log many different type of events. If we are talking about the web systems, the volume of users being on-line simultaneously can be very large. From this point of view, the number of logs can be tremendous. That’s why it is so important to set up good levels of detail during this process. Usually there are following levels:

Fatal – Highest level: important stuff down

Error – For example application crashes / exceptions.

Warn – Incorrect behaviour, but the application can continue

Info – Normal behaviour like mail sent, user updated profile etc.

Debug – Executed queries, user authenticated, session expired

Trace – Begin method X, end method X etc. According to: https://github.com/NLog/NLog/wiki/Log-levels

Good configuration of log levels gives us a parameter to set details level dynamically depending on our needs. More specific logs can give more information, but it takes much more disk space.

Information scope

Depending on the logging purpose we should choose a set of properties which we want to store for each log message. It can be divided in following groups and all 4 information group should be included:

when the situations occurs,

the situations occurs, where in code,

in code, who caused this situation,

caused this situation, what happened.

Logging scope

The last important part to consider during log configuration is when we want to save a log. Of course it depends mostly on application, but we can define some general rules what we should log:

all exceptions ,

, validation failures ,

, authentication success and fail,

authorization failures ,

, system events (startup, closing, changing date),

use of high-risk functionality (closing operational day).

Of course, logging too much information can be more harmful than helpful. That’s why you have to be sure that you don’t log any of information listed below:

duplicated failures ,

, regular data changes,

fragile information (security of personal data , passwords ),

(security of , ), application source code.

Resources

https://www.owasp.org/index.php/Logging_Cheat_Sheet

https://www.owasp.org/index.php/OWASP_Security_Logging_Project

https://developer.atlassian.com/confdev/development-resources/confluence-architecture/logging-guidelines